When “who were running” is in commas, it's a nonrestrictive element--an aside. It's describing the monks, which means every monk was running. You can leave it out and it doesn't change the core meaning of the sentence.

When “who were running” isn't in commas, it's a restrictive element -- it modifies the noun “monks.” Only some of the monks were running, and it was those running monks who jumped aside. Leaving it out would leave out important information about what happened.

Buy Grammar Girl's latest book, 101 Words to Sound Smart and never be at a loss for words!