Gandalf's power limitations are mentioned several times (or at least alluded to) throughout the trilogy.

Forgive me for not finding quotes and paraphrasing:

When he was preparing to rescue the Dwarves from the Misty Mountain Goblins, he mentioned that he spent some TIME conjuring up his magic. In order for it to be as powerful as it was, he needed to actually focus and prepare it.

In the fight with the Wargs in the treetops, he lit pine cones on fire and tossed... note, he did not send flaming streams or fireballs down at them. He had to have something physical to work with... something that would actually burn. This was further evidenced when the Fellowship was trudging up Caradhras. He told Legolas (I think) that he couldn't light snow on fire, he needed dry wood in order for his conjuring to work.

When faced with the Balrog on the outside of the chamber of Mazarbul, after the balrog shattered his spell, I believe he mentioned that he didn't have the TIME to conjure a more powerful spell.

Those are just a couple of examples. His magic with the Trolls was deceptive and not really offensive. His fight with the Wargs in The Fellowship showed a bit more power, but again, I think he had a little more time to prepare as they knew (from the howls) that the Warg pack was approaching.

As Gandalf the White, I got the feeling that he was more powerful than Gandalf the Grey and that his magic was enhanced. So, anything he did in that form, I think shouldn't be counted in the argument.

So, using those examples as precedent, being attacked unprepared by a very physically powerful and imposing orc, I don't feel it's too outlandish to have Gandalf be on his heels for a bit. He was never shown in the books having a one-on-one physical battle with another character, so I think it will be interesting and new (to the audience) to see that Gandalf IS vulnerable, physically.