This must have been changed somewhere along the development path - I vaguely remember testing this some (long) time ago and mages behind troops moved but didn't cast then.

However, if you don't want your water mage to move up to the nearest, rearmost squad, simply set him to "cast spells", and he'll stay put.

Whoa, they actually do cast? I'm with you I would have bet 100% that they would move and NOT cast. If they do follow the troops and cast tho... that could provide a niche for your weak mages. If they can only manage the close range (5-10) evoc spells you could set them to follow and they could get up close to the enemy and blast them. Hmm...

Exactly. Except for their tendency to senselessly cast BoW and then kill each other with it, this is a very handy tactical tool.

Also it seems that when you put them on 'stay behind troops' they will fire ranged weapons on and off, keeping their fatigue down and their spellcasting going longer, but on 'cast spells' they ignore their ranged weapons and just cast until they pass out.

Testing reveals that switching the final order from 'stay behind troops' to 'cast spells' or, for that matter, to null, does not alter the behaviour. I still get mages running across the battlefield to get close to other mages, archers, or illithids and casting BoW. There is some variation, though, sometimes cast BoW THEN run to where it can inflict the most casualties on their own side, sometimes run to position first, THEN cast BoW.

*sigh*

I don't know which is dumber, the mage doing that in the first place, or the other units inside the area of effect simply standing there until they die from it, but either way, there's a stupidity overload going on for sure.

There are certain limitations to the game engine atm - your perceived 'stupidity overload' is the result of, sorry to says that, your ignorance towards them:

1) Commanders do what they are ordered to - literally. They do not 'think about' what might be your intentions. The only freedom of choice is what spell to use when ordered to 'cast'. In that case, the AI will mostly go for buff (target:self) spells first. If there are targets in range, the battle engine will try out several different ranged spells and choose the one which deals the most damage. That does not mean it will do maxed damage, or any damage at all, because the random factors are newly calculated when the spell is actually cast.

2) Units do not 'know' about battlefield hazards. They move into poison clouds as well as into a breath of winter. Likewise, units do not 'know' about their own auras/fields.

Ok, another quick test:

A mage who is ordered to "cast spells", but can cast absolutely no spell defaults to "stay behind troops" it seems. (E.g. Blood-1 mage without slaves and low research - only spell would be 'bleed' which requires a slave).

"sometimes cast BoW THEN run to where it can inflict the most casualties"

Obviously, the only spell the mage could cast at that time was BoW. As he was ordered to cast, he did just that. Afterwards, he was ordered (by the game engine) to "stay behind troops" - and he did just that.

"sometimes run to position first, THEN cast BoW"

Behaviour like that I couldn't recreate unless I ordered the mage to move via "attack" order. If ordered to cast a specific spell 4x and then cast spells, the mage always stays put until running completely out of spells. In that case, he casts "BoW" first and moves on the next turn.
Exception might be heroically quick or quickened mages. The additional action points screw up the order of orders (if that makes sense).

"to where it can inflict the most casualties on their own side"

As unit do not know about auras, fields, units around them (other than who is 'rearmost', 'nearest', 'archer' etc), they can't deliberatly run where they can cause the most casualties. They go for the rearmost unit. Simply make sure your rearmost unit is 100% cold-resistant and keeps a healthy distance from the rest of your troops and you'll be fine. If you bunch up all your vulnerable mages, either by setup or use of "stay behind troops", the 'stupidity overload' is surely not on the games side

Admittedly, there are some possibilities for improvement with the battle engine as well as with the game engine in general

__________________
As for AI the most effective work around to this problem so far is to simply use an American instead, they tend to put up a bit more of a fight than your average Artificial Idiot.... James McGuigan on rec.games.computer.stars somewhen back in 1998 ...