OK, I am suddenly, for no apparent reason, reminded of a game for the
IBM PC called Romance of the Three Kingdoms. This is a game that involved
larger goals than Olympia, as you had generals, and could kill other
generals, and were in charge of land, as opposed to in charge of men.
The similarities are as follows:

Each region had a number (Peasant Loyalty) which was what I consider to
be similar to Loyalty and O.Goss' happiness factor. I found it quite
useful to keep peasant loyalty up, and not too hard to do (however, that
game is less balanced than Olympia, as I see it). But I like the idea.
It worked. it was a way to get rid of my rice in RTK, although I usually
had too much anyway :)

Anyway, my point is that the loyalty went down fairly quickly. In general,
pesants don't like to be taxed. But there were Generals, who were single
men, who worried about their own men seperately than you. You gave them
gifts of stuff, and they got really nice to you, represented as Loyalty.
If you got a general to 100% loyalty, he would not defect, no matter what.
However, there were ways to lower that loyalty to get the general away from
the owner. The general's loyalty didn't change unless someone tried to
change it. SO, maybe I can come up with a compromise.

Have a formula based on unit size that indicates the reduction of loyalty.

A large unit of unskilled idiots with no money will in general feel like
they are getting the shaft.

a large unit of skilled idiots (say, one skill at level 1-5 or so) will
feel like they owe the faction owner something. Not much. just something.

a large unit of skilled geniuses (say, several skills at 1-5, or one or two
skills at 5-10) will be very pleased with the owner of the faction and not
desert any time soon.

a small unit of unskilled idiots with no money will feel just like the large
unit.

a small unit of skilled idiots will probably feel like they are elite, even
though they are idiots. little loyalty loss

a small unit of skilled geniuses are elite, and feel gratitude to the faction
owner and would have VERY little loytalty loss.