Peter Singer - Famine, Affluence and Morality

In Peter Singer's view he feels that everyone should live within "MARGINAL UTILITY". I do not feel that this is feesible. He does not take into account that the people he wants to give up their money have earned it or he hasn't yet set any standards for what is Marginal Utility, from a rich person to a middle class person to a poor. I am looking for help in gathering thoughts and views on why this marginal utility view is wrong.