Allergy? Sounds like two possibilities - one is you are lactose intolerant - the sugar in milk requires any enzyme to digest it and some people have a low level or lack it and bloat as a reult. Use of lactaid or similar will help with this. The second issue is actually an allergy to the milk protein. If you stay off of it for a month or two and then occasionally add it back, see if you tolerate it.