Postfix relay server // origin IPs getting blacklisted

Good morning, we've got a postfix relay server (ubuntu 10.04) with lots of clients relaying mail through it. It's not particularly high volume, we don't allow large mail shots.
Recently I've noticed some mail getting bounced due to blacklisting (particularly from barracuda) However it's not our servers IP that's getting blacklisted, it's the IP that our clients are sending from.
Sometimes they're blacklisted because the IP isn't static and also some domestic dsl ip ranges get blacklisted.
The thing that puzzles me is that surely the servers at the other end should be looking at our relay servers IP and not the ultimate origin?