Looking at your above traceroute, the problem is occuring around steps 4 and 5, before and as your packets leave NZ. It could be a contended international link and you're probably experiencing latency and throughput issues to other Australian sites as well.

Unfortunately, theres nothing to be done from the ozfortress/Infinite side, this is an issue for you to take up with ihug (or your isp).