Well, don't know if this is the total cause, but the market and Gvoice notifications seem to be using port 5228. As soon as I opened that in my firewall, I receive GV notivications, and can download from the market over wifi.
Gmail notifications seem to use it too.