I had the same issue. I built OpenSSH from source and still had the problem. I installed the latest updates and the problem is gone. I don't know which of the updates fixed it - none of them seemed obviously related to ssh. Here are the versions of OpenSSH & the updates that "fixed" the issue:

My good luck went away after the next update, though it's not clear what the update had to do with it. Unfortunately, I don't have time to dig into it now. Hopefully, someone with more expertise in OpenSSH can help.

I have the same problem. I've tried several different things and I'm hoping maybe someone else can relate or better yet, know of a solution. I'm not able to ssh to machines but I'm able to do so to local machine on my local network. Also, I've noticed that when I'm not using wireless and a direct wired connection instead, I can ssh just fine anywhere. I've tried uninstalling and reinstalling openssh but that didn't help. This is happening on my laptop which is running Kubuntu. I've upgraded a desktop machine using a wireless interface running Ubuntu and there aren't any problems with it.

For me it works fine with wired connection but not with wireless. I tried also putty client + wireless and that also works fine.
So the problem is with the openssh-client and the wireless connection.
Not sure if a workaround is needed in the openssh package or the network level.
I did not try the original openssh-client package as shipped with Hardy to see if that is ok.

Gabriel's post of his environment kind of struck me with the answer. I also have a BCM4310 USB Controller and was using NDISWrapper before upgrading to Hardy. I noticed after the upgrade that I was using the "wl" alternate driver and it was showing up as one of my "Restricted Drivers" and before it wasn't there. I followed the instructions here to restore my wireless to use the previous driver:

and voila! I can now ssh and scp using my wireless interface. It might have been possible to simply uncheck the enabled box in the restricted drivers application and restart but I know for sure the above link fixed my problems. I hope this helps someone.

I replace also the in-kernel driver with ndiswrapper and now everything looks fine.
I checked also the page regarding the kernel driver http://linuxwireless.org/en/users/Drivers/b43#supported and the model I have (BCM 4310 Rev 1) does not appear in the supported list. Strangely it worked (sort of).

I was going nuts trying to figure out if it was a problem with my Dlink wireless router, openssh-server, or openssh-client. Finally, I tried putty, which worked perfectly. That helped me google down the problem to this thread.

reassigning package as this seems to be a bug caused by the 'wl' driver. Could some people provide feedback of which version of LRM and which kernel was being used at the time? There are two versions of 'wl' out there right now.

it seems this might be fixed in the regular ubuntu repositories, but the dell mini repositories still haven't been updated? the bug referred to by boyan states this to be the case... hope it gets updated in the dell mini repos soon!

Today, the same problem is appearing on all of our Amazon Cloud Servers. We are using 4 different client machines, wireless or not. It's not related to drivers. I know SSH does reverse lookup by default. Can it be the issue ?