In fact, the code in question remains all the way 'til today...
I quickly tried to reproduce this using FC3 Test 3 on an x86_64 w/ a
3c905B card, but it was successful.
Can you attach the output of running sysreport on the listener? Do
you have any more specific recreation instructions?
I'll attach some information from my test in the next comment. I'll
also install RHEL3 on this box and retry...

OK, I am seeing the "eth0: memory shortage" messages under RHEL3 U4...
There is a patch pending that upgrades the RHEL3 3c59x to be almost
identical to the RHEL4 version of the driver. It is associated w/ bug
133843 and will probably be in RHEL3 U5 (it was too late for U4). I
will try that and see if the messages disappear...

I am still looking at this (although I will be out of town for the
rest of this week)...
The "eth0: memory shortage" message comes from the 3c59x driver when
he starts to fail at allocating memory for incoming packets.
Presumably your netcat sessions time-out when the delays caused by the
memory shortage get big enough to kill the session.
I repeated the test w/ output redirected to /dev/null. I did not get
any of the "eth0: memory shortage" messages, and the test completed
about 10% faster. To me, this suggested that at least some of the
congestion is caused by writing to disk.
Are there any netcat options (perhaps -i and/or -w) that can improve
netcat's tolerance?
Is it possible you could use a receiver system w/ a faster disk subsystem?
Please note that I never saw netcat fail. My sender and receiver are
very "close" on the network. Could you improve the network between
the sender and the receiver?
I have a preliminary NAPI patch for the 3c59x driver. I didn't see
much difference with it, but it may be worth testing? I'll attach it
in case you are feeling frisky... :-) It is against RHEL3 U4, BTW...

Got the patch, but not U4. Will it work against the U3 source?
I've been using netcat with '-w 3'. Maybe adding a few more seconds
will keep it from timing out. Haven't tried the '-i' option before.
It'd be rather hard to find a faster disk system. I invested quite a
lot of money to make this one very big and very fast :-). It has nine
10K rpm U320 drives in a hardare RAID5 array on a U160 channel. The
sender is a measley 5400 rpm EIDE notebook drive.
Like yours, my sender and receiver are also close -- on the same desk
with the Fast Etherswitch. Total patch cable length is about 10 feet.

Using the 'i' option slows things down to an impossible crawl.
Increasing the wait time interval to '-w 30' changes nothing. The
transfer still fails at exactly the same place in two attempts. This
is too precise to be explained by ring buffer exhaustion.
Despite the error message we're seeing, I'm now skeptical of the buggy
driver hypothesis. I now suspect we're seeing the end result of
something else entirely, but haven't a clue what it might be. What's
so special about a repeating file size of 9,883,033,600 bytes on a
disk with 142,794 logical 8,225,280 byte cylinders??

I re-ran the transfer sequence after first slowing the source NIC down
to 10Mbps. This time the transfer took more than 12 hours rather than
22 minutes, but the listener again aborted when the received file size
reached 9,883,033,600 bytes. There were no eth0:... errors in the log
file. The exact error reported on the source end was:
# dd if=/dev/hda5 bs=2048 | nc 192.168.1.2 30000 -w 3
dd: reading `/dev/hda5': Input/output error
4825700+0 records in
4825700+0 records out
After getting the same result after slowing the transfer rate by 90%,
I think it's unlikely that we're seeing the result of overdriving the
listener's NIC or the large drive array. It must be something that's
speed-independent, like an overflowing counter. In nc? In dd? Where?

Strange, as I was always able to complete my transfers. Hmmm...
Have you tried this?
dd if=/dev/hda5 bs=2048 > /dev/null
I'd be curious to know if the results are different. Have you tried
the test w/ a different sender?

Sorry to be so tardy getting back to you, John. Your suggestion to
pipe to /dev/null ultimately lead me to the answer: A consecutive
string of 160 bad sectors had formed on the source drive. It was
killing every dd transfer with an input/output error (at the same
place, of course). A friend turned me on to dd_rescue which has the
ability to survive those errors and keep imaging. (Red Hat should
consider adding dd_rescue to the RHEL bag of tools.)
Certainly those "eth0: memory shortage errors" still need to be
tracked down and fixed, but in the end they turned out to be a red
herring and not the cause of this problem.
I'm very sorry to have wasted your time on this wild goose chase.