Simple question: I remember hearing somethere that when one machine exceeds a "number" of simutanious TCP connections all sorts of problems occur. I am asking this question specifically about .net's sockets (which are pretty much wrappers for winsock).

Generally, no such thing as "all kinds of problems occur" is happening.

You can have a maximum of 65536 TCP connections with each other machine that has a distinct IP address (or a total of 65536 connections with a group of machines behind NAT) because the port number is 16 bits and a connection is identified by address and port. It is very unlikely that you will ever run into this limit.

Some operating systems impose per-process and per-user quotas on how many descriptors you can create. Again, you're rather unlikely to run into trouble with this limit.

The only real difficulty with having many ("many" as in 5,000 or 10,000) connections is how to actually handle them, that is how to schedule sending and receiving your data so nobody is unfairly stalled and you don't spend excessive work figuring out which socket has data, and which socket is ready to send, etc.
Since you said .NET and Winsock, you will most likely be interested in looking into completion ports for that matter. Handling 10k connections is no challenge for a completion port.

You can have a maximum of 65536 TCP connections with each other machine that has a distinct IP address (or a total of 65536 connections with a group of machines behind NAT) because the port number is 16 bits and a connection is identified by address and port. It is very unlikely that you will ever run into this limit.

Actually, that is not completely correct. A connection is identified by the complete tuple (local ip, local port, remote ip, remote port), so you could have (theoretically) many more distinct connections between two machines or machines behind a NAT.