Forget about nouveau, it's missing too much features to be usable for gaming.

Use the proprietary nvidia driver. As for the card, depends on your budget - Kepler cards go from the extremely cheap (but not really usable for gaming) 610 to the uber expensive monster that is 680. I'd go for the middle ground, either 650 or 660.

Thanks Gusar's suggestion.
Proprietary driver is ok for me too, but it will be the last choice.

Ant P,
Is radeon open driver in active development? I see most of the articles on the web are about nouveau.

radeon doesn't support gallium3d, does it?
I heard that if wine implements directx using gallium3d (has wine already done that?), the performance will be at the same level of windows.
it is the main reason why I think NV card is better.

Wine does not use gallium. In fact, the directx tracker has been removed from it, because no one used it and it suffered bitrot. At least I recall reading that, but I'd have to check. In any case what is definitely true is that wine doesn't use it and there were never plans to use it.

radeon is in active development, but it's in no better position than nouveau: like nouveau, it has poor performance and lacks proper power management. And support for Southern Islands (high end HD7xxx) cards isn't finished yet, so those don't have any 3d accel yet.

Especially if you plan to use wine, Nvidia is a better choice. Historically that's what wine devs used for their development. This has changed, the wine devs now do care a lot more about radeon too, but Nvidia is still in a better position.

Be aware as well (if you are trying to use free drivers only) that Radeon cards require use of non-free (closed-source) binary blob firmware from the Linux kernel. This is a dirty little secret often overlooked. If you build your gentoo-sources kernel with the "deblob" USE flag this firmware is stripped out, and you will lose acceleration.

I'm working on trying to get 3D acceleration on my system using only the noveau drivers, but I've had trouble because I'm on a low budget and can't buy newer cards. You can check these links for information on 3d support for specific cards:

My personal experience has been that the Nvidia 6600 GT and everything older I have don't work correctly with 3D games under Gnu/Linux. (Screen gets fragmented and eventually the program crashes.) Unfortunately I haven't been able to test anything newer. Going by the chart in the feature matrix you likely want something from the NV50 column, such as the GeForce 8800.

I don't know anything about Wine or how it integrates with the Linux graphics systems._________________frigidcode.com

You need ucode blobs for most wireless cards too, I don't see why this should be a big deal. Especially when there's real issues with radeon, like lack of proper power management and low performance that might not make it possible to play modern games.