Vid card fan going out

Hey, the little POS fan on my Diamond Viper V770U (TNT2 Ultra) is starting to make a serious racket. This card is in my server, and never gets pushed into anything beyond 2D Windows apps. There is also a pretty decent amount of airflow in the case, with a side blowing fan directly over the AGP/CPU area of the motherboard. You think it'll hurt it if I just disconnect the fan plug?