As Adam said, the only performance issue caused by defragging an almost-unfragmented drive is the time it takes to run the defrag. But note that the less fragmented a drive is, the quicker the defrag will run.