With quick development of digital images and the availability of imaging tools, massive amounts of images are created. Therefore, efficient management and suitable retrieval, especially by computers, is one of themost challenging fields in image processing. Automatic image annotation (AIA) or refers to attaching words, keywords or comments to an image or to a selected part of it. In this paper, we propose a novel image annotation algorithm based on neighbor voting which uses fuzzy system. The performance of the model depends on selecting the right neighbors and a fuzzy system with the right combination of features it offers.Experimental results on Corel5k and IAPR TC12 benchmark annotated datasets, demonstrate that using the proposed method leads to good performance.