It has been suggested that specular component would contribute to color constancy since it reflects the color of illuminant. It was also reported that specular highlight improved color constancy in CG images. However, it is not clear how specular component contributes to color constancy of real objects under natural environments, which consist of a variety of materials and clues to illumination. In this study, we examine whether the specular reflection of a surface influences to color constancy by comparing papers with the different levels of glossiness. We built a booth arranged like a normal room illuminated by lamps with correlated color temperature 5000K or 2700K. Test samples were three-dimensional wavy surfaces covered with a gloss, a semi-gloss and a matt paper, respectively. We tested five color samples: 5YR 6/6,5P 6/6,5BG 6/6,5GY 6/6 and N6 in Munsell notation. Observers evaluated the color appearance of test samples using an elementary color naming method. Two viewing conditions were tested: a natural viewing condition and a limited-view condition in which observers only viewed a test sample through a black viewing box. As a result, the color appearance of test samples under two illuminations showed small differences in the natural viewing condition, meaning good color constancy. However, they did not show any systematic differences between the glossiness of paper, implying little contribution of specular component compared to other information in the surrounding. In the limited-view condition, the results still showed color constancy in some degree, suggesting the three-dimensional shape contributed to color constancy even though there was no information in the surrounding. We, however, did not find any differences in the glossiness of paper even in this condition. Our results suggest that the specular reflection of color paper has little effect on color constancy in a real environment.