Surface irregularity of optical elements is one of the errors caused in manufacturing process, which has a bad influence on optical system image quality. This image quality deterioration can’t be neglected especially in some all-reflective optical systems. A method by rotating the mirrors for compensating the surface irregularity is put forward in the paper. Firstly, the surface irregularity of all the mirrors is analyzed and the most closely matched mirrors are chosen for one set of system alignment. Then, the wavefront characteristic of optical system and the surface irregularity of each mirror represented by Zernike polynomial are studied, and the relationship between them is analyzed. The calculation of the rotate angle is described in detail. A numerical simulation of the method has been performed for two sets of three-mirror optical system to verify the ability and accuracy of the method. The results show that the astigmatism of the optical system caused by the surface irregularity can be decreased and the image quality of both the two systems can be improved effectively. The method is especially suitable for multiple sets of optical systems alignment.