Vehicle detection in aerial images is of great interest in the field of remote sensing. Many methods such as the sliding-window-based detection have been successfully developed. A simple and effective mechanism to improve the existing methods is proposed. Vehicle in aerial images usually appears in arbitrary directions. Previous algorithms need to repeat the search at a pixel with all the possible orientations, which often bring the problem of increasing false alarms and computational complexity. To solve this problem, image local orientation is introduced into detection that provides a proper search direction for each pixel. Experimental results on a public database, unmanned aerial vehicle (UAV) and satellite images demonstrate the effectiveness and promising improvements in comparison with existing techniques.