Scientifically predicting the vulnerability of buildings to analyze the weak links of urban disaster prevention, is the key step to prevent and mitigate the earthquake loss. As Huizhou building survey results, a small number of sampling buildings have detailed drawings, can be carried out specific seismic damage, most buildings without detailed drawings, there is no suitable accurate calculation model. In this paper, the seismic damage index is calculated for the brick building with detailed drawings. The inpact index of seismic damage is identified and quantified according to Huizhou building general survey data.Then the sample datebase and the model of KPCA and GA_LSSVR are established. The model is used to predict the seismic damage index of Huizhou brack masonry building without detailed drawings,and the seismic damage matrix is obtained. This paper uses the KPCA to extract the characteristics of the damage factors, and uses the LSSVR to predict the seismic damage index. The relevant parameters in LSSVR were optimized by GA.This forecasting model’s availability and effectiveness are verified in the text.