Abstract: The purpose of this work developed a coupled RFE and object-based random forest method for seismic damage identification of mountain buildings from UAV images. An object-oriented multi-level classification system is established by ESP2 algorithm, and then a random forest classifier is used to identify the building damage based on the features selected by Recursive feature elimination (RFE). The results show that the best segmentation scales for vegetation, river and building are 196, 156 and 130, respectively, and 24 features are selected by RFE with the overall accuracy of 95.3% and Kappa coefficient of 0.807. The method can be used to identify the seismic damage of buildings.