Flexibility is an important index in the assessment of bridge load-carrying capacity. However, it is still a challenge to obtain accurate bridge flexibility information. The estimation quality of structural scaling factor is closely related to the identification accuracy of flexibility. Combined deterministic-stochastic subspace identification algorithm is a linear system identification method which can effectively extract structural modal parameters from output-input data. However, some unavoidable factors often result in the appearance of spurious, or so-called non-physical modes. How to eliminate the non-physical scaling factor is the key problem for identifying high quality flexibility matrix. In this paper, a 3-step procedure are implemented to achieve better estimation of the scaling factor. In the first phase, subspace identification algorithm is adopted to calculate basic modal parameters from state-space models of different orders and eliminate the spurious modes using stabilization diagram. Subsequently, the modal scaling factor is calculated based on the results obtained in the previous step and preliminarily extract physical scaling factors using the proposed frequency domain similarity index. Finally, the proposed relative scaling factor difference threshold is used to further eliminate the false modes. The quality of the techniques will be assessed using simulated data.