Deep learning shows strong capability in pattern recognition tasks such as object detection and speech recognition. It has a more powerful feature learning ability than general machine learning methods that require extraction of manual features. In this paper, the object detection method is applied to locate and classify lesions for the detection of medical breast masses. At the same time, an idea of transfer learning is introduced, based on the network of Faster RCNN. Furthermore, five feature extractors of the network, which are ResNet101, inception V2, inception V3, Mobilenet, and inception ResNet V2, are adopted in order to explore the impact for the model by using five feature extractors. The experimental data are DDSM dataset, and the performance differences of the models with five feature extractors in detecting benign and malignant breasts are compared separately based on the ROC trade-off curves. The results demonstrate that the classification model based on Inception ResNet V2 feature extractor has a significant performance, compared with the other four feature extractors.