Abstract: According to the problems encountered in image understanding and classifying in the complex environment, a novel scene feature extraction method was proposed. This method adopts perceptive features of orientation, color and intensity to construct saliency map and the maximum operation is used to get the more accurate orientation information. A region segmentation algorithm is designed to accelerate feature extraction. Then, region segmentation descriptor is constructed. Finally, image matching is implemented through the support vector machine (SVM). Compared with traditional methods, it has a good invariance in image scaling, translation and robustness across a substantial range of affine distortion. Meanwhile, it has good classification results. The experimental results demonstrate that the method is well applied to scene classification and retrieval with better timeliness.