Abstract: A normalized composite index named analogue deviation was defined to describe a degree of approximation of data samples, based on the similarity and Euclidean distance coefficient. The matchability between images of the same face can be expressed via the analogue deviation of vector sets of image grayscale data. Combining the SVM face detection and tracking template updating with the analogue deviation matching of image grayscale, a new face tracking algorithm based on analogue deviation matching was proposed taking account of both the information on color values and color's spatial positions in image, thus making the algorithm more accurate. Experimental results showed that algorithm proposed can find out the best region matchable to template data in the grayscale matrix of simulated image. In addition, the algorithm is highly robust during static or dynamic face tracking because of its strong resistance to environmental interference.