A nonlinear controller was proposed to realize the hovering control for the hex-rotor aircraft by using the inertial optical flow. Meantime, an auxiliary hovering control system was presented to overcome the influence of external factors. Firstly, the spherical camera was used to get the image of target plane, and the projection from target point to camera lens was realized by dynamic mapping. Then the translational optical flow of target point was obtained by comparing the two continuous time images. Finally, the divergent translational optical flow was utilized as the feedback to implement the stable hovering control for the hex-rotor aircraft. The indoor textured floor was selected as the target plane, and this experiment was tested ten times totally. The actual flight tests showed that the errors on horizontal and vertical directions were all less than ±5cm, which verified the feasibility of the proposed nonlinear controller and auxiliary hovering control system.