Abstract: To investigate systematically the coupling fault caused by the interaction between oil-film instability and rotor-bearing rub-impact, the coupling fault was simulated on a rotor test rig with the vibration signals acquired from rotor's speed up/down. The vibration signals in vertical direction are analyzed by 3D-waterfall spectrum, reassigned wavelet scalograms, amplitude spectrum and axial trajectory. The results show that the reassigned wavelet scalogram can extract well the time and frequency information on the rub-impact fault. When the coupling fault of rub-impact with oil film instability occurs, the oil film instability plays a major role in the rotor system, and the rub-impact caused by oil film instability will result in the two combination frequency components due to rotating and oil whirling. In addition, axial trajectory is found irregular and chaotic because there is no common factors found in the combined frequencies, which make them irreducible.