Abstract: Experiments were carried out to investigate the oil-film instability fault behaviors, including oil-film whirling and oscillating, on a special-designed test rig for single-span rotor system. The rotor shaft vibration responses were measured in the processes of both stable and speed up-down running. When the oil-film is oscillating, in the time-domain the rotor vibration is featured with complex multi-period and the center routes of rotor shaft are all of irregularly diffused curves. The sub-frequency, i.e. , 0.44 times the frequency, is shown obviously in both the power-density and holographic spectra. An analysis of time-frequency domain when the rotor running up and down, as indicated by 3-D frequency spectrogram and time-frequency indexing chart, revealed the progress how a rotor oil-film changes from whirling to oscillating state. Results of the experiment and analysis are of importance to further study on the faults due to oil-film instability and the implementation of monitoring and predicting of rotating machines.