Due to the star connection of the windings, the impact of the third harmonic which does not exist in three-phase permanent magnet synchronous motor (PMSM) cannot be ignored in five-phase PMSM. So the conventional sensorless control methods for three-phase PMSM cannot be applied for five-phase PMSM directly. To achieve the sensorless control for five-phase PMSM, an iterative sliding mode observer (ISMO) is proposed with the consideration of the third harmonic impact. First, a sliding mode observer (SMO) is designed based on the fivephase PMSM model with the third harmonic to reduce the chattering and obtain the equivalent signal of the back electromotive force (EMF). Then, an adaptive back EMF observer is built to estimate the motor speed and rotor position, which eliminates the low-pass filter and phase compensation module and improves the estimation accuracy. Meanwhile, by iteratively using the SMO in one current sampling period to adjust the sliding mode gains, the sliding mode chattering and estimation errors of motor speed and rotor position are further reduced. Besides, the stability of the SMO and the adaptive back EMF observer are demonstrated in detail by Lyapunov stability criteria. Experiment results verify the effectiveness of the proposed observer for sensorless control of five-phase PMSM.