It is found in the data processing of matched filed source localization experiment that in the case of the slight mismatch of sound speed profile, the louder source below disappears while the weaker source above comes out. Based on the experimental phenomenon, the influences of the sound speed profile on the source localization of different depths are investigated. Firstly, the simulation is conducted in order to extract the influence of the uncertain factors in the sea test, from which it is further concluded that the deeper source is greatly affected. Then another simulation about the mismatch of the depth of the thermocline is conducted in the typical shallow-water negative gradient environment, from which a preliminary conclusion is drawn that the most sensitive depth to the mismatch of the sound speed profile is around 10 meters below the lower boundary thermocline. Finally, this pheonomenon is reasonably explained through a theoretical analysis from two aspects of the normal mode theory.