According to the mechanism of nearly degeneration four-wave mixing that occurs in a sufficiently nonlinear resonant gaseous medium caused by phase-modulated laser beam, the electromagnetism excitation and dynamic evolvement of modulation transfer spectroscopy(MTS) technique is studied theoretically, and then mathematic models of dispersion spectrum and absorption spectrum of MTS are established too. The results show that when the frequency of modulation signal source is equal to 0.72 times the linewidth of absorption line approximatively, the intensity of dispersion signal reaches a maximum value; it can obtain a better frequency-stabilization result when modulation factor is set to be nearly 0.3; the instability of demodulation phase has a big effect on the absorption spectrum, but on dispersion spectrum, the effect is tiny when the demodulation phase fluctuates near 90.