Based on the phase information in time domain of a Mach-Zehnder interferometer (MZI), a method of calibrating the wavenumber of the source is proposed. Cross-correlation of wrapped phase in time-domain of the MZI is adopted to determine drifts among axial-lines. Owing to non-strictly periodic characteristics of wrapped phase in time-domain of the MZI, the determinable range of shift is unlimited. Synchronization of signals in time domain is then performed to correct their corresponding shifts. The obtained results demonstrate that it is feasible to realize phase measurement with high precision even under unstable swept source for the optical coherence tomography system.