Spectral-domain Optical Coherence Tomography (SD-OCT) systems generally have nonlinear sampling problems in wavenumber domain. In order to realize the precise positioning of the discrete interfaces and the high-quality reconstruction of OCT images under conventional fast Fourier transform, it is necessary to solve the precise calibration problem of the absolute wavenumber of the discrete sampling points in the spectrometer. In this paper, an absolute wavenumber calibration method based on the absolute phase of the characteristic spectral line and the constraint polynomial fitting phase under precise optical path difference was proposed. In the sample arm of the SD-OCT system, metal gauges with precise thickness difference are used to obtain the absolute phase value of the characteristic spectral line, and the phase wrapping times corresponding to the characteristic spectral line are further accurately solved. Thus, this method overcomes the 2π ambiguity of spectral phase in conventional interferometric phase methods. At the same time, combined with the polynomial fitting phase of the high signal-to-ratio region under window constraint, the accurate calibration of the absolute wavenumber of each sampling point is realized. Finally, comprehensive comparisons between the proposed method and the traditional resampling method in terms of discrete interface positioning, axial resolution and image reconstruction quality verifies the significant advantages of this method.