Single-photon source is an essential element in quantum information processing, and extensively used in the proof-in-principle demonstration in quantum physics, quantum imaging, quantum cryptography, etc. Considering the operating temperature and system complexity, it is a favorable option to choose spontaneous parametric down-conversion (SPDC) combined with the enhancement effect of a cavity. When generating significant single-photon source via the cavity-enhanced type-II spontaneous parametric down-conversion method, there appears inevitable birefringence effect which obviously influences the resonance condition. In order to compensate for birefringence effect, different approaches have been used such as introducing compensating crystal, placing a half-wave plate, tuning the temperature of the nonlinear crystal, customized conjoined double-cavity structure, and cluster effect. In this work, two quarter-wave plates, with an angle of 45° between the optical axis and the crystal axis, are placed in the cavity to ensure the double resonance of signal photon and idler photon. In the process, the signal photon and idler photon generated simultaneously have different polarizations perpendicular to each other through the type-II nonlinear crystal. Considering horizontally polarized photon, its polarization is changed into left circular polarization by the first quarter-wave plate and then returns as vertical polarization. After traversing a long optical path, it shifts to right circular polarization through the second quarter-wave plate. When the photon passes through the same quarter-wave plate again, the polarization state is originally converted into horizontal polarization state. Then the photon completes a round-trip. The other photon with vertical polarization experiences the same process. As a result, the signal photon and idler photon travel identical optical path. The general explanation is described by the Jones matrices, with the emphasis on the transformation of the polarizations of photons. This method can effectively compensate for birefringence effect, achieving double resonance by using a relatively simple device under the condition of smaller intra-cavity loss and more flexible for adjustment. The signal (idler) photon has a sub-natural linewidth of $1.01( 1.08 )\;{\rm{MHz}} $ , demonstrating the feasibility of the proposed technique. This introduced compensating method paves the way to the realization of single-photon quantum source applied to the research of single-photon-single-atom quantum information processing, quantum interface and quantum network node with a single cesium atom confined in the strongly coupled cavity quantum electrodynamics system.