Based on the reconstruction of the polarization parameters in a channel modulating polarization imaging system, the polarization features of the target could be extracted effectively. Considering that the reconstruction of polarization parameters can provide important reference for target recognition, material analysis, remote sensing and bio-medical treatment, the research on accurate reconstruction of polarization parameters is now urgently required. In order to improve the accuracy of polarization parameter reconstruction, we first study the influence of sample frequency of interference fringes on the imaging process. For the same carrier frequency, conjugate spectra are separated and also the spectra are not aliasing for two adjacent spectral lines. It is concluded that to prevent the image spectrum from aliasing, the sample frequency should be at least 4 times the maximum fringe frequency of the polarization interference image. Then we study Stokes parameter reconstruction method when the spectral line positions of interference image are changed by assembling error. Since different Stokes parameters are amplitude modulated at different frequencies, we apply segment filters to split the frequency domain into different regions, and seek for the largest spectrum in corresponding regions. The largest spectrum in different regions can be used to determine the spectral line position of polarization carrier frequency, and the two-dimensional images of the target are rebuilt in sequence by the frequency shifting, spectral filtering, and Fourier inversion transforming. According to the above method, we could obtain an exact polarization rebuilding image when the line position of polarization carrier frequency is modified. Finally, we use the computer simulation and experiment to verify the feasibility and effectiveness of such a rebuilding method. The results demonstrate that the reconstruction of polarization parameters in channel modulating polarization imaging by this rebuilding method is better than by the traditional theoretical rebuilding method. In detail, the mean square error between the reconstruction and original input image could be suppressed to 0.001 while the peak-signal-to-noise ratio is improved and the structural similarity index measurement could be more than 0.9 by utilizing the new rebuilding method. It turns out that the reconstruction method with great superiority can provide a promising reference for further research of channel modulating polarization imaging system.