Refraction is an important factor influencing radiative transfer since it can change both the propagation path and polarization state of electromagnetic wave. In order to discuss the influence of atmospheric refraction on radiative transfer process, a Monte Carlo vector radiative transfer model, which takes atmospheric refraction into account, is introduced. By using this model, photon random movement in uniform atmospheric layer and at the interfaces between adjacent layers is simulated, Stokes vectors and degrees of polarizations of both directly transmitted and diffuse light, and irradiance at the specific layer is also calculated. The model is validated under two conditions: with taking atmospheric refraction into account, and comparing the simulation results with those in the literature; with taking refraction index distributed homogeneously in space, in which case the model is validated against DISORT and RT3. So, the results indicates that our model is accurate and reliable. The influences of atmospheric refraction on the Stokes vectors of diffuse light in different directions are discussed for pure molecular atmosphere, with only Rayleigh scattering considered. Simulations are performed respectively for different solar zenith angles, for different atmospheric profiles, for aerosols with different types and particle shapes, and for clouds with different base heights and optical depths, and correspondingly, the effect of atmospheric refraction on radiative transfer process is discussed as well. Simulation results show that Stokes vector of diffuse light is influenced by atmospheric refraction to a certain extent, especially for light with a zenith angle ranging from 70 to 110, and with the increasing of solar zenith angle, the influence becomes stronger. When atmospheric profile changes, the effect of atmospheric refraction on polarized radiance field is also changed, for which the possible reason is that deference between atmospheric profiles leads to the variation of refraction index profile. When aerosol and cloud are taken into account, the influence of atmospheric refraction is reduced because of the decreasing of the ratio between side-scattering energy and backward scattering energy. Comparing the simulation results for different aerosol particles shows that the influences of atmospheric refractions in mineral and sea salt aerosol are much stronger than that in water soluble aerosol, besides, there is also great discrepancy among results for aerosols with different shapes. This phenomenon may be explained by the differences in scattering ability and spatial distribution of scattering energy among different aerosols. For cloud, there is no significant difference in result among different cloud base heights, while with the increasing of cloud optical depth, the influence of atmospheric refraction on polarized radiance is gradually weakened.