The optical system of spaceborne directional polarimetric camera that employs an ultra wide-angle lens for its multiangle, a filter wheel for its multispectral and also its multipolarization capability, a CCD itself for its imaging has a certain polarization effect, which can affect the radiometric accuracy of the non-polarized channels of the instrument. The transmittance of the oblique light rays that are incident on the optical element surfaces is sensitive to the orientation of the electric field, contributing to the linear polarization effect of optical system. The precise polarization measurement and calibration of the passive imaging polarimetry are in urgent need to eliminate the instrumental polarization effect and to improve its radiometric accuracy for observation scenes. The non-polarized channel radiometric model considering the linear polarization effect is deduced in detail by analyzing the instrumental principle and optical structures. Moreover, the reasonably simplified model is established based on the actual lens characteristics. A calibration method in which completely linearly polarized light with different kinds of polarization angles irradiates sparsely the instrument within full field of view and subsequently fits the response in the least square sense, is proposed and simulated. In addition, the measured relative errors of the intensity of incident light with different kinds of polarization states caused by the calibration deviations of instrumental principal physical parameters are analyzed and calculated, such as the azimuthal angle of single pixels, explicit optic polarization rate and low frequency spatial relative transmittance. The actual instrumental parameter values and their calibration deviation amounts are acquired by carrying out the laboratory calibration experiment for instrument and combining the least square fitting. Furthermore, the maximum radiometric calibration relative error caused by the deviation of the physical parameter called explicit optic polarization rate is calculated to be 0.4%, fulfilling completely the requirement of radiometric relative accuracy 5% and retaining abundant tolerance. The study provides a theoretical basis and an experimental guidance in high accurately measuring radiation, calibrating and processing data for the instrumental non-polarized channels with full field of view.