Underwater imaging is widely applied to mariculture, archaeology, and hydrocarbon exploration, because it can provide the information about visualized target. Among various underwater imaging techniques, polarization imaging is of particular interest to us, due to its simple system structure and low cost. It images the waterbody through using the polarization characteristics of light, specifically, the background light and target light. Active polarization imaging method illuminates a target scene with an artificial polarized light source to provide polarization information for imaging. But in neritic area, active imaging leads to complex light scattering conditions when artificial light and natural light are superimposed together, which further leads to poor image quality. Passive underwater polarization imaging attempts to recover a clear image by utilizing the polarization characteristics of background light and target light. However, serious color cast always appears in the final image, resulting from light absorbed by water, which may further result in target distortion. In this manuscript, we present a passive underwater polarization imaging method for detecting a target in neritic area. A depth-information-based underwater Lambertian reflection model is established by incorporating the depth information into the traditional Lambertian reflection model. First, we attribute the light changes in color and brightness of a Lambertian surface to the spatial variation of the light. According to Lambertian reflection model, the appearance of a target on a detector depends on the light source, the surface reflectance, and the camera sensitivity function. But in underwater imaging, light attenuation at different wavelengths also varies with depth. By analyzing the transmission characteristics of background light in water, we build a physical relationship between the depth information of the scene and the background light. After that, we take the depth information as the weight of light intensity distribution. Then we calculate the product of the light intensity and the camera sensitivity function in the underwater scene according to gray world algorithm, and the real color information of the target can be obtained. Finally, the clear image of an underwater target scene can be obtained, where color cast is calibrated and background light is removed. Underwater experiments are conducted to demonstrate the validity of the proposed method. Besides, the quantitative analyses also verify the improvement of the quality in final target image. Compared with conventional passive underwater polarization imaging methods, the proposed method is capable of detecting targets in various conditions, with the color cast problem solved. It can provide underwater images with better quality and valid detailed information. Furthermore, the proposed method is easy to conduct with no need to change the conventional polarization imaging system and is promising in various practical applications.