Optical heterodyne detection is one of the key techniques for developing synthetic aperture ladar. The decoherence effect caused by the roughness of the target surface is particularly important for the practical use of the synthetic aperture ladar. The effect of the target surface roughness on the echo signal detection is discussed in the paper. One-dimensional and two-dimensional Gaussian random rough surface model are created by the use of the Monte Carlo method. The detection process of the target echo signal is simulated to study the effect of the target surface roughness on the intermediate frequency signal. And two optical heterodyne detection experiments are conducted in which the targets each include a rough surface and a smooth surface to verify the serious decoherence effect of the target roughness on the synthetic aperture laser radar echo signal. Meanwhile, the wavefronts of both the local oscillator and the signal beam are monitored at the position of the detector with a digital wavefront analyzer. The measured phase distribution accords well with the simulation result. It is shown that the phase of the echo signal from a rough surface has a serious distortion. The results obtained could be useful for designing the parameters of the laser source and the receiver, as well as for estimating the detection range of the ladar system.