Extreme ultra-violet (XUV) light and soft X-ray are widely used to detect the microscopic structure and observe the ultra-fast physical process. It is found that high order harmonic with the frequency as high as that of the waterwindow waves and the pulse duration as short as attosecond can be obtained in the laser-plasma interaction. Due to these features, high order harmonic (HH) is a promising alternative to generating ultra-short XUV light and X-ray. Recently, HHs have been observed in the experiments. However, the frequency spectrum is not complete compared with the results predicted theoretically and numerically. It might relate to the damage of the grating target surface by a long laser repulse. In this article, the effect of target surface roughness on the high order generation (HHG) in the interaction between ultra-intense laser pulse and grating targets is investigated by surface current model and particle-in-cell simulations. We find that both the spatial and spectral domains of harmonics are modulated by the periodical structure of the grating due to the optical interference. The roughness on the surface significantly distorts the modulation effect and leads to different radiation angle and spectral distributions. For the ideal grating, only harmonics satisfying matching condition in a certain direction can be enhanced and the radiation power is restricted in the direction nearly parallel to the target surface. When the surface roughness of the grating target is considered, the matching condition is not valid and the harmonics are scattered into the direction away from the target surface. Comparing with the ideal grating target, most of the harmonic energy is concentrated in the low order harmonics and the intensities of the harmonics decrease rapidly with increasing HH order when surface roughness is considered. The results show good consistence with the phenomena observed in previous experiments and provide the technical reference for exploring the future applications of HHs and HHG.