The extreme ultraviolet (EUV) lithography technology required for high-end chip manufacturing is the first of 35 “bottleneck” key technologies that China is facing currently. The high conversion efficiency EUV source and low out-of-band radiation play a significant role in the application of the EUV lithography system. In this work, the EUV source and out-of-band radiation are studied by using laser irradiated solid Sn target and low-density SnO ₂target. The result shows that a strong EUV radiation at a wavelength of 13.5 nm is generated when the laser irradiates the two forms of Sn targets. Owing to the self-absorption effect of the solid Sn target plasma, the maximum intensity of the wavelength is not located at the position of 13.5 nm, which is working wavelength of EUV lithography system. However, the peak radiation spectrum is located at the position of 13.5 nm with low-density SnO ₂target due to its weaker plasma self-absorption effect. In addition, the satellite lines are weaker in low-density SnO ₂target than in the solid Sn target, so that the spectrum efficiency of the EUV at 13.5 nm (2% bandwidth) is increased by about 20%. On the other hand, the experimental study of the out-of-band radiation is carried out. The out-of-band radiation spectral results show that the out-of-band radiation is mainly dominated by the continuum spectrum. Compared with the solid Sn target, the low-density SnO ₂target contains a part of the low Zelement O ( Z= 8), resulting in a low-intensity continuum spectrum. In addition, the collision probability of ion-ion and electron-ion both become low when the laser irradiates the low-density SnO ₂target, resulting in a short out-of-band radiation duration time. Therefore, the out-of-band radiation generated by the laser irradiated on the low-density SnO ₂target is weak based on the above reasons. The angular distribution of out-of-band radiation measurement results shows that the intensity of out-of-band radiation decreases with the angle increasing. A cosine function $A \cos ^\alpha \theta$ can fit the angular distribution of the total radiation.