Railway plays a major role in our daily life and national economy. In recent years, people payed much more attention to the safety operation of the high-speed train. In fact, the rail cracks originate from surface micro cracks will directly affect the safety of high-speed train. Therefore, it is vital to detect the rail surface micro cracks. Numerous nondestructive testing methods have been developed and applied in the detection of high speed rail cracks, such as magnetic particle testing, eddy current testing, and ultrasonic testing, etc. However, all the above conventional methods could only achieve crack information from the point of one-dimensional signal but not effective for the detection of surface micro cracks. A surface defect detection method based on photoacoustic (PA) signal from high speed rail is proposed soas to detect the surface crack more exactly and visually. Simulation and experiments are designed to validate the proposed method. Firstly, three models of high-speed rail with transverse crack, oblique crack, and scale stripping are established respectively. Meanwhile, the PA effect is simulated by finite element analysis and K-wave. Then, PA image of the rail surface is reconstructed by time inversion reconstruction algorithm, and some parameters, such as the center frequency of ultrasonic sensor and the laser power are also confirmed in further simulation. Subsequently, an experimental platform is established to collect the actual PA signal from a rail surface and to reconstruct PA images of the rail surface and shallow layer. The crack appearing in PA images are clear enough to show the receive crack information, such as sizes, propagating directions, and locations, which can be used to evaluate the rail states and decide processing scheme. It is proved that clear images of rail surface and shallow layer can be received by the detecting method of high-speed rail surface defects based on photoacoustic signal, and the surface cracks can be detected effectively.