The rough sea bottom has a large effect on underwater acoustic propagation and underwater acoustic detection applications. By using the typical shallow water environment from the Yellow Sea, the acoustic propagation characteristics under the condition of both periodic rough sea bottom and strong negative thermocline layer are systematically analyzed by using the parabolic equation model RAM (where RAM stands for range-dependent acoustic model) and ray theory. For a low-frequency and short-range acoustic source, the transmission loss (TL) increases up to about 5–30 dB due to the existence of the periodic rough bottom. Abnormal TLs and pulse arrival structures with different source depths, different periods and heights of the rough bottom are analyzed and summarized. Specifically, when the period of the rough bottom is constant, TL increases with the height of the rough bottom increasing. When the height of the rough bottom is constant, the effect of the rough bottom on the sound propagation becomes smaller with the increase of the period. The mechanism of the TL difference caused by rough bottom is explained by using the ray theory. The incidence and reflection angle of the sound ray on the sea bottom are changed due to the periodic rough bottom, which makes small grazing angles of some of the rays incident at sea bottom become large grazing angles, and the bottom loss increases. On the other hand, the change of the reflection angle increases the number of ray interaction with the sea bottom, causing the reversion propagation. Therefore, the energy of the sound field will attenuate with range increasing. The influence of the periodic rough bottom on the sound pulse propagation is mainly reflected in the energy conversion between sound rays (or normal modes) with different angles, the increasing of energy attenuation of some sound rays with large angles, and the decreasing of multipath structure. The change of the arrival time and relative amplitude of the multipath structure affect the frequency spectrum of the sound field, which will affect the performance of the method based on matching field localization. Most of existing studies focus on the influence of the change in large scale sea bottom topography on the sound field, but there are few studies on small scale periodic sea bottom fluctuations, and the relevant summary of the law of sound propagation is lacking. When sonar is used in the actual shallow water environment, more attention should be paid to the influence of the periodic rough bottom. In addition, the present research results also have important reference significance for the spatial accuracy of surveying and mapping of sea bottom topography.