The rectangular waveguide grating slow-wave structure (SWS) is a new type of RF system of millimeter traveling wave tube (TWT). However, it has narrow pass band. For the purpose of broadening the bandwidth of this circuit, it is necessary to study the influence of groove shapes on the characteristics. In this paper, the dispersion equation of a rectangular waveguide grating SWS with arbitrary grooves is derived by means of an approximate field-theory analysis, in which the continuous profile of the groove is approximately replaced by a series of steps, and the field continuity at the interface of two neighboring steps and the matching conditions at the interface between the groove region and the interaction region are ensured. The cold test on dispersion characteristics of a rectangular groove SWS shows that the theoretical results are in good agreement with the experimental results. We have calculated the dispersion characteristics and the coupling impedance of the slow-wave structures with some special groove shapes. It shows that the dispersion characteristics of the triangle-groove structure is the weakest and the coupling impedance of it is the lowest, while the dispersion characteristics of the inverted-trapezoid-groove structure is the strongest and the coupling impedance of it is the highest.