It is a radical but pressing problem to measure the group refractive index of dispersive material both accurately and rapidly in optical engineering field. In this paper, a white-light spectral interferometric system employing a fiber-optic spectrometer is developed for solving this problem. The system takes advantage of the characteristic of fiber-optic spectrometer which can get all the interferometric information by a single shot image without any mechanical scanning. Compared with the traditional windowed Fourier transform, a wavelet transform algorithm is used to directly extract the group delay rather than the phase from the interferogram. Therefore, the new method can provide a simple and fast solution, while still maintaining high accuracy. Based on this white-light Michelson interferometric system, fused silica and BK7 glass samples are measured respectively and the results agree well with the theoretical values over a broad spectral range. At the end of this paper, different mirror positions are used to verify the repeatability of our method.