In imaging spectrometers, area array detectors are usually used as photoelectric conversion devices, but the inconsistency of the spectral response among pixels can distort the collected target spectra. To improve the spectral radiometric accuracy of imaging spectrometers, calibrating and correcting the inconsistency of the spectral response among pixels is essential. The signal received by each pixel of area array detector of the indirect imaging spectrometer is usually the superposition of the target multi-spectral radiation signals or full-spectral radiation signals. Therefore, its relative spectral radiometric calibration requires measuring the spectral response of each pixel at different wavelengths on the array detector. Under the ideal conditions, the response values of each pixel in the area array detector are different, so the indirect imaging spectrometer cannot simply calibrate the relative spectral response (RSR) function between pixels by using the method of “monochromator + integrating sphere”. In this work, taking the interferometric imaging spectrometer for example, we analyze the influence of the inconsistency of the RSR among pixels on the target spectral radiation measurement accuracy, and propose a system-level RSR function measurement method for the indirect imaging spectrometer based on the Fourier transform modulation calibration source. In addition, we establish a mathematical model for calibrating the RSR function,and provide guidelines for selecting calibration system parameters such as light source, spectral resolution, and OPD sampling interval. The simulation results show that under the ideal noise-free condition, the 1% spectral response inconsistency among pixels results in a relative error of 1.02% to the recovered spectra. After RSR correction, the relative error of the recovered spectra of different rows decreases to 0.08%. Furthermore, in this work we simulate and analyse the influence of spectral signal-to-noise ratio on the calibration accuracy of the RSR function, and point out that increasing the brightness of the calibration light source, extending exposure time, and combining multi-frame interferograms can enhance RSR function calibration accuracy in practical applications. The research result can provide a theoretical basis for realizing the relative spectral radiometric calibration of indirect imaging spectrometer, which is of great significance in promoting quantitative spectral remote sensing.