The performance of two-color middle wavelength photovoltaic HgCdTe detector is simulated numerically for design optimization. Three recombination mechanisms (radiation, Auger, and Shockley-Read-Hall(SRH)recombination), trap-assisted tunneling, and band-to-band tunneling are considered in two-dimensional model. The tunneling through barrier layer is calculated by transfer matrix method. The n-p-p-p-n structure is designed in simultaneous mode. The effect of SRH recombination electron lifetime in p-region on spectral response is examined, and the dependence of crosstalk on composition gradient of barrier layer is analyzed. Simulation results show that the quantum efficiency decreases rapidly with decrease of SRH electron lifetime in p-region,and at least about 10ns of SRH electron lifetime is essential for good performance of the detector. Crosstalk decreases to the steady value determined by optical crosstalk as the composition gradient of barrier layer increases to about 0.03, so critical composition gradient of about 0.03 is necessary for suppressing the electrical crosstalk.