The geometric structures of transition metals (Fe, Co, Ni and Zn) doped rutile TiO2 are studied using the first-principles method based on the density functional theory. The lattice parameters, the electronic energy band structure, and the optical properties are calculated and discussed. The results show that the errors between calculated and experimental values of lattice parameters are less than 3.6%. Appropriate dopants of transition metal ions not only influence the band structure of rutile TiO2 system and broaden the scope of light absorption, but also play an important role in trapping electrons, improving the effective separation of electronic-hole pair and enhancing light absorption ability. The optimum Fe, Co, Ni, Zn doped rutile TiO2 systems in theory are Ti0.75Fe0.25O2, Ti0.75Co0.25O2, Ti0.75Ni0.25O2, Ti0.17Zn0.17O2, respectively. The 3d orbits of Fe, Co, Ni split into two groups of energy bands, t2g and eg states contribute to the higher level of valence band and the lower level of conduction band, respectively, which is conducive to the generation of electronic-hole pair and the enhancement of photocatalytic performance of rutile TiO2. Zn 3d orbit is completely filled with electrons, and the electrons are hardly excited, so the photocatalytic activity of rutile TiO2 is not obviously improved.