In this work, the first-principles method based on materials studio(a soft ware) and the density functional theory is used to invesigate the properties of charge reflention and charge endurance in HfO2 as a trapping layer in charge trapping memory (CTM). Two supercell models are optimized for the monoclinic HfO2, separately. One contains a four-fold-coordinated O vacancy defect (VO4), and the other is a co-doped composite defect consisting of a VO4 and an Al atom. Interaction energies, formation energies, Bader charge, density of states and trapping energy are calculated for the two models. According to the calculated results of interaction energies and formation energies, it is found that the structure is the most stable and the defect is the most easily formed when the distance between the two kinds of defects is of 2.216 in the co-doped composite defect system. The trapping energy results show that the co-doped composite defect system can trap both electrons and holes. Moreover, the trapping ability of the co-doped composite defect is enhanced significantly as compared with the VO4 defect. Bader charge analysis shows that the co-doped composite defect system provides a more preflerable site for the charge reflention. Calculations of the density of states show that the co-doped composite defect system has a strong effect on the trapping energy of holes. Calculated energy changes after program/erase cycles show that the endurance is improved obviously in the co-doped composite defect system. In conclusion, the date reflention and endurance in the trapping layer of monoclinic HfO2 can be improved by doping of the substitutional impurity Al. This work may provide a theoretical guidance for performance improvement with respect to the date reflention and endurance of CTM.