Holographic optical element (HOE) lens is an imaging element fabricated through recording wavefront by interference. Because of its advantages of small form factor and wavelength, angle selectivity and arbitrary wavefront formation, it has a good application prospect in augmented reality display. To make the system more compact, the HOE lens is adopted as an off-axis optical element. At the same time, according to diffraction principle, its wavelength response is more sensitive than those of traditional refractive and reflective optical elements. Thus the fabrication and design of a full-color HOE lens is a challenge to optimizing the free-space head-up display system. To systematically analyze the HOE imaging system, the conjugate relation between the object and image is derived by scalar diffraction theory. Then the Gaussian conjugate imaging equation is obtained and the off-axis aberration of distortion and astigmatism in the HOE imaging system are analyzed. In addition, A head-up display with field of view (FOV) of 18° and eyebox of 10 mm is simulated and its imaging process is visualized through the geometric optics method of k-vector diagram and ray-tracing. A full-color HOE lens with high diffraction efficiency is fabricated by interference. Its average peak diffraction efficiency is 56.7%, reaching a high level in the world. A prototype of augmented reality system is established by integrating laser pico-projectior with HOE lens. The experimental results of distortion effect and astigmatism effect of the system are obtained, which are consistent with the simulation results. The modulation transfer function (MTF) parameter of the system is measured, and its definition basically meets the requirements of the human eyes for resolution. The aberration of the system is analyzed and the optimization method is proposed. To optimize the monochromatic image quality, an extra cylindrical lens is added to ensure the same optical power of meridian and sagittal plane to eliminate the astigmatism. Besides, a freeform wavefront is designed by the geometric construction method and forms a freeform HOE to deal with the distortion problem. The local recording freeform wavefront can be calculated by the imaging equation. When full-color HOE is applied to the display system, the images of three channels may separate in the space because of their different reconstruction wavelengths and angles. We propose a pre-compensation method of recording process to solve this problem. If these above-mentioned problems can be solved, due to its good image uniformity, sufficient field angle and eyebox area, the head-up display based on HOE lens with extra optical power will have a better application in augmented reality technology.