Fabricating integratable and high-efficiency optical polarization devices is one of the fundamentally important challenges in the field of terahertz optics. Compared with the traditional polarization materials such as quartz crystal and liquid crystal, MgO crystal is one of the most important potential candidates for fabricating terahertz optical devices due to its high transmittance in terahertz frequency region. To determine the birefringence characteristics of MgO crystal in the terahertz frequency region, the modulation of the polartization state of a terahtertz wave through a $\left\langle {100} \right\rangle $ crystalline MgO flake is studied using terahertz focal plane imaging method. Within this approach, the polarization of a terahertz wave can be intuitively identified from the imaging of the amplitude and the phase of the z-direction component of terahertz electronic field. By measuring the imaging of both the amplitude and the phase of terahertz field with and without passing through the $\left\langle {100} \right\rangle $ crystalline MgO flake, it is found that the left and right circularly polarized light are converted into perpendicular linearly-polarized light after passing through the MgO flake. The polarization direction of the linearly polarized light changes with the rotating of MgO flake along the direction perpendicular to the light propagation. The conversion between the linearly polarized light and the circularly polarized light is analyzed by using the Jones matrix approach. These properties indicate that the $\left\langle {100} \right\rangle $ crystalline MgO flake acts as a quarter wave plate for terahertz waves. To further identify the character of terahertz quarter wave plate, the refractive index of the ordinary and extrordinary light within terahertz frequency region of crystalline MgO crystal are measured by using transmission terahertz time-domain spectroscopy system. By comparing the phase difference between the ordinary and extraordinary light after passing through the MgO flake, it is shown that a quarter of wavelength difference between the ordinary and extraordinary light is obtained. These results indicate that the $\left\langle {100} \right\rangle $ crystalline MgO crystals can be used to fabricate quarter wave plates and relevant polarization devices in the terahertz band.