Quantum phase transition of ultracold atomic gas is one of the core contents in the study of quantum correlational many-body systems. In this paper, two-dimensional (2D) optical lattices are generated by a single fold retroreflected laser beam, and this scheme is used to experimentally design and implement the 2D optical lattice of double wells suitable for isolating and manipulating an array of individual pairs of atoms and predict a topological semimetal in the high orbital bands in this 2D lattice. Two types of optical lattice structures are produced by controlling the laser polarization. One type is the usual 2D optical lattice, which is formed by two independent one-dimensional(1D) optical lattices in two directions and named in-plane lattice, and the other type is the lattice that is formed by the interference between two one-dimensional optical lattices in two directions and called out-plane lattice. When
87Rb BEC (Bose-Einstein condensation) is loaded into the 2D optical lattice, the quantum phase transition between superfluid state and Mott insulator state is observed by controlling the tunneling and in-site interaction. And the phase transition from superfluid state to Mott insulator is judged by observing whether there are interferential lattice points in momentum space. The lattice depths of two cases can be calibrated by Kapitza-Dirac scattering in the ultracold atomic experiment through the time-of-flight absorption imaging. In the in-plane optical lattice, some incorrect points appear in the 45° direction, because the linear polarization degree of beam is impure after being reflected by mirrors and two direction of beam are not completely orthogonal to each other. It is obvious that the two cases have different phase transition points, which is due mainly to the difference in structure. For the in-plane lattice, there are two independent 1D optical lattices, and for the out-plane lattice, the two direction beams mutually interfere with each other, therefore, two optical lattices are not independent of each other. The atoms come back to BEC by reducing the potentials of optical lattice to zero; the temperature of system is slightly higher, because of the jitter of the light lattice. The different behaviors of quantum phase transition are analyzed for two types of optical lattices. This work will provide a platform for the future study of large spin system and strong correlation physics in optical lattices.