Optical quantum memory plays an important role in scaling-up linear optical quantum computations and longdistance quantum communication. For effectively realizing such tasks, a long-lived and highly-efficient quantum memory is needed. The dynamic electromagnetically-induced-transparency (EIT) process can be used for completing an absorptive storage scheme in an atomic ensemble. In such a process, the quantum states of coming single photons can be coherently transformed into spin waves associated with coherences between atomic ground levels via switching off controlling light beam. For storing a single-mode optical signal, a pair of ground levels is involved. While for storing an optical polarization qubit, i.e., two orthogonal polarization modes, the coherence between two pairs of ground levels will be involved. Also, to obtain a high efficiency in the EIT optical storage, the optical-depth of the cold ensemble should be high. For prolonging the coherent time of the spin waves stored in atomic ensemble, decoherence between spin waves due to atomic motion and non-uniform Zeeman shift of ground levels should be effectively suppressed. Recently, a long-lived and highly-efficient optical quantum memory for single-mode storage in a high-optical-depth cold atom ensemble has been experimentally demonstrated via the gradient echo memory scheme (2016 Optical 3 100). While, for the optical polarization qubit storage, a long lifetime (in ms) and high-fidelity EIT storage experiment has been achieved by our group, but the storage efficiency in the experiment is very low (8%) due to lower optical depth of the cold ensemble (2013 Phys. Rev. Lett. 111 240503). The storage efficiency in long-lived storage of two orthogonal polarization modes still needs further improving. Here in this paper we demonstrate an experiment of long-lived and highly-efficient storage of two optical orthogonal polarization modes in a high optical-depth cold atomic ensemble via dynamic EIT process. For achieving a long lifetime in the storage experiment, we follow the two steps, which are used in our previous work (2013 Phys. Rev. Lett. 111 240503). 1) We make the signal and writing-reading light beams collinearly pass through the cold-atom cloud along the z direction to suppress the decoherence between the spin waves due to atomic motion. 2) We apply a moderate magnetic field (13.5 G) to the cold-atom ensemble to lift Zeeman degeneracy. So, the magnetic-field-sensitive transitions are removed from EIT system and the two optical orthogonal polarization modes are stored as two magnetic-field-insensitive spin waves. In contrast to our previous experiment, we finish the storage in the high optical-depth cold atomic ensemble. To obtain such a high optical-depth cold atomic ensemble, we expand the diameters of the trapping laser beams and use a pair of rectangular magnetic coils in a magnetic optical trap (MOT) to prepare a cigar-shaped cold atomic ensemble. The MOT magnetic field is further compressed, and then the optical-depth of the cold atomic ensemble increases up to ~11 in the present experiment, which allows us to achieve a storage efficiency of 30%, which exceeds the previous value (8%). At an MOT repetition rate of 10 Hz, the measured zero-delay storage efficiencies for the two orthogonal polarization modes are symmetric, which go up to ~30%. The 1/e-folding lifetimes of the two orthogonal polarization modes rise up to 3 ms. We also measure the dependence of the zero-delay retrieval efficiency on the MOT repetition rate F and find that the storage efficiency is still more than 20% when the repetition rate F is 50 Hz. The present results will allow one to achieve a long lifetime and highly-efficient quantum memory for photonic polarization qubit and then find applications in scaling-up linear-optical quantum computations and long-distance quantum communication.