The well-known Su-Schrieffer-Heeger (SSH) model predicts that a chain of sites with alternating coupling constant exhibits two topological distinct phases, and at the truncated edge of the topological nontrivial phase there exists topologically protected edge modes. Such modes are named zero-energy modes as their eigenvalues are located exactly at the midgaps of the corresponding bandstructures. The previous publications have reported a variety of photonic realizations of the SSH model, however, all of these studies have been restricted in the systems of time-reversal-symmetry (TRS), and thus the important question how the breaking of TRS affects the topological edge modes has not been explored. In this work, to the best of our knowledge, we study for the first time the topological zero-energy modes in the systems where the TRS is broken. The system used here is semiconductor microcavities supporting exciton-polariton quasi-particle, in which the interplay between the spin-orbit coupling stemming from the TE-TM energy splitting and the Zeeman effect causes the TRS to break. We first study the topological edge modes occurring at the edge of one-dimensional microcavity array that has alternative coupling strengths between adjacent microcavity, and, by rigorously solving the Schrdinger-like equations (see Eq.(1) or Eq.(2) in the main text), we find that the eigen-energies of topological zero-energy modes are no longer pinned at the midgap position:rather, with the increasing of the spin-orbit coupling, they gradually shift from the original midgap position, with the spin-down edge modes moving toward the lower band while the spin-up edge modes moving towards the upper band. Interestingly enough, the mode profiles of these edge modes remain almost unchanged even they are approaching the bulk transmission bands, which is in sharp contrast to the conventional defect modes that have an origin of bifurcation from the Bloch mode of the upper or lower bands. We also study the edge modes in the two-dimensional microcavity square array, and find that the topological zero modes acquire mobility along the truncated edge due to the coupling from the adjacent arrays. Importantly, owing to the breaking of the TRS, a pair of counterpropagating edge modes, of which one has a momentum k and the other has -k, is no longer of energy degeneracy; as a result the scattering between the forward-and backward-propagating modes is greatly suppressed. Thus, we propose the concept of the one-dimensional topological zero-energy modes that are propagating along the two-dimensional lattice edge, with extremely weak backscattering even on the collisions of the topological zero-energy modes with structural defects or disorder.