Solitons as self-supported solitary waves are one of the most fundamental objects in nonlinear science. With the realization of Bose-Einstein condensate, matter-wave solitons have aroused enormous interest due to their potential applications in atomic transport and atomic interferometer. In recent years, the artificial spin-orbit coupling has been realized in ultracold atoms, thus providing a new platform to study the nonlinear matter wave solitons under a gauge field, and a variety of novel soliton phenomena have been successively predicted. In this paper, we analyze the effects of linear Zeeman splitting on the dynamics of bright-bright solitons in spin-orbit coupled two-component Bose-Einstein condensate, via the variational approximation and the numerical simulation of Gross-Pitaevskii (GP) equations. For the SU(2) spin-rotational invariant attractive atomic interaction in a uniform case without external trap, we take a hyperbolic secant function as the variational Ansatz for bright soliton in variational approximation, and derive the Euler-Lagrange equations describing the evolution of the Ansatz parameters. By solving the time-independent Euler-Lagrange equations, we find two stationary solitons each with a finite momentum for a weak spin-orbit coupling due to the linear Zeeman splitting. Linearizing the Euler-Lagrange equations around these stationary solitons, we further obtain a zero-energy Goldstone mode and an oscillation mode with frequency related to linear Zeeman splitting: the former indicates that the continuous translational symmetry of the stationary solitons will be broken under a perturbation, and the later shows that the stationary solitons will oscillate under a perturbation. Furthermore, by solving the time-dependent Euler-Lagrange equations, we also obtain the exact full dynamical solutions of Ansatz parameters, and observe that the linear Zeeman splitting affects the period and velocity of soliton's oscillation and linear motion, which may provide a new method to control the dynamics of solitons. All the variational calculations are also confirmed directly by the numerical simulation of GP equations.