At the cosmological scale, there exist many anisotropic anomalies in the low-l multipoles of the CMB angular power spectrum. Especially, the normals to the octopole and quadrupole planes are aligned with the direction of the cosmological dipole at a level inconsistent with Gaussian random. The inconsistency indicates that the anomalies may not be boost effect from the CMB rest frame to the peculiar frame. It hints us that the boost invariance might be violated on a cosmological scale. There are some discrepancies between the astronomical and cosmological observations, and the predictions are solely based on general relativity and the standard model for elementary particle physics. The solutions are the introduction of dark matter and dark energy. However, all the experiments aiming at finding dark matter particles give negative result and it is still a mystery:what the dark energy is comprised of. We suppose that the Lorentz symmetry begins to be violated partly from the scale of galaxy and utilize the very special relativity symmetry group E(2) as an example to illustrate the Lorentz violation effect on the large-scale effective gravity. A local E(2) but Lorentz invariant gauge theory can be constructed based on the equivalence principle and the gauge principle. To realize the E(2) symmetry, the closure requirement of Maurer-Cartan eqnarray on E(2) algebra needs to be satisfied by postulating constraint conditions among the components of the Lorentz connection. The local Lorentz invariant gauge theory with a Hilbert-Einstein action is a theory with torsion in general case. However in the case of scalar matter source, the theory is exactly the theory of general relativity with Levi-Civita connection and zero torsion. In the E(2) gauge theory case, the closure requirement of Maurer-Cartan eqnarray for E(2) algebra postulates 12 constraint eqnarrays among the components of the Lorentz connection and the eqnarrays of motion for connection reduce the number of independent components of connection to 12. The eqnarrays of motion for the tetrad field do not contain only the involved tetrad field components nor these relevant independent components. So the whole number of variables needed to be solved is 12 more than that in general relativity while there are 12 more eqnarrays in the meantime. The torsion or the contortion field of the E(2) gauge theory is non-trivial even in the case of scalar matter source distribution. Decompose the connection into Levi-Civita one and the contortion part and rewrite the eqnarrays for tetrad field in the formalism of general relativity, then there will appear an effective energy-momentum tensor contributed by the contortion distribution, in addition to the ordinary matter source distribution even for the case of scalar matter source. We expect it to contribute at least part of the dark matter effect. We also examine the holding of the first and second Bianchi identities induced by Jacobi identity of the E(2) gauge theory. The approach of our modified gravity is different from other approach of modified gravity in the sense that we construct the modified gravity by modifying the spacetime symmetry on a large scale and the emergence of effective energy-momentum tensor caused by Lorentz violation effect is due to a purely large scale effect.