The unconstrained face images collected in the real environments include many complicated and changeable interference factors, and sparsity preserving projections (SPP) cannot well obtain the low-dimensional intrinsic structure embedded in the high-dimensional samples, which is important for subsequent sparse representation classifier (SRC). To deal with this problem, in this paper we propose a new method named discriminative sparsity graph embedding based on histogram of rotated principal orientation gradients (DSGE-HRPOG). Firstly, it extracts multi-scale and multi-directional gradient features of unconstrained face images by HRPOG feature descriptor and incorporates them into a discriminative feature dictionary of sparse representation classifier. Secondly, it seeks an optimal subspace of HRPOG feature dictionary in which the atoms in intra-classes are as compact as possible, while the atoms in inter-classes are as separable as possible by adopting the proposed DSGE dimensionality reduction method. Finally, an optimal algorithm is presented in which the low-dimensional projection and the sparse graph construction are iteratively updated, and the accuracy of unconstrained face recognition is further improved. Extensive experimental results on AR, Extended Yale B, LFW and PubFig databases demonstrate the effectiveness of our proposed method.