High Reynolds number helical vortex system possesses a dominant characteristic of helicopter rotor flow field, whose spatiotemporal evolution is one of the most important factors affecting the aerodynamic performance. In such a type of flow field, vortex interaction due to flow unsteadiness and non-linearity possesses the most common characteristic, whose complexity and tightly coupling property make it very hard to understand its physical behaviors. Also, the multi-scale characteristic of the helical vortex evolution poses a severe challenge to the computational fluid dynamics community. In this paper, a hybrid numerical method, blending 5th order weighted essentially non-oscillatory and 6th order cenitral schemes, implemented in a finite volume overset grid framework based on adaptive mesh refinement technique, are adopted to capture the evolution of vortical structure in a high resolution manner. The highly-resolved flow field of Caradonna-Tung rotor with two blades in hover, with a tip Mach number of 0.439 and a tip Reynolds number of 1.92×106, is obtained using delayed detached eddy simulation method. The averaged pressure coefficient distributions at 50%R, 68%R, 80%R, and 96%R stations show good agreement with experiment data, and the vortex trajectories during the stable stage, as well as the instantaneous turbulent kinetic energy distribution in the wake region, and also validate the computed result. In order to reveal the underlying physical mechanism of the helical vortex structure evolution, proper orthogonal decomposition analysis and Lagrangian coherent structures are adopted as a post processing procedure, which brings more details about the unsteady vortex system. The evolution characteristics of the vortex system are revealed as follows. 1) Trailing edge vortex sheet rolling-up and interaction with tip vortex strongly affect vortical stability and downstream nonlinear vortex-vortex behaviors. 2) The vortical system exhibits the spatiotemporal stability at an age less than 720°, and the vorticity decays with age and trajectories by power law, the distribution of circumferential velocity and the evolution of vortex core radius agree well with theoretical models. 3) Results of proper orthogonal decomposition analysis show that the mode of free stream and point vortex combination plays critical roles in the state transition of flow field. 4) Lagrangian coherent structure further gives the evolution process of helical vortex, and reveals the flow characteristics of vortex pairing and co-rotating, showing the effect of trailing edge vortex roll up phenomenon in the vortical system evolution.