In comparison with uniformly magnetized states, vortex structures demonstrate a rich frequency spectrum of spin-wave (SW) excitations. However, a detailed theoretical description of the magnetic modes is generally still a challenge due to the difficulty of analytic calculation, except for the well-defined symmetric circular states. In contrast, the method of micromagnetic simulations combined with Fourier analysis is shown to be very powerful for gaining insight into the nature of magnetic excitation modes. Vortex excitation modes have been reported to be directly influenced by the geometric symmetry of the elements and/or the nature of the initial perturbation of pulse field. In order to understand how the reduced symmetry affects the vortex SW modes, we perform the micromagnetic simulations on vortex modes excited in a submicron-sized thin ellipse. In order to excite the spin-wave modes, a short in-plane Gaussian field pulse is applied along the short axis direction. After the pulse, the off-centered vortex core moves following an elliptical trajectory around its equilibrium position. Simulations provide the time evolution of the local magnetizations (at each discretization point) and dynamics of the spatially averaged magnetization. To determine the mode frequencies, the spectrum is obtained from the average magnetization through Fourier transformation from time domain the frequency domain. By means of Fourier analysis, a variety of azimuthal SW modes can be observed in the excitation spectrum. The ellipse in single vortex state has a twofold rotational symmetry with a rotation of πup around the z-axis (out-of plane) and can be described by the C2 group. The observed azimuthal modes can be divided into two categories according to their symmetry. Two modes occur alternately with increasing azimuthal number, indicating that the magnetic excitation modes remain to keep the symmetry of the ellipse structure. Their frequencies are found to increase linearly with the azimuthal index number. An increase of the SW frequency with increasing number of nodal planes is rather well known, which results from the competition between exchange and dipolar energy terms. According to the temporal evolution of the ellipse's spatially averaged energy densities, our micromagnetic simulation shows that the average exchange energy is significantly higher than the magnetostatic energy, suggesting that the exchange interaction plays a more important role in the excitation modes. The exchange energy density is mainly focused on the core origin while the largest contribution of the magnetostatic energy is distributed near the long axis. Thus, we can conclude that the exchange interaction provides the principal contribution to the vortex energy in such small ellipses with a single vortex state, resulting in the increasing frequency versus the azimuthal number, that is observed.