More and more overmoded structures have been adopted in electronic vacuum devices because they can increase the power capacity and the frequency of the output signals. Overmoded structure device can more easily work in a low magnetic field. Overmoded structure may cause mode competition, which will reduce the output power. Mode selection in overmoded structure is studied in this article. Dispersion relation is obtained and solved numerically. The results show that the appropriate selection of working point that ensures the device working in a traveling wave state can realize transverse mode selection and appropriation selection of parameter of electron beam and the structure can realize the axial mode selection. The calculation results are verified by particle simulation.