In recent years, quantum Rabi model has aroused considerable interest because of its fundamental importance and potential applications in quantum technologies. For a conventional cavity-quantum-electrodynamic (cavity-QED) system involving the interaction between an atom and photons in a cavity, the atom-photon coupling frequency is much smaller than the transition frequency of the atom and the frequency of the cavity mode. This cavity-QED system is usually described by the Jaynes-Cummings model in which the rotating-wave approximation can be adopted by neglecting the counter-rotating coupling terms in the Hamiltonian of the system. However, by designing the unique structure of the superconducting circuit, the ultrastrong-coupling regime can be achieved in a circuit-QED system in which the counter-rotating coupling terms become as important as the rotating terms. Thus, the rotating-wave approximation cannot be used in the ultrastrongly coupled circuit-QED system. Owing to the ultrastrong coupling, this circuit-QED system is described by the standard quantum Rabi model when a superconducting qubit is coupled only to a single resonator mode. In this work, we experimentally study an ultrastrongly coupled circuit-QED system consisting of a four-junction superconducting flux qubit and a muti-mode coplanar-waveguide resonator. The transmission-spectrum measurement and numerical simulations show that the system is in the ultrastrong-coupling regime. By changing the photon number in the resonator, we observe the frequency shift of the flux qubit via the spectroscopic measurement. This frequency shift contains the contributions from not only the rotating-coupling terms but also the counter-rotating terms, which is in good agreement with the theory. The result indicates that this ultrastrongly-coupled quantum system can be used as a good platform to investigate the quantum Rabi model and has potential applications in various aspects of quantum technology, such as quantum simulation, ultrafast quantum gates, entangled-state preparation and protected qubits.