In recent years, new optoelectronic materials such as GaN-based thin-film semiconductors and rare-earth-ion doped luminescent materials have aroused the interest of many researchers. The GaN-based semiconductors have wide and direct energy gaps which could be adjusted to cover the whole visible light spectrum region by doping. They have been successfully applied to fabrications of blue lasers and light emitting diodes. The rare-earth-ion doped luminescent materials have exhibited many advantages in luminescent properties such as intense narrow-band emissions, high conversion efficiency, wide emission peaks ranging from ultraviolet to near infrared, long lifetime ranging from nanoseconds to milliseconds, and good thermal stability. They have been widely applied in the fields of illumination, imaging, display, and medical radiology. So far, the studies on GaN-based thin-film semiconductors and rare-earth-ion doped luminescent materials focus mainly on their growth and linear optical properties. In contrast, the investigations of the nonlinear optical properties of these materials, which have potential applications in many fields, are still lacking. In this paper, GaN-based thin-film semiconductors, such as undoped GaN, Mg-doped GaN and InGaN/GaN multiple quantum wells, are successfully grown by metal-organic chemical vapor deposition. Their nonlinear optical properties are studied by using an 800-nm femtosecond laser light. The nonlinear optical properties are different when the laser light is focused on different positions of the samples. The competition between different nonlinear optical effects reflect directly the competition in stimulated luminescence energy. And particularly, it is closely related to the density of energy states, stimulated luminescence energy, and the sample band gap energy difference. In addition, the competition between different nonlinear optical effects, such as multiphoton-induced luminescence and second harmonic generation, is clearly revealed and is manifested in the dependence of the nonlinear optical signal on excitation intensity in this investigation. And also, the competition mechanism is preliminary studied in this paper.