The nano-diamond has been a hot topic in the field of nano-science and nanotechnology for its optical properties. Much effort has been devoted to improving the fluorescence and Raman scattering intensity of nitrogen-vacancy (NV) center in nano-diamond by using plasmon resonance effect in sensing area. A combination of Ag nanoparticle and diamond can not only take advantage of the stability and biocompatibility of diamond, but also enhance the local electric field around NV center through the Ag nanoparticles, thereby speeding up the radiation of the fluorescent near the surface of the substrate, improving the strength and stability of the fluorescence, and greatly broadening the application areas of Raman spectroscopy. In this paper, we mix the nano-diamonds with Ag nanoparticles to improve the fluorescence and Raman scattering intensity on the basis of the localized surface plasmon resonance effect. The influences of Ag mass concentration on the Raman spectrum and fluorescence intensity are investigated. The results show that when the concentration of nano-Ag nanoparticles reaches up to 5 wt%, the light intensity becomes saturated, but the concentration further increases up to a value more than 7 wt% the light intensity begins to decline. Then the corresponding radiative transition rate and the fluorescence quantum efficiency are investigated, and based on these researches, influences and mechanism of surface plasmon resonance (SPR) enhancement are discussed thoroughly. We deduced that the fluorescence enhancement is mainly due to the enhanced surface plasmon field caused by transfer of surface plasmon resonance energy and the energy transfer between surface plasmon and excited state of NV centers. When the concentration of Ag nanoparticles reaches an appropriate value, a suitable distance between metal nanoparticles and diamond is obtained, thereby ensuring the strong local electric field forming on the metal surface, accelerating the emitting photons of diamond in the excited state, and also suppressing the transfer of non-radiative energy, eventually leading to the increase of diamond fluorescence emission intensity.