The fractal is a kind of geometric figure with self-similar character. Phase transition and critical phenomenon of spin model on fractal lattice have been widely studied and many interesting results have been obtained. The ${S^4}$ model regarded as an extension of the Ising model, can take a continuous spin value. Research of the ${S^4}$ model can give a better understanding of the phase transition in the real ferromagnetic system in nature. In previous work, the phase transition of the ${S^4}$ model on the translation symmetry lattice has been studied with the momentum space renormalization group technique. It is found that the number of the fixed points is related to the space dimensionality. In this paper, we generate a family of diamond hierarchical lattices. The lattice is a typical inhomogenous fractal with self-similar character, whose fractal dimensionality and the order of ramification are ${d_{\rm{f}}} = {\rm{1}} + \ln m/\ln {\rm{3}}$ and $R = \infty $ , respectively. In order to discuss the phase transition of the ${S^4}$ model on the lattice, we assume that the Gaussian distribution constant ${b_i}$ and the fourth-order interaction parameter ${u_i}$ depend on the coordination number ${q_i}$ of the site on the fractal lattices, and the relation ${b_i}/{b_j} = {u_i}/{u_j} = {q_i}/{q_j}$ is satisfied. Using the renormalization group and the cumulative expansion method, we study the phase transition of the ${S^4}$ model on a family of diamond lattices of $m$ branches. Removing the inner sites, we obtain the system recursion relation and the system corresponding critical point. Furthermore, we find that if the number of branches is $m = 2$ or $m > {\rm{1}}2$ (fractal dimensionality ${d_{\rm{f}}} = {\rm{1}}{\rm{.63}}$ or ${d_{\rm{f}}} > {\rm{3}}{\rm{.26}}$ ), the system only has the Gaussian fixed point of ${K^ * } = {b_2}/2$ , $u_2^ * = 0$ . The critical point of the system is in agreement with that from the Gaussian model on the fractal lattice, which predicts that the two systems belong to the same university class. We also find that under the condition of ${\rm{3}} \leqslant m \leqslant {\rm{1}}2$ (fractal dimensionality ${\rm{2}} \leqslant {d_{\rm f}} \leqslant {\rm{3}}{\rm{.26}}$ ), both the Gaussian fixed point and the Wilson-Fisher fixed point can be obtained in the system, and the Wilson-Fisher fixed point plays a leading role in the critical properties of the system. According to the real space renormalization group transformation and scaling theory, we obtain the critical exponent of the correlation length. Finally, we find that the critical points of the ${S^4}$ model on a family of diamond lattices depend on the value of the fractal dimensionality. The above result is similar to that obtained from the ${S^4}$ model on the translation symmetry lattice.