The photoelectric mast equipped on the underwater vehicle is the key equipment for photoelectric tracking. While the vehicle moves under water, especially, at high speed, more complex vortexes are generated at the surface, which will give rise to great disturbance to the stability of optical axis. In this paper, firstly, based on the basic control equations of electromagnetic field and fluid mechanics, the effects of the Lorentz force on flow field structure and vortex induced vibration are numerical simulated with using the finite volume method with hierarchy grids. Secondly, the structural characteristics, transfer functions and PID control strategies of fast steering mirror (FSM) are analyzed. Finally, combining the transfer function of FSM and the force characteristics, the effect of the composite control on the stability of submarine photoelectric tracking system is discussed by MATLAB. The results show that the Lorentz force can adjust the boundary layer and suppress vortex induced vibration, based on which the FSM can be used to further improve the accuracy of the optical tracking system. This research offers a new exploration in the field of electromagnetic fluid control, as well as a novel development of the traditional research direction of fluid mechanics. Therefore it appears to have a certain scientific significance and practical value.