A deception tracking system based on optical hidden visual code is proposed. The system uses visual cryptography to decompose the secret image into a number of realistic masked images, which can be used to conceal the secret information. One of the masked images is embedded with a fragile watermark to ensure that it is not modified. This image serves as an inspection key to verify the other images, and the inspection key is transmitted separately. The rest of the camouflaged image is hidden in the phase key using the phase recovery algorithm, which ensures good invisibility during transmission. If the pixel arrangement of the masked image is tampered with by a dishonest participant, it is called a fraudulent image. Each phase key is distributed to different participants to ensure that the corresponding deceiver can be identified when the spoofing image is traced. In the extraction process, only the diffraction transformation of the phase key is needed to obtain the mask image. During the inspection, the inspection key is incoherently superimposed with any masked image, and the appearance of the verification image indicates whether the masked image has been tampered with, thereby achieving the purpose of deception tracking. The secret image can be obtained by incoherently superimposing the masking images, provided that the number of superimposed masking images is is greater than or equal to the threshold k, along with the inspection key. When the inspection key is superimposed with any masked image, if there is a spoofed image, no verification image will appear, and as a result, the secret image will not be restored. If there is no spoofed image, the verification image will appear, indicating that the secret image can be restored by covering all the images. The system can be used to track internal fraudsters when actual information is transmitted through invisible visual cryptography.