Stress distribution of optical materials can be measured by using the laser feedback effect. Owing to the non-linear movement of the feedback mirror, the result accuracy of the system will decrease. In this work, we measure the displacement of the feedback mirror by using a high precision quasi-common-path laser feedback interferometry. The displacement-time function is obtained by a high-end fitting method. The stress measurement error induced by the non-linear movement of the feedback mirror is calculated according to the displacement-time function and a three-mirror cavity equivalent model, and the correction for the system accuracy is achieved. The results show that the different movement direction of the feedback mirror gives rise to an opposite variation trend of error. Measurement error can be reduced by averaging the results in different directions. In the study we analyze the influence of the non-linear movement of the feedback mirror on the measurement accuracy, and a method of improving the error is proposed. This method is significant for correcting the measurement results and improving the accuracy.