In recent years, the transmission security of image information has become an important research direction in the internet field. In this work, we propose a quantum image chaos encryption scheme based on quantum long-short term memory (QLSTM) network. We find that because the QLSTM network has a complex structure and more parameters, when the QLSTM network is used to improve the Lorenz chaotic sequence, its largest Lyapunov exponent is 2.5465% higher than that of the original sequence and 0.2844% higher than that the sequence improved by the classical long-short term memory (LSTM) network, while its result is closer to 1 and more stable in the 0–1 test. The improved sequence of QLSTM network has better chaotic performance and is predicted more difficultly, which improves the security of single chaotic system encryption. The original image is stored in the form of quantum states by using the NCQI quantum image representation model, and the improved sequence of QLSTM network is used to control the three-level radial diffusion, quantum generalized Arnold transform and quantum W-transform respectively, so that the gray value and pixel position of the quantum image are changed and the final encrypted image is obtained. The encryption scheme proposed in this work obtains the average information entropy of all three channels of RGB of greater than 7.999, the average value of pixel number change rate of 99.6047%, the average value of uniform average change intensity of 33.4613%, the average correlation of 0.0038, etc. In the test of statistical properties, the encryption scheme has higher security than some other traditional methods and can resist the common attacks.