In order to build a robust background model and improve the accuracy of the foreground object detection, we give a comprehensive consideration on the same location pixels of the relevance of time and the correlation of space with its adjacent pixels; and based on the classic ViBe of random algorithm ideas, a kind of complex background model and foreground detection method is proposed. Using the first n series of images to initialize the background model with the sample consistency principle, we can avoid the appearance of the “Ghost” phenomenon; and get the difference between each pixel and its multiple sample value in the background model, and then compute the sum and the average. The average shows the dynamic degree of the background point which is the corresponding pixel background of dynamic feedback information. We get the adaptive clustering threshold and adaptive updating threshold with the dynamic feedback to make random clusters realize the adaptability to dynamic background and combine the global disturbance threshold with the local pixel level judgment threshold to implement the immunity of illumination with slow changes, fast changes or sudden changes, so that we can segment the prospect target accurately. By selecting neighborhood pixels to update the neighborhood background randomly in terms of spatial information dissemination mechanism, a good detection effect is obtained in the case of camera shake. Through multiple sets of test data, experimental results show that this algorithm can significantly improve the adaptability and robustness of the background model such as dynamic backgrounds, illumination changes, and camera shake. The algorithm can well apply to the occasion of moving targets in infrared image detection, and expand its application range. Without any image preprocessing and morphological post-processing, the original detection accuracy of foreground is superior to other algorithms.