The conflicts between pedestrians and vehicles at unsignalized intersections are rather dangerous. Besides the motion features, pedestrians and vehicle drivers also have behavioral features, so modelling the crossing behaviors of pedestrians and vehicles from the sociological aspect is necessary. Through an in-depth analysis of the interference mechanism between pedestrians and vehicles at unsignalized intersections, in this paper, a dirty faces game to analyze the crossing behaviors of pedestrians and vehicles as “rational man” is proposed. The formation process of common knowledge, the producing mechanism of dominant strategy and the kinetic process of dirty faces game are analyzed. Under the assumption of probability of the dominant strategy and distribution of crossing times, theoretical derivations of the benefits gained by pedestrians and vehicles as well as the conflict probability are given. Results show that a collision is more likely to occur when the theoretical time of pedestrian passing through is closer to that of the vehicle driver, and this is consistent with the actual situation. Furthermore, the influences of vehicle type, waiting time and heterogeneity of pedestrians on the conflict probability are also considered.