When intense laser pulse irradiates a target surface, the energetic processes of generation and expansion of laser-induced plasma will affect a localized pressure impulse around the irradiation zone. As a result, pulsed laser ablating granular target can drive a physical phenomenon of grain ejection. In this work, taking dry glass beads with different grain sizes as an example of granular targets and using high-speed video camera, we experimentally investigate the grain-size dependent dynamics of grain ejection driven by nanosecond laser pulses. The measured video sequences clearly exhibit that the laser-driven grain ejection process can be separated into two regimes: early-stage fast ejecting process and later-stage slow ejecting process. We find that there exists an obvious grain size effect on the kinetic energy of grains in the early-stage ejecting process. In addition, the temporal evolution of transient ejection of a curtain diameter $ D\left(t\right) $ corresponding to the later-stage ejecting process obeys the well-known “point source” law, $ {D\left(t\right)=\alpha t}^{\beta } $ , where both parameters $ \alpha $ and $ \beta $ depend on grain size. The observations mentioned above can be reasonably explained by considering the grain size dependent efficiency of impulse coupling between grain and plasma flow and plasma features generated by interaction of laser pulse with granular targets. These experimental results improve the understanding of the mechanism of laser-driven grain ejection.