Taking into account the location, size and also the shape of the defector, we calculated the values of the selectivity and the effective transmittance solid angle of the hexapole magnet system, and compared the results with those of the previous method. The dependence of the selectivity and the effective transmittance solid angle on various experimental parameters, and the ways to improve the performance of the hexapole magnet system were discussed. The selectivity was measured by laser-induced fluorescence (LIF) and the results were in good agreement with the theoretical ones.