The geometry, electronic structure and magnetic property of boron nitride nanotube (BNNT), whose boron/nitride atoms are substituted by 5d atoms (B5d or N5d), are investigated by first-principles calculations based on density functional theory. The pure-BNNT and BNNT with boron vacancy (VB) or nitrogen vacancy (VN) are also investigated for comparison. Results show that the local symmetry of B5d system is similar to C3v, however the N5d system exhibits a large geometric deviation from C3v. The total magnetic moments of doped systems are different from each other, and B5d system present a strong regularity. The total density of states is presented, where impurity energy levels exist. The impurity energy levels and total magnetic moment can be explained by the molecular orbital theory under C3v symmetry.