Two-dimensional WTe ₂possesses a special crystal symmetry, leading to novel properties such as quantum spin Hall effect and nonlinear Hall effect. Determining the details of its crystal structure is essential for understanding these interesting properties. Here, we report an optical study on the crystal symmetry of monolayer, bilayer, and trilayer WTe ₂, using temperature and polarization dependent Raman spectroscopy and optical second harmonic generation (SHG). We find that monolayer WTe ₂is noncentrosymmetric as indicated by its sizable SHG, in contrast to the commonly believed centrosymmetric 1 T'structure. The polarization dependence of the SHG is consistent with the C _spoint group. Bilayer WTe ₂exhibits SHG signal more than one order of magnitude higher than in the monolayer and trilayer samples, with its temperature dependence reflecting the ferroelectric phase transition, evidencing strong inversion symmetry breaking induced by layer stacking and interlayer-sliding ferroelectricity. We also observe prominent second-order resonant Raman scattering peaks only in monolayer and bilayer WTe ₂, but not in thicker samples, and their temperature dependence indicates an electronic structure highly sensitive to interlayer coupling. These results will be useful for further exploring the properties of atomically thin WTe ₂.