This study presents an isogeometric fast multipole boundary element method (IGA FMBEM) in acoustic problems and a related sensitivity-based shape optimization algorithm for acoustic structures. In the isogeometric analysis, Non-Uniform Rational B-Splines (NURBS) are used to accurately represent structural geometry. Refinements and shape changes for the design model are easily implemented without mesh regeneration, which largely reduces subsequent communication with the original description. The control points are set as design variables in the shape optimization procedure given that their variations can flexibly result in shape changes. For field variable approximation, the spline space is different from that for geometry representation. This separation provides the flexibility to choose a spline space more suitable to physical analysis. Acoustic shape sensitivities with respect to control points are calculated by the sensitivity boundary integral equation (BIE) based on the direct differentiation method. The singular integrals in the sensitivity BIEs are formulated explicitly under the isogeometric discretization. The minimization of sound pressure on the reference surface is selected as design objective. The gradient-based optimization solver is finally introduced for optimization iteration after the acoustic state and sensitivity information are obtained. The fast multipole method (FMM) is applied to improve overall computational efficiency. In order to conquer the fictitious eigenfrequency problem of the original boundary integral equation method in solving exterior acoustic problems, the Burton-Miller method is adopted in this study. The correctness and validity of the proposed methods are demonstrated through a number of numerical simulations, while the performance of the sensitivity-based optimization algorithm is observed in the shape optimization of several acoustic structures.