Structural damage detection is a key part for effective structural health monitoring in engineering, which can ensure the safety and reliability of a structure system in its service time, Various methods have been proposed for structural damage detection.  Here the damage detection method which combines FEM modeling and convolutional neural network (CNN) is presented. The vibration mode shapes, mode curvature differences, modal strain energy, etc can be obtained by FEM. The CNN can be trained by FEM data and tested against experimental results. We show that the method can give accurate prediction results on damage locations and levels.