Layout/structure designs are encountered in a variety of fields. For problems with many design degrees of freedom, efficiency of design methods becomes a major concern. In recent years, machine learning methods such as artificial neural networks have been used increasingly to speed up the design process [1-3]. A main issue of many such approaches is the need for a large corpus of training data that are often generated using high-dimensional simulations. The high computational cost associated with training data generation largely diminishes the efficiency gained by using machine learning methods. In this work, an adaptive artificial neural network-based generative design approach is proposed and developed. This method uses a generative adversarial network to generate design candidates and thus the number of design variables is greatly reduced. To speed up the evaluation of the objective function, a convolutional neural network is constructed as the surrogate model for function evaluation. The inverse design is carried out using the genetic algorithm in conjunction with two neural networks. A novel adaptive learning and optimization strategy is proposed, which allows the design space to be effectively explored for the search for optimal solutions. As such the number of training data needed is greatly reduced. The performance of the proposed design method is demonstrated on two design problems: heat source layout design for thermal management and architectured material design for achieving near theoretical limit of elastic stiffness.