Neural networks are being used increasingly often as alternatives to traditional statistical models. As a result, their performance needs to be examined in a statistical framework. Following a brief overview of many types of neural networks, details concerning the implementation of the single hidden layer feedforward neural network (SHLFNN) are presented. The focus of the presentation is on the application of this network in a regression setting. One area where the SHLFNN is being used more frequently is in response surface modeling based on designed experiments. Due to the small sample sizes typically employed by response surface designs, the ability of the SHLFNN to accurately approximate the underlying model is questionable. The results of a simulation which compares the performance of the SHLFNN with that of a second order polynomial model are presented. Finally, methods are explored for combining the SHLFNN model with a linear model. Such a combined model has advantages over each of its components. The combined model will be able to approximate any underlying nonlinear function better than a linear model, and it will allow for easy assessment of the impact of any effects of interest to the researcher, an ability that is lost when only the SHLFNN model is used.