Using PSO algorithm for training neural networks

Artificial Neural Networks (ANN) have a wide range of applications in the field of science & technology. An attempt was made previously in a research study to train neural networks using the PSO algorithm. The main objective of this previous study was to predict the species based on flowers characteristics by using GPU Computing and CUDA programming language. However, some limitations were observed in the implementation of PSO on CPU and the GPU to obtain necessary accuracy on the data. This previous study provided lower accuracy of 0.337% (for both CPU and GPU) by using GPU kernels. Besides, all parallel parts of the PSO algorithm were not mapped to the GPU architecture and implemented as GPU kernels. The main objective of my project is to improve the CPU accuracy and to add GPU kernels. In the current study, greater accuracy was achieved in the C# implementation by improving the CPU random number generation that uses large prime numbers as seeds in the C++ implementation. Further I have also improved GPU implementation by incorporating an additional kernel and integrating GPU randomization to the existing kernel by using CuRAND library. The results from the study have achieved 0.87% CPU accuracy for training and testing data. Increasing the number of epochs resulted in reaching an optimum of 1.0% accuracy. GPU accuracy obtained for the same is about 0.67%. But, the GPU implementation does not provide a speedup compared to its CPU implementation for the neural network used in this study. To leverage the power of GPU, we require a network which needs more than 200 particles for its training. Hence, future studies may explore research on investigating such large neural networks by comparing CPU and GPU runtimes.