Frequent inventory data of container nurseries is needed by growers to ensure proper management and marketing strategies. In this paper, inventory data are estimated from aerial images. Since there are thousands of nursery species, it is difficult to find a generic classification algorithm for all cases. In this paper, the development of classification methods was confined to three representative categories: green foliage, yellow foliage, and flowering plants. Vegetation index thresholding and the support vector machine (SVM) were used for classification. Classification accuracies greater than 97% were obtained for each case. Based on the classification results, an algorithm based on canopy area mapping was built for counting. The effects of flight altitude, container spacing, and ground cover type were evaluated. Results showed that container spacing and interaction of container spacing with ground cover type have a significant effect on counting accuracy. To mimic the practical shipping and moving process, incomplete blocks with different voids were created. Results showed that the more plants removed from the block, the higher the accuracy. The developed algorithm was tested on irregular- or regular-shaped plants and plants with and without flowers to test the stability of the algorithm, and accuracies greater than 94% were obtained.