High dimensional density estimation with sparse grids

Abstract

Density estimation is a classical and well studied problem in modern statistics. In the case of low dimensional problems there are a wide array of well understood and efficient non-parametric methods in the existing literature. In contrast, high dimensional density estimation remains a very challenging problem in this field. As the dimension of the variable space increases there is an exponential growth in both the computational complexity of the problem and in the required sample size. This is a fundamental difficulty, and no single technique can avoid it altogether. Instead, techniques must be developed that take advantage of problem specific information or simplifying assumptions. In this thesis we study a particularly relevant situation in modern statistics, in which there is a large quantity of data and the underlying dimension of the problem is reasonably high. In this situation, the most flexible methods are too computationally expensive, and specification of reasonable parametric models becomes too difficult. We propose a range of techniques which utilize the concept of Sparse Grids. This is a technique that exploits additional regularity assumptions to allow a more efficient grid based representation of a function. This grid like approach allows an efficient treatment of massive data sets, while still possessing a reasonable degree of flexibility. We present different options for estimating density functions using a Sparse Grid. These estimators are examined first through a systematic study of the estimators theoretical properties, then through numerical examples of their real world performance and computational feasibility.