Item

Abstract

We introduce a new framework for regression between multi-dimensional spaces. Standard methods for solving this problem typically reduce the problem to one-dimensional
regression by choosing features in the input and/or output spaces. These methods, which
include PLS (partial least squares), KDE (kernel dependency estimation), and PCR
(principal component regression), select features based on different a-priori judgments as
to their relevance. Moreover, loss function and constraints are chosen not primarily on
statistical grounds, but to simplify the resulting optimisation. By contrast, in our
approach the feature construction and the regression estimation are performed jointly,
directly minimizing a loss function that we specify, subject to a rank constraint. A
major advantage of this approach is that the loss is no longer chosen according to the
algorithmic requirements, but can be tailored to the characteristics of the task at hand;
the features will then be optimal with respect to this objective. Our approach also
allows for the possibility of using a regularizer in the optimization. Finally, by processing the observations sequentially, our algorithm is able to work on large scale problems.