Strength and Limitations of Shallow Networks. Tutorial

0453644 - ÚI 2016 CH eng A - Abstrakt
Kůrková, Věra
Strength and Limitations of Shallow Networks. Tutorial.
Engineering Applications of Neural Networks. Cham: Springer, 2015 - (Iliadis, L.; Jayne, C.). XXIV-XXIV. ISBN 978-3-319-23981-1. ISSN 1865-0929. E-ISSN 1865-0937.
[EANN 2015. International Conference /16./. 25.09.2015-28.09.2015, Rhodes]
Institucionální podpora: RVO:67985807
Kód oboru RIV: IN - Informatika

Although originally biologically inspired neural networks were introduced as multilayer computational models, later shallow (one-hidden-layer) architectures became dominant in applications. Recently, interest in architectures with several hidden layers was renewed due to successes of deep convolutional networks. These experimental results motivated theoretical research aiming to characterize tasks which can be computed more efficiently by deep networks than by shallow ones. This tutorial will review recent developments regarding theoretical analysis of strength and limitations of shallow networks. The tutorial will focus on the following topics: Universality and tractability of representations of multivariable mappings by shallow networks, Trade-off between maximal generalization capability and model complexity, Limitations of computation of highly-varying functions by shallow networks, Probability distributions of functions which cannot be tractably represented by shallow networks, Examples of representations of high-dimensional classification tasks by one and two-hidden-layer networks. Attendees will learn about consequences of these theoretical results for the methodology of choosing a neural network architecture and about open problems related to deep and shallow architectures.
Trvalý link: http://hdl.handle.net/11104/0254404