Efficient learning in boltzmann machines using linear response theory
Entity
UAM. Departamento de Ingeniería InformáticaPublisher
M I T PressDate
1998-07-01Citation
10.1162/089976698300017386
Neural Computation 10.5 (1998): 1137–1156
ISSN
0899-7667 (print); 1530-888X (online)DOI
10.1162/089976698300017386Editor's Version
http://www.mitpressjournals.org/doi/abs/10.1162/089976698300017386Subjects
InformáticaRights
© 1998 Massachusetts Institute of TechnologyAbstract
The learning process in Boltzmann machines is computationally very expensive. The computational complexity of the exact algorithm is exponential in the number of neurons. We present a new approximate learning algorithm for Boltzmann machines, based on mean-field theory and the linear response theorem. The computational complexity of the algorithm is cubic in the number of neurons.
In the absence of hidden units, we show how the weights can be directly computed from the fixed-point equation of the learning rules. Thus, in this case we do not need to use a gradient descent procedure for the learning process. We show that the solutions of this method are close to the optimal solutions and give a significant improvement when correlations play a significant role. Finally, we apply the method to a pattern completion task and show good performance for networks up to 100 neurons.
Files in this item
Google Scholar:Kappen, Hilbert Johan
-
Rodríguez Ortiz, Francisco Borja
This item appears in the following Collection(s)
Related items
Showing items related by title, author, creator and subject.