Mutual Information Decay Curves and Hyper-parameter Grid Search Design for Recurrent Neural Architectures
Citation:
Mahalunkar A., Kelleher J.D. (2020) Mutual Information Decay Curves and Hyper-parameter Grid Search Design for Recurrent Neural Architectures. In: Yang H., Pasupa K., Leung A.CS., Kwok J.T., Chan J.H., King I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1333. Springer, ChamDownload Item:
Abstract:
We present an approach to design the grid searches for hyper-parameter optimization for recurrent neural architectures. The basis for this approach is the use of mutual information to analyze long distance dependencies (LDDs) within a dataset. We also report a set of experiments that demonstrate how using this approach, we obtain state-of-the-art results for DilatedRNNs across a range of benchmark datasets.
Sponsor
Grant Number
Science Foundation Ireland (SFI)
13/ RC/2106
Author's Homepage:
http://people.tcd.ie/kellehjd
Author: Kelleher, John
Type of material:
Conference PaperCollections
Series/Report no:
Communications in Computer and Information Science;1333;
Availability:
Full text availableKeywords:
benchmark datasets, DilatedRNNs, Long Distance Dependencies, Recurrent neural architectures, Hyper-parameter tuning, Vanishing gradientsDOI:
http://dx.doi.org/10.1007/978-3-030-63823-8_70Metadata
Show full item recordLicences: