Deep Learning in the Presence of Label Noise: A Meta-Learning Approach

Download

thesis(2).pdf

Date

2021-3-12

Author

Algan, Görkem

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

3272
views

33
downloads

Image classification systems recently made a giant leap with the advancement of deep neural networks. However, these systems require an excessive amount of labeled data to be adequately trained. Gathering a correctly annotated dataset is not always feasible due to practical challenges. Because of these practical challenges, label noise is a common problem in real-world datasets. This thesis presents two novel label noise robust learning algorithms: MSLG (Meta Soft Label Generation) and MetaLabelNet. Both algorithms are powered by meta-learning techniques and share the same learning framework. Proposed algorithms generate soft labels for each instance according to a meta-objective, which is to minimize the loss on the small meta-data. Afterward, the main classifier is trained on these generated soft-labels instead of given noisy labels. In each iteration, before conventional learning, the proposed meta objective reshapes the loss function so that resulting gradient updates would lead to model parameters with the minimum loss on meta-data. Different from MSLG, MetaLabelNet can work with dataset consists of both noisily labeled and unlabeled data, which is a problem setup that is not considered in the literature up to now. To prove the validity of the proposed algorithms, they are backed with mathematical justification. Extensive experiments on datasets with both synthetic and real-world label noises show the superiority of the proposed algorithms. For comparison with the state-of-the-art methods, proposed algorithms are tested on widely used noisily labeled benchmarking dataset Clothing1M. Both algorithms beat the baseline methods with a large margin, where MSLG achieves 2.3\% and MetaLabelNet achieves 4.2\% higher than the closest method. Results show that presented approaches are fully implementable for real-world use cases. Additionally, a novel label noise generation algorithm is presented for the purpose of generating realistic synthetic label noise.

Subject Keywords

Deep learning, Label noise, Noise robust, Noise cleaning, Meta-learning

URI

https://hdl.handle.net/11511/89551

Collections

Graduate School of Natural and Applied Sciences, Thesis

Suggestions

OpenMETU
Core

Image classification with deep learning in the presence of noisy labels: A survey Algan, Gorkem; Ulusoy, İlkay (2021-03-01) Image classification systems recently made a giant leap with the advancement of deep neural networks. However, these systems require an excessive amount of labeled data to be adequately trained. Gathering a correctly annotated dataset is not always feasible due to several factors, such as the expensiveness of the labeling process or difficulty of correctly classifying data, even for the experts. Because of these practical challenges, label noise is a common problem in real-world datasets, and numerous metho...
MetaLabelNet: Learning to Generate Soft-Labels From Noisy-Labels Algan, Gorkem; Ulusoy, İlkay (2022-01-01) Real-world datasets commonly have noisy labels, which negatively affects the performance of deep neural networks (DNNs). In order to address this problem, we propose a label noise robust learning algorithm, in which the base classifier is trained on soft-labels that are produced according to a meta-objective. In each iteration, before conventional training, the meta-training loop updates soft-labels so that resulting gradients updates on the base classifier would yield minimum loss on meta-data. Soft-labels...
Deep Learning-Based Hybrid Approach for Phase Retrieval IŞIL, ÇAĞATAY; Öktem, Sevinç Figen; KOÇ, AYKUT (2019-06-24) We develop a phase retrieval algorithm that utilizes the hybrid-input-output (HIO) algorithm with a deep neural network (DNN). The DNN architecture, which is trained to remove the artifacts of HIO, is used iteratively with HIO to improve the reconstructions. The results demonstrate the effectiveness of the approach with little additional cost.
Representing temporal knowledge in connectionist expert systems Alpaslan, Ferda Nur (1996-09-27) This paper introduces a new temporal neural networks model which can be used in connectionist expert systems. Also, a Variation of backpropagation algorithm, called the temporal feedforward backpropagation algorithm is introduced as a method for training the neural network. The algorithm was tested using training examples extracted from a medical expert system. A series of experiments were carried out using the temporal model and the temporal backpropagation algorithm. The experiments indicated that the alg...
Position estimation for timing belt drives of precision machinery using structured neural networks KILIÇ, Ergin; DOĞRUER, CAN ULAŞ; Dölen, Melik; Koku, Ahmet Buğra (2012-05-01) This paper focuses on a viable position estimation scheme for timing-belt drives using artificial neural networks. In this study, the position of a carriage (load) is calculated via a structured neural network topology accepting input from a position sensor on the actuator side of the timing belt. The paper presents a detailed discussion on the source of transmission errors. The characteristics of the error in different operation regimes are exploited to construct different network topologies. That is, a re...

Citation Formats

G. Algan, “Deep Learning in the Presence of Label Noise: A Meta-Learning Approach,” Ph.D. - Doctoral Program, Middle East Technical University, 2021.