Multisource region attention network for fine-grained object recognition in remote sensing imagery

2019-07
Sümbül, Gencer
Cinbiş, Ramazan Gökberk
Aksoy, Selim
Show full item record
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 United States License.
452
views
146
downloads
Fine-grained object recognition concerns the identification of the type of an object among a large number of closely related subcategories. Multisource data analysis that aims to leverage the complementary spectral, spatial, and structural information embedded in different sources is a promising direction toward solving the fine-grained recognition problem that involves low between-class variance, small training set sizes for rare classes, and class imbalance. However, the common assumption of coregistered sources may not hold at the pixel level for small objects of interest. We present a novel methodology that aims to simultaneously learn the alignment of multisource data and the classification model in a unified framework. The proposed method involves a multisource region attention network that computes per-source feature representations, assigns attention scores to candidate regions sampled around the expected object locations by using these representations, and classifies the objects by using an attention-driven multisource representation that combines the feature representations and the attention scores from all sources. All components of the model are realized using deep neural networks and are learned in an end-to-end fashion. Experiments using RGB, multispectral, and LiDAR elevation data for classification of street trees showed that our approach achieved 64.2% and 47.3% accuracies for the 18-class and 40-class settings, respectively, which correspond to 13% and 14.3% improvement relative to the commonly used feature concatenation approach from multiple sources.
Deep learning, Fine-grained classification, Image alignment, Multisource classification, Object recognition
https://doi.org/10.1109/TGRS.2019.2894425
https://hdl.handle.net/11511/28659
IEEE Transactions On Geoscience And Remote Sensing
Department of Computer Engineering, Article

Suggestions

Fine-grained object recognition and zero-shot learning in multispectral imagery
Sumbul, Gencer; Cinbiş, Ramazan Gökberk; AKSOY, SELİM (2018-05-05)
We present a method for fine-grained object recognition problem, that aims to recognize the type of an object among a large number of sub-categories, and zero-shot learning scenario on multispectral images. In order to establish a relation between seen classes and new unseen classes, a compatibility function between image features extracted from a convolutional neural network and auxiliary information of classes is learnt. Knowledge transfer for unseen classes is carried out by maximizing this function. Per...
Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagery
Sumbul, Gencer; Cinbiş, Ramazan Gökberk; Aksoy, Selim (2018-02-01)
Fine-grained object recognition that aims to identify the type of an object among a large number of subcategories is an emerging application with the increasing resolution that exposes new details in image data. Traditional fully supervised algorithms fail to handle this problem where there is low betweenclass variance and high within-class variance for the classes of interest with small sample sizes. We study an even more extreme scenario named zero-shot learning (ZSL) in which no training example exists f...
Rescoring detections based on contextual scores in object detection
Zorlu, Ersan Vural; Akbaş, Emre; Department of Computer Engineering (2019)
To detect objects in an image, current state-of-the-art object detectors firstly definecandidate object locations, and then classify each of them into one of the predefinedcategories or as background. They do so by using the visual features extracted locallyfrom the candidate locations; omitting the rich contextual information embedded inthe whole image. Contextual information can be utilized to complement the informa-tion extracted locally and thereby to improve object detection accuracy. Researchershave p...
Weakly supervised instance attention for multisource fine-grained object recognition with an application to tree species classification
Aygunes, Bulut; Cinbiş, Ramazan Gökberk; Aksoy, Selim (2021-06-01)
Multisource image analysis that leverages complementary spectral, spatial, and structural information benefits fine-grained object recognition that aims to classify an object into one of many similar subcategories. However, for multisource tasks that involve relatively small objects, even the smallest registration errors can introduce high uncertainty in the classification process. We approach this problem from a weakly supervised learning perspective in which the input images correspond to larger neighborh...
Deep metric learning with distance sensitive entangled triplet losses
Karaman, Kaan; Alatan, Abdullah Aydın; Department of Electrical and Electronics Engineering (2021-2-12)
Metric learning aims to define a distance that is able to measure the semantic difference between the instances in a dataset. The most recent approaches in this area mostly utilize deep neural networks as their models to map the input data into a feature space by finding appropriate distance metrics between the features. A number of loss functions are already defined in the literature based on these similarity metrics to discriminate instances in the feature space. In this thesis, we particularly focus on t...
Citation Formats
G. Sümbül, R. G. Cinbiş, and S. Aksoy, “Multisource region attention network for fine-grained object recognition in remote sensing imagery,” IEEE Transactions On Geoscience And Remote Sensing, pp. 4929–4937, 2019, Accessed: 00, 2020. [Online]. Available: https://doi.org/10.1109/TGRS.2019.2894425.
METU IIS - Integrated Information Service open@metu.edu.tr