Abstract
This book presents a novel image representation that allows to access natural scenes by local semantic description. During semantic modeling, local image regions are classified into semantic concepts
classes such as water, rocks, and foliage. Images are represented through the frequency of occurrence of the local semantic concepts. This image representation is demonstrated to be well suited for modeling the semantic content of heterogeneous scene categories,
and thus for categorization and retrieval. Furthermore, the image
representation based on semantic modeling qualifies for ranking
natural scenes according to their semantic similarity. This
application is of special interest for content-based image retrieval systems that rely on the correct ordering of the returned images.
In two psychophysical experiments, the human perception of the employed natural scenes has been studied. A categorization and a typicality ranking experiment showed that humans are very consistent in classifying scenes and in rating their semantic typicality with
respect to five scene categories. Based on these findings, a novel
perceptually plausible distance measure is introduced that allows to
automatically rank natural scenes with a high correlation to the human
ranking.
Finally, the work discusses the problem of performance evaluation in content-based image retrieval systems. When searching for specific local semantic content, the retrieval results can be modeled statistically. Closed-form expressions for the prediction and the
optimization of retrieval precision and recall are developed that permit to optimize precision and recall by up to 60.
