Ionescu R.T., Popescu M. Knowledge Transfer between Computer Vision and Text Mining. Similarity-based Learning Approaches

Springer, 2016. — 265.Machine learning is currently a vast area of research with applications in a broad range of fields such as computer vision, bioinformatics, information retrieval, natural language processing, audio processing, data mining, and many others. Among the variety of state-of-the-art machine learning approaches for such applications are the similarity-based learning methods. Learning based on similarity refers to the process of learning based on pairwise similarities between the training samples. The similarity-based learning process can be both supervised and unsupervised, and the pairwise relationship can be either a similarity, a dissimilarity, or a distance function.
This book studies several similarity-based learning approaches, such as nearest neighbor models, local learning, kernel methods, and clustering algorithms. A nearest neighbor model based on a novel dissimilarity for images is presented in this book. It is used for handwritten digit recognition and achieves impressive results. Kernel methods are used in several tasks investigated in this book. First, a novel kernel for visual word histograms is presented. It achieves state-of-the-art performance for object recognition in images. Several kernels based on a pyramid representation are presented next. They are used for facial expression recognition from static images. The same pyramid representation is successfully used for text categorization by topic. Moreover, an approach based on string kernels for native language identification is also presented in this work. The approach achieves state-of-the-art performance levels, while being language independent and theory neutral. An interesting pattern can already be observed, namely that the machine learning tasks approached in this book can be divided into two different areas: computer vision and string processing.
Despite the fact that computer vision and string processing seem to be unrelated fields of study, image analysis and string processing are in some ways similar. As will be shown by the end of this book, the concept of treating image and text in a similar fashion has proven to be very fertile for specific applications in computer vision. In fact, one of the state-of-the-art methods for image categorization is inspired by the bag of words representation, which is very popular in information retrieval and natural language processing. Indeed, the bag of visual words model, which builds a vocabulary of visual words by clustering local image descriptors extracted from images, has demonstrated impressive levels of performance for image categorization and image retrieval. By adapting string processing techniques for image analysis or the other way around, knowledge from one domain can be transferred to the other. In fact, many breakthrough discoveries have been made by transferring knowledge between different domains. This book follows this line of research and presents novel approaches or improved methods that rely on this concept. First of all, a dissimilarity measure for images is presented. The dissimilarity measure is inspired by the rank distance measure for strings. The main concern is to extend rank distance from one-dimensional input (strings) to two-dimensional input (digital images). While rank distance is a highly accurate measure for strings, the empirical results presented in this book suggest that the proposed extension of rank distance to images is very accurate for handwritten digit recognition and texture analysis. Second of all, a kernel that stems from the same idea is also presented in this book. The kernel is designed to encode the spatial information in an efficient way and it shows performance improvements in object class recognition and text categorization by topic. Third of all, some improvements to the popular bag of visual words model are proposed in the present book. As mentioned before, this model is inspired by the bag of words model from natural language processing and information retrieval. A new distance measure for strings is introduced in this work. It is inspired by the image dissimilarity measure based on patches that is also described in the present book. Designed to conform to more general principles and adapted to DNA strings, it comes to improve several state-of-the-art methods for DNA sequence analysis. Furthermore, another application of this novel distance measure for strings is discussed. More precisely, a kernel based on this distance measure is used for native language identification. To summarize, all the contributions presented in this book come to support the concept of treating image and text in a similar manner.
It is worth mentioning that the studied methods exhibit state-of-the-art performance levels in the approached tasks. A few arguments come to support this claim. First of all, an improved bag of visual words model described in this work obtained the fourth place at the Facial Expression Recognition (FER) Challenge of the ICML 2013 Workshop in Challenges in Representation Learning (WREPL). Second of all, the system based on string kernels presented in this book ranked on third place in the closed Native Language Identification Shared Task of the BEA-8 Workshop of NAACL 2013. Third of all, the PQ kernel for visual word histograms described in this work received the Caianiello Best Young Paper Award at ICIAP 2013. Together, these achievements reflect the significance of the methods described in the present book.Motivation and Overview
Learning based on Similarity
Part I Knowledge Transfer from Text Mining to Computer Vision
State-of-the-Art Approaches for Image Classification
Local Displacement Estimation of Image Patches and Textons
Object Recognition with the Bag of Visual Words Model
Part II Knowledge Transfer from Computer Vision to Text Mining
State-of-the-Art Approaches for String and Text Analysis
Local Rank Distance
Native Language Identification with String Kernels
Spatial Information in Text Categorization
Conclusions