Representation Learning on Large and Small Data
Chun‐Nan Chou, Chuen‐Kai Shie, Fu‐Chieh Chang, Jocelyn J. Chang, Edward Y. Chang · 2019
Extracting useful features from a scene is an essential step in any computer vision and multimedia data analysis task. The approaches in feature extraction can be divided into two categories: model-centric and data-driven. This chapter focuses on how neural networks, specifically convolutional neural networks (CNNs), achieve effective representation learning. It reviews representative CNN models proposed since 2012. The chapter deals with the small data problem. It presents how features learned from one source domain with big data can be transferred to a different target domain with small data. Deep learning has its roots in neuroscience. CNNs are composed of two major components: feature extraction and classification. The common practice of transfer representation learning is to pretrain a CNN on a very large dataset and then to use the pretrained CNN as either an initialization or a fixed feature extractor for the task of interest.