Textual Data
Pablo Ariel Duboue · Cambridge University Press eBooks · 2020
Extending the dataset from chapter 6 with the full Wikipedia page for each settlement, this chapter exemplifies natural language processing techniques to work with textual data. Textual problems are a domain that involves large number of correlated features, with feature frequencies strongly biased by a power law. Such behaviour is very common for many naturally occurring phenomena besides text. The very nature of dealing with sequences means this domain also involves variable length feature vectors. A central theme in this chapter is context and how to supply it within the enhanced feature vector to increase the signal-to-ratio available to the ML. As many times the target information (population) appears within the target page (as high as 53% of the cases in exact form), this problem is closely related to the NLP problem of Information Extraction. The chapter showcases different ways to approach the problem using words-as-features in the bag-of-words paradigm, then proceeding to do heavy feature selection using mutual information, stemming, modelling context with bigrams and skip-bigrams. More advanced topics include feature hashing and embeddings using word2vec.