The Measured Words: How Computers Analyze Text
Geoffrey Rockwell, Stéfan Sinclair · The MIT Press eBooks · 2016
How exactly do computers handle texts? This second chapter is an introduction to how computers process texts as strings, or sequences of encoded characters. To understand what a computer can and can’t do one needs to understand how texts are represented and manipulated on computers as code. The chapter walks readers new to computer-assisted text analysis through how a computer can finds patterns in strings, can break a string into shorter tokens like words, and how tokens can be synthesized into new distributions of words in the form of concordances or visualizations. These analytical processes are not, as many argue, turning texts into numbers so much as different logical operations that split, enrich and recombine the texts for rereading. Thought of this way, the computer is not going to replace us so much as provide different views onto a text that can help with interpreting it.