Storing Data in Files
Nils B. Weidmann · Cambridge University Press eBooks · 2023
Storing Data in FilesIn this and the next chapters of this book, we will focus on tools for data contained in files: Your data resides in physical files on your hard disk, from where it is opened with a software of your choice, processed in various ways, and then stored again in a file.This is by far the most common workflow used in social science projects.Why do we need files at all?The answer is very simple: We use files for permanent data storage.When you work with a dataset in R (or in some other software, such as Excel or Stata), the table(s) -for example, the data frames in R -are temporarily stored in your computer's main memory.This is the part of your computer where data and programs are kept for fast access during the actual operation of your system.The problem is that this volatile memory does not function anymore when you turn off your system, and the entire content (and, thus, your data) disappears.Therefore, every computer has another type of data storage that remains persistent even when the system is shut down.This is usually your hard disk drive, but it can also be a network drive or a cloud storage folder.When we save tabular data contained in the computer's main memory to files, we need to make sure that the tabular structure is preserved.Recall our discussion of the importance of data structure in the initial chapters of this book -a persistent storage of data in files would be useless if the actual structure of the data were lost.Therefore, there are different ways in which tabular data can be stored as files, such that the tabular structure is preserved.For a given file, the file type typically indicates if it contains a data table and how this table is stored in the file.You are probably familiar with file types for text documents (e.g., the Word format indicated by the .docxextension) or for graphics (e.g., the JPEG format using the .