Simulator Data Reduction
Michelle Reyes · 2011
Simulators produce a potentially overwhelming volume of data. The raw data generated by a simulator need to be parsed, aggregated, and combined to produce summary variables that relate to the theoretical constructs underlying the research questions that motivated the study. This process of transformation can be quite complex and error-prone, and manual reduction using a spreadsheet is often infeasible. Driving simulator studies provide a more detailed account of human behavior than many other experimental approaches. This chapter provides suggestions for exploring this rich source of information through three basic steps: planning, writing, and testing. Planning emphasizes a focus on data reduction throughout the entire research process that begins with links to the theoretical constructs being measured and manipulated in the study. Writing describes a series of tips to avoid common frustrations in developing code for data reduction. Testing advocates a systematic plan that includes automatic checks for the bounds of the reduced variables and visualization to identify unexpected failures of the software. The data reduction demands of simulator studies repeatedly frustrate both novice and experienced researchers. Data reduction requires the power of software and researchers often find themselves victims of the many pitfalls associated with developing software. Undiscovered errors in the data reduction process have the potential to invalidate a research program and undermine the collective understanding of driver behavior. Future trends toward standardized scenarios and measures might avoid many data reduction challenges, but such standardizations make it more likely for researchers to blindly interpret outcome variables without careful consideration for how they relate to the theoretical constructs of interest. The substantial differences between simulator platforms make it difficult for the content of this chapter to address the particular challenges any particular researcher will likely face. Data from different simulators reflect different underlying assumptions, definitions, and hardware configurations (e.g., eye tracker), but the general processes described in this chapter should help avoid the pitfalls commonly confronted when interpreting simulator data by increasing the opportunities for finding errors.