An agenda for addressing bias in conflict data
Erin L. Miller, Roudabeh Kishi, Clionadh Raleigh, Caitriona Dowd · Scientific Data · 2022
With increased availability of disaggregated conflict event data for analysis, there are new and old concerns about bias. All data have biases, which we define as a systematic inclination, prejudice, or directionality to information. Bias occurs when a dataset deviates from a pure/comprehensive model of reality in non-random ways that may produce misleading and harmful inferences if not accounted for properly. In conflict data, skepticism about potentially damaging biases can cause doubt about whether data collection procedures create systematic omissions, inflations, or misrepresentations due to the aforementioned prejudices or directionality. As curators and analysts of multiple large, popular data projects, we are uniquely aware of biases that are present when collecting and using event data. We have observed that researchers can significantly misinterpret the effects of biases, both overstating and understating the significance of certain biases. We contend that it is necessary to advance a more nuanced discussion that goes beyond the question of whether biases exist, and articulate the responsibilities of everyone in the data ecosystem – collectors, researchers, and those interpreting and applying findings – to more thoughtfully reflect on potential biases, use data in good faith, and acknowledge limitations of data collection and critical interpretation.