On Quantifying the Accuracy of Maximum Likelihood Estimation of Participant Reliability in Social Sensing
Dong Wang, Lance Kaplan, Charų C. Aggarwal · 2011
This paper presents a condence interval quantication of maximum likelihood estimation of participant reliability in social sensing applications. The work is motivated by the emergence of social sensing as a data collection paradigm, where humans perform the data collection tasks. A key challenge in social sensing applications lies in the uncertain nature of human measurements. Unlike well-calibrated and well-tested infrastructure sensors, humans are less reliable, and the likelihood that participants’ measurements are correct is often unknown a priori. Hence, it is hard to estimate the accuracy of conclusions made based on social sensing data. In previous work, we developed a maximum likelihood estimator of reliability of both participants and facts concluded from the data. This paper presents an analytically-founded bound that quanties the accuracy of such maximum likelihood estimation in social sensing. A condence interval is derived by leveraging the asymptotic normality of maximum likelihood estimation and computing the approximation of Cramer-Rao bound (CRB) for the estimation parameters. The proposed quantication approach is empirically validated and shown to accurately bound the actual estimation error given sucient number of participants under dierent sensing topologies.