Integrating Crowd and Machine Learning in an Intelligent Interface: A Case Study of Oil Spill Detection in Satellite Images
Rifat Mehreen Amin, Feng Chen, Linda S. Hirsch, Changkun Ou, Tran Vu La, Andreas Martin Butz · 2024
Object detection tasks still often require manual image analysis. Using Machine Learning (ML) instead creates accountability challenges, necessitating experts for model refinement, which is costly and takes time. We investigate integrating crowd knowledge as a cost-effective alternative. While human capabilities in recognizing complex patterns and perceiving variations can still outperform machines and improve an imperfect ML model, ML predictions can compensate for the crowd’s lack of expertise. Our investigation (N=28 non-expert) in oil spill detection shows that adopting an ML-assisted UI elevates precision and recall by over 11% and increases efficiency by 29% compared to a non-assisted UI. Considering agreement among non-expert crowd workers further improved precision by 8% and recall by almost 5%, which is also substantially beyond pure ML performance. Our work contributes an approach for combining crowd knowledge and ML to advance human-AI collaboration in oil spill detection.