On-Device Content Moderation
Anchal Pandey, Sukumar Moharana, Debi Prasanna Mohanty, Archit Panwar, Dewang Agarwal, Siva Prasad Thota · 2021
With the advent of internet, not safe for work (NSFW) content moderation is a major problem today. Since, smartphones are now part of daily life of billions of people, it becomes even more important to have a solution which could detect and suggest user about potential NSFW content present on their phone. In this paper we present a novel on-device solution for detecting NSFW images. In addition to conventional pornographic content moderation, we have also included semi-nude content moderation as it is still NSFW in a large demography. We have curated a dataset comprising of three major categories, namely nude, semi-nude and safe images. We have created an ensemble of object detector and classifier for filtering of nude and semi-nude contents. The solution provides unsafe body part annotations along with identification of semi-nude images. We extensively tested our proposed solution on several public dataset and also on our custom dataset. The model achieves F1 score of 0.91 with 95% precision and 88% recall on our custom NSFW_16k dataset and 0.92 MAP on NPDI dataset. Moreover it achieves average 0.002 false positive rate on a collection of safe image open datasets.