OSIH-BERT: A Model for Predicting Sexual Health Information Based on Pre-trained Model and Voting Mechanism (Preprint)
Jie cheng, zhi-meng zhang, Ling-tao liu, Zhi-jian liu, Xing-zhi chen · 2025
BACKGROUND Adolescent sexual and reproductive health constitutes a critical component of the right to the highest attainable standard of physical and mental health for all[1].However, adolescent sexual and reproductive health rights (ASRHR) are currently faced with severe challenges[2], According to reports by Together for Girls, 82 million girls and 69 million boys experienced some form of child sexual violence in 2024[3];the World Health Organization (WHO) reported that approximately 190 million individuals (25%) may experience physical or sexual violence from intimate partners before the age of 20[4]. Adolescents often seek sexual health-related assistance through question texts on internet platforms[5],rendering the mining of user-generated texts by adolescents a proactive role in the prevention and intervention of adolescent sexual violence. Nevertheless, as the first step in mining, classifying sexual health information remains difficult for existing general models to address effectively due to the complex semantics[6]and elusiveness[7] of sex-related information, thus creating an urgent need for the construction of a targeted specialized model. Text classification in the field of natural language processing (NLP) enables the organization of large volumes of unstructured text data into specific categories, providing technical support for identifying sexual health needs. The evolution of text classification systems has advanced from information retrieval[8-10], [11], [12, 13], recommendation systems [14], sentiment analysis [15, 16], and public opinion analysis [17].Pre-trained models (PTMs) [18] to information filtering[11],text categorization[12, 13],recommendation systems [14],sentiment analysis[15, 16], and public opinion analysis[17]. PTMs[18] a further advancement in text classification, have developed from word embeddings to contextual encoders, addressing the challenges of deep learning models that require large-scale data to prevent overfitting due to their numerous parameters[19].Unlike traditional unidirectional models that capture text information solely from left-to-right or right-to-left, Bert-based[20] PTMs employ bidirectional training, enabling simultaneous consideration of contextual information from both sentence directions and more accurate and comprehensive semantic understanding of vocabulary. In the realm of sexual health information, classifying related texts carries significant implications. GM Barrientos et al.[6] utilized machine learning techniques to categorize pornographic/sexual content questions online, aiming to protect children from harm. TT Nguyen et al.[21] applied large language models (LLMs) to identify sexual predators in online discussions and comments, contributing to a safer internet environment. N Potha et al.[22] used data mining techniques on online platforms to recognize instances of sexual violence and abuse against adolescents, thereby safeguarding ASRHR. These examples demonstrate that deep learning and other technologies can be leveraged to achieve the goal of protecting ASRHR. Although text classification technologies have made progress in the medical and health field, numerous challenges persist. For instance, sexual health information from diverse sources varies significantly in language expression and professional complexity, which existing classification models struggle to fully adapt to[23] . Additionally, the unique needs and psychological characteristics of adolescents have not been adequately considered[24].Therefore, this study aims to propose a specialized model for binary classification of adolescents’ online consultation data and question texts to identify online inquiries related to sexual health (OISH), providing model and data support for subsequent research. This approach not only enhances the efficiency and accuracy of screening sexual health information but also assists healthcare workers and educators in promptly identifying adolescents’ sexual health issues, thereby promoting their physical and mental well-being. OBJECTIVE Globally, sexual violence against adolescents occurs frequently, and their adolescent sexual and reproductive health rights (ASRHR) face significant challenges. METHODS Consultation data and question texts were crawled from "Xunyi Wenyao" and "Youwen Bida" using the "Octopus" crawler program. After data processing, domain experts were invited to participate in manual annotation. Multiple pre-trained models (PTMs) were trained, and following performance evaluation, a model was constructed by integrating a voting mechanism. Performance evaluations were conducted on both the training dataset and the adolescent dataset. RESULTS OSIH-BERT integrates ERNIE 3.0, rbt6, and Bert-base-chinese through a voting mechanism. Compared with a single PTM, this model has approximately a 2% performance improvement. CONCLUSIONS This model demonstrates high accuracy and discriminative ability in the classification task of adolescent sexual and reproductive health-related online inquiries (OSIH). It can be used to classify adolescents' questions and requests regarding sexual health, providing technical support for the protection of ASRHR. CLINICALTRIAL none