Developing Indonesian Medical Corpora Using the Latent Dirichlet Allocation Method and Filtering Out Non-Medical Terms and Non-Noun Words
Mohammad Teduh Uliniansyah, Agung Santosa, Dian Isnaeni Nurul Afra, Siska Pebiana, Lyla Ruslana Aini, Elvira Nurfadhilah, Desiani, Gunarso, Asril Jarin, Hammam Riza · 2023
As part of the research collaboration among BRIN, Solusi247, and the Harapan Kita Heart and Blood Vessel Hospital to develop an Indonesian medical speech recognition system that requires a medical text corpus, we have collected 300,063 medical Q&A articles (HTML files) about various diseases. This study is a preliminary step before developing the medical speech recognition system. As is known, a symptom of a disease can also be a symptom of another disease. For example, tiredness can be a flu or low blood pressure symptom. We intended to create medical corpora by clustering the medical documents according to the type of disease being conversed. The clustering was done using the Latent Dirichlet Allocation (LDA) method and the c_v measure. Before the clustering, we modified the articles by normalizing the medical terms and treating certain compound words as a single word. In the experiments, we used stop words containing non-medical terms and other stop words consisting of non-nouns. The experimental results showed that, on average, the c_v coherence score of the modified corpus is better than the original corpus, with a mean difference of 0.0122 and 0.0197, showing that the modification has a good impact.