Evaluation of Artificial Intelligence Supported Osteoarthritis Information Texts: Content Quality and Readability Analysis
İlhan Celil Özbek · Fiziksel Tıp ve Rehabilitasyon Bilimleri Dergisi · 2025
ABS TRACT Objective: This study aimed to comprehensively evaluate the quality,readability,and understandability of artificial intelligence-supported texts related to osteoarthritis(OA).Material and Methods: The most frequently searched keywords related to osteoarthritis were identified using Google Trends.Additionally, frequently asked questions about osteoarthritis were identified.These keywords and questions were entered into Chat-GPT.The Ensuring Quality Information for Patients tool(EQIP) was used to assess the clarity of information and quality of writing.Flesch-Kincaid-readability-tests (Reading-Ease and Grade-Level) and the Gunning-Fog-Index (GFI) were used to assess the readability of the texts.The reliability and usefulness of the texts were assessed using the reliability and usefulness scale.Results: The average scores were: EQIP 62.01±6.61,FKRE 31.85±12.44,FKGL 13.26±2.12,GFI14.52±2.41,reliability 5.10±1.02,andusefulness 4.89±0.76.Our study concludes that ChatGPT responses to osteoarthritis are generally of "good-quality with minor-issues".Additionally, we determined that the texts produced were of complexity such that they would require approximately 13 years of education.When the EQIP score obtained from texts created using keywords was compared with the EQIP score obtained from texts created using questions,a statistically significant difference was observed (p<0.001).However, when examined in terms of FKRE, FKGL, GFI, reliability-scale, and usefulness-scale scores between the two groups, no statistically significant difference was found.(respectively, p=0.063, p=0.059, p=0.194, p=0,466, p=0,499).Conclusion: Thisstudy reveals that ChatGPT texts on OA have certain deficiencies in quality and readability.In conclusion, it emphasizes that online resources and AI tools play an important role in information provision in the field of healthcare,but quality and readability control should be ensured.In addition to ensuring patients have access to accurate, reliable, and understandable information, this can help them make more informed and effective health decisions by increasing their health literacy.