A Proposed S.C.O.R.E. Evaluation Framework for Large Language Models : Safety, Consensus, Objectivity, Reproducibility and Explainability
Ting Fang Tan, Kabilan Elangovan, Jasmine Chiat Ling Ong, Nigam Haresh Shah, Joseph J.�Y. Sung, Tien Yin Wong, Xue, Lan, Nan Liu, Haibo Wang, Chang Fu Kuo, Simon Arthur Chesterman, Zee Kin Yeong, Daniel Shu Wei Ting · arXiv (Cornell University) · 2024
A comprehensive qualitative evaluation framework for large language models (LLM) in healthcare that expands beyond traditional accuracy and quantitative metrics needed. We propose 5 key aspects for evaluation of LLMs: Safety, Consensus, Objectivity, Reproducibility and Explainability (S.C.O.R.E.). We suggest that S.C.O.R.E. may form the basis for an evaluation framework for future LLM-based models that are safe, reliable, trustworthy, and ethical for healthcare and clinical applications.