Design of Scoring Models for Trustworthy Risk Prediction in Critical Patients
Paolo Barbini, Gabriele Cevenini · InTech eBooks · 2011
Prediction of an adverse health event (AHE) from objective data is of great importance in clinical practice.A health event is inherently dichotomous as it either happens or does not happen, and in the latter case, it is a favourable health event (FHE).In many clinical applications, it is relevant not only to predict AHEs happening (diagnostic ability) but also to estimate in advance their individual risk of occurrence using ordered multinomial or quantitative scales (prognostic ability) such as probability.An estimated probability of a patient's outcome is usually preferred to a simpler binary decision rule.H o w e v e r , m o d e l s c a n n o t b e d e s i g n e d b y o p t i mising their fit to true individual risk probabilities because the latter are not intrinsically known, nor can they be easily and accurately associated with an individual's data.Classification models are therefore usually trained on binary outcomes to provide an orderable or quantitative output, which can be dichotomised using a suitable cut-off value.Model discrimination refers to accurate identification of actual outcomes.Model calibration, or goodness of fit, is related to the agreement between predicted probabilities and observed proportions and it is an important aspect to consider in evaluating the prognostic capacity of a risk model (Cook, 2008).Model calibration is independent of discrimination, since there are risk models with good discrimination but poor calibration.A well-calibrated model gives probability values that can be reliably associated with the true individual risk of outcomes.Many models have recently been proposed for diagnostic purposes in a wide range of medical applications and they also provide reliable estimates of individual risk probabilities.Two different approaches have been used to predict patient risk.The first approach is based on estimation of risk probability by sophisticated mathematical and statistical methods, such as logistic regression, the Bayesian rule and artificial neural networks (Dreiseitl & Ohno-Machado, 2002;Fukunaga, 1990;Marshall et al., 1994).Despite their great accuracy, these models are unfortunately not widely used because they are hard to design and call for difficult calculations, often requiring dedicated software and computing knowledge that doctors do not welcome, besides being difficult to incorporate in clinical practice.The second approach creates scoring systems, in which the predictor variables are usually selected and scored subjectively by expert consensus or objectively using statistical methods (den Boer et al.,