All Models Are Wrong and Yours Are Useless: Making Clinical Prediction Models Impactful for Patients

All models are wrong and yours are useless: making clinical prediction models impactful for patients Florian MarkowetzCheck for updates Most published clinical prediction models are never used in clinical practice and there is a huge gap between academic research and clinical implementation.Here, I propose ways for academic researchers to be proactive partners in improving clinical practice and to design models in ways that ultimately benefit patients."All models are wrong, but some are useful" is an aphorism attributed to the statistician George Box.There is humility in claiming your model is wrong, but there is also bravado in implying your model might be useful.And, honestly, I don't think it is.I think your model is useless.How would I know?I don't even know who you are.Well, it is a bet.A bet I am willing to take because the odds are ridiculously in my favour.I will explain what I mean in the context of clinical prediction models.My points apply to a wide range of preclinical models, both computational and biological, but my own core expertise is with clinical prediction tools.These are computational models from statistics, machine learning or AI that try to predict clinically relevant variables and ultimately aim to help doctors to treat patients better.The papers describing them make claims like "this model can be used in the clinic"; generally softened with words like "might", "could", "potential", "promise", or other techniques to reduce accountability.The Box quote offers a yardstick to measure the success of these models; not by how correctly they describe reality but by how useful they are in helping patients.And in general, almost none of these tools ever help anyone.There is a wealth of systematic reviews in different fields to show how many models have been proposed and how few have even been validated, let alone been adopted in the clinic.For example, 408(!) models for chronic obstructive pulmonary disease were systematically reviewed 1 and as a summary the authors bleakly note "several methodological pitfalls in their development and a low rate of external validation".And whatever biomedical area you work in, your experiences will mirror this resultmany novel prediction models, little help for patients.I believe that a model designed to be used for patients is useless unless it is actually used for patients.