Unpublished Alan Turing studies could provide an alternative to the evaluation of diagnostic tests
By Paul McKeigue
From Statistical Methods in Medical Research
We are entering a new era of “precision medicine” where prevention and treatment is based on tests that can stratify people by risk and predict which therapy will work best for each individual. However, as research on diagnostic tests has grown, it has become clearer that the current, widely-used statistical methods for evaluating the performance of these tests have serious limitations.
Since the 1980s the C-statistic or AUROC (Area Under the Receiver Operating Characteristic) defined as the probability that a randomly chosen pair of individuals with and without disease will be correctly classified, has been used to quantify the performance of a diagnostic test. However, the C-statistic does not tell us how the diagnostic test would perform when used to stratify people by risk. For instance, to evaluate a test for bowel cancer we might want to estimate what proportion of cancers would be missed and what proportion of people without cancer would undergo colonoscopy if the policy was that everyone with risk above some given threshold should be referred for colonoscopy. Another difficulty is that it does not quantify the incremental contribution of a biomarker to an existing diagnostic test.
In a paper published in Statistical Methods in Medical Research, I propose an alternative approach based on estimating the weight of evidence. The background to this work lies in unpublished studies of Alan Turing, who in 1941 at Bletchley Park investigated the distribution of weights of evidence to decide the best strategy for breaking the ENIGMA code. Turing discovered some key properties of this distribution, which were extended in 1968 by his former assistant Jack Good, by then one of the most influential Bayesian statisticians of the 20th century.
Article details
Quantifying performance of a diagnostic test as the expected information for discrimination: Relation to the C-statistic
Paul McKeigue
DOI: 10.1177/0962280218776989
About
Paul McKeigue, Professor of Genetic Epidemiology and Statistical Genetics Usher Institute of Population Health Sciences and Informatics University of Edinburgh Medical School.