A Heirarchical Model of Efficacy for Diagnostic Tests

This model was originally proposed in 1991 by Fryback and Thornbury. The key feature of this ‘conceptual continuum’ is that, while a test may perform well at one level, this does not imply that it’s performance will be satisfactory at higher levels.

It is worth being aware of this model when critically appraising literature as it may
a) Indicate the limitations of the study in question.
b) Suggest avenues for further research, if the study in question provides good evidence about test performance at a low level in the heirarchy.

It is important to be aware of this model if using Radiology research data to formulate clinical policy.
The levels, in increasing order of efficacy were defined as

1 Technical (Can I see it ?)
2 Diagnostic Accuracy (How well can I see it ?)
3 Diagnostic Thinking (Is there an impact on diagnosis ?)
4 Therapeutic (Does the test change therapy ?)
5 Outcome (Does the therapy change patient outcome ?)
6 Societal (Cost-benefit, cost-effectiveness etc.)

What is the highest level you can attribute to your study?

Level, Tpye of Efficacy and Examples 

Level 1 Technical efficacy 
Resolution of line pairs 
Modulation transfer function change 
Gray-scale range 
Amounts of mottle 
Sharpness 
Computerized imaging parameters 

Level 2 Diagnostic accuracy efficacy 
Yield of abnormal or normal diagnoses in a case series 
Diagnostic accuracy (percentage of correct diagnoses in case series) 
Sensitivity, specificity, and positive/negative predictive value in a defined 
clinical problem setting 
Measures of area under the receiver operating characteristic (ROC) curve 

Level 3 Diagnostic thinking efficacy 
Number (percentage) of cases in a series in which image was judged 
"helpful" to making the diagnosis 
Entropy change in differential diagnosis probability distribution 
Difference in clinicians’ subjectively estimated diagnosis probabilities 
before and after test information 

Level 4 Therapeutic efficacy 
Number (percentage) of times image was judged "helpful" in planning 
patient care in a case series 
Percentage of times medical or surgical procedure avoided due to image 
information 
Number or percentage of times planned therapy pretest changed after the 
image information was obtained (retrospectively inferred from clinical 
records) 
Number or percentage of times clinicians’ prospectively stated therapeutic 
choices changed after test information 
? Patient utility assessment 

Level 5 Patient outcome efficacy 
Percentage of patients improved with test vs without test 
Morbidity (or procedures) avoided after having image information 
Change in quality-adjusted life expectancy 
Expected value of test information in quality-adjusted life years (QALYs) 
Cost per QALY saved with image information 
Patient utility assessment (eg. Markov modeling, time trade-off) 

Level 6 Societal efficacy 
Benefit-cost analysis from societal viewpoint 
Cost-effectiveness analysis from societal viewpoint 


References 
1. Intermediate Outcomes: Diagnostic and Therapeutic Impact. Thornbury JR. 1999 Academic Radiology; 6 (Suppl. 1): S58-65

2. The Efficacy of Diagnostic Imaging. Fryback DG, Thornbury JR. 1991 Med Decis Making;11:88-94

Administrator Login +