Pros & Cons of EBR

Pros - Points in favor:

These have been dealt with in detail in the literature. For example, Radiology’s special review discussed EBR in detail in 2001 [1]. The originators of Evidence Based Medicine took this topic on in the BMJ in 1996. The following link will take you directly to their manuscript – we’ll let them speak for themselves! The article is entitled ‘EBM – what it is and what it isn’t.’

Cons - Limitations of EBP:

These have been discussed in more detail in the literature [1, 2]. Currently, key considerations are:

• The EBM line of reasoning is relatively new to radiologists and is not part of the radiology curriculum. Radiologists may, therefore, find the concepts too obscure, complex and non-intuitive.
• Searching can be difficult and the best evidence may not be readily available despite internet-based search tools; papers may be inadequately or inappropriately indexed; many abstracts are never published as papers and there is a tendency for authors and editors not to submit or publish ‘negative studies’. In 2001, it was reported that the weakest link in production of good-quality critical appraisals was identification of relevant articles. In one study, 22% of appraisals missed the most relevant articles to answer the clinical question [3]. This should be a focus for training in evidence-based medicine and critical appraisal skills.
• It can be difficult to find good data on diagnostic questions. You may have to go into the raw data of well-designed but poorly analysed studies and calculate sensitivity, specificity, confidence intervals, predictive values and likelihood ratios. Software (such as the software on this site) can help, but you must allow for time and effort. In time, we hope that academic rewards will accrue within Radiology (as within Medicine) for the production of high-quality secondary literature. The academic centers will then address common problems for us. Right now, there is very little secondary evidence about Radiology available.
• For assessment of therapeutic interventions, randomised controlled trials (RCTs) are the penultimate step in the evidence base. This is fine if you find one or more. This hierarchical approach to evidence is sometimes mis-interpreted as meaning that other types of studies are not useful. This is not the case. RCTs are not perfect tools and their "generalisability" can be limited. Strong RCTs are scarce. The following points about RCTs [4] should be considered:

  - Treatment effects obtained from randomised and non-randomised studies may differ, but one method does not give a consistently greater effect than the other

  - Treatment effects measured in each type of study best approximate when the exclusion criteria are the same and where potential prognostic factors are well understood and controlled for in the non-randomised studies

  - Subjects excluded from randomised controlled trials tend to have a worse prognosis than those included, and this limits generalisability

  - Subjects participating in randomised controlled trials evaluating treatment of existing conditions tend to be less affluent, educated, and healthy than those who do not; the opposite is true for trials of preventive interventions

• Other authors have also noted that the results of well-designed observational studies (with either a cohort or a case-control design) do not systematically overestimate the magnitude of the effects of treatment as compared with those in randomised, controlled trials on the same topic [5].
• In reviewing clinical evidence we must be reluctant to adopt too detailed criteria for good and bad science and to freeze criteria for validity. Study methods themselves need to evolve. The randomised controlled trial was developed over half a century and refined in the slipstream of important clinical questions, rather than the reverse. At the same time, much knowledge gained before randomised controlled trials came into being survived into the era of the randomised controlled trial [6].
• Case reports are the lowest form of evidence in the EBM ‘hierarchy.’ They may, in certain circumstances, provide valuable information [7].

Evidence, judgement and the role of the expert in the age of EBP

A superficial understanding of the EBM principles might lead some to consider that if evidence is categorised as being of low validity it should be discarded. This has never been the intent of the EBM exponents provided better evidence has not been discarded or overlooked. It is important to stress this, because the final decisions are easy when the evidence is conclusive. When application of the EBM principles has retrieved only weak evidence expert judgement will, as always, be required to weight and rank it appropriately.

In 1999, a thought-provoking article on this topic.was published in the Canadian literature [8]. The theme is that, Judgment in medicine is the ability to perceive differences, the ability to discriminate between a banal observation and a significant one, the ability to perceive degrees, the ability to estimate the degree of coherence of a set of theses. Evidence confers subjective probability to a hypothesis based on assessments of differences (for example, risks versus benefits).

The most striking example of the difference between judgement and evidence comes from 1983, when experts suspected that AIDS was probably due to an infectious agent and there were isolated reports of patients who had received blood products and who could have secondarily developed the disease. U.S. blood banks took measures to screen blood from high-risk groups. The Canadian Red Cross considered that the probability of developing AIDS from blood products was low and took a less aggressive approach. Historically, the 1997 verdict of the Commission of Inquiry on the Blood System in Canada that the Red Cross should not have required "conclusive evidence" before taking action to reduce the risk of AIDs is a landmark decision in this respect. It serves as a reminder that judgement is still required in an evidence-based world and that sometimes, real leadership requires difficult decisions be taken based on weak evidence. In his article, Auclair concludes "there may be no shortcuts to the acquiring of sound clinical judgment; perhaps 'judgment is the understanding that comes with age'."

On a lighter note, if you don’t want to engage in ‘Evidence Based Practice’, here are some alternatives!

‘The magnificent seven’

... and one other option

Would you like to use an easier rating scale for evidence?

or read some comments on ‘the magnificent seven.’

References

1. Evidence-based radiology: a new approach to the practice of radiology. Radiology 2001; 220 (3):566-575. [ link ]

2. Malone DE ,MacEneaney PM, Applying 'technology assessment' and 'evidence based medicine' theory to interventional radiology. Part 1: Suggestions for the phased evaluation of new procedures. Clin Radiol 2000; 55 (12):929-937. [ link ]

3. Coomarasamy A, Latthe P, Papaioannou S, Publicover M, Gee H, Khan KS, Critical appraisal in clinical practice: sometimes irrelevant, occasionally invalid. J R Soc Med 2001; 94 :573-577. [ link ]

4. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C, Interpreting the evidence: choosing between randomised and non-randomised studies. BMJ 1999; 319 :312-315. [ link ]

5. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000; 342 (25):1887-1892. [ link ]

6. Knottnerus JA, Dinant G, Medicine Based Evidence, a Prerequisite for Evidence-Based Medicine. BMJ 1997 Nov 1;315(7116):1109-10 [ link ]

7. Vandenbroucke JP, Case reports in an evidence-based world [editorial]. J R Soc Med 1999; 92 (4):159-163. [ link ]

8. Auclair F, On the Nature of Evidence. J Royal Coll Phys and Surg Canada 1999; 32 :453-455. ]

9. Malone DE, Staunton M. Evidence-Based Practice in Radiology: Step 5 (Evaluate) - Caveats and common questions. Radiology 2007; 243(2):319-328.[ link ]