There is a lot of criticism of systematic reviews, both in medical journals and social media. It is claimed that systematic reviews have a bias towards non-recommendation of an intervention, often in the face of one or two very well structured randomized clinical trials. How do we handle this anomaly in our clinical practice, especially when patients are often very knowledgeable due to internet searches?
From the Editor: The thoughtful question regarding the value of systemic reviews and meta-analyses gets a thoughtful answer from our respondents. Please read.
By Doctor Jonathan D. Campbell1,2, David Price2,3, Alison Chisholm2
Systematic reviews and meta-analyses synthesize the findings of multiple evidence sources and present them as one, consolidated result. Yet with simplification of data comes the potential to over-simplify complex topics.[i] The challenge for clinicians and well-read patients is to realize that they must first take a step back from the systematic review’s bottom line to consider also the consistency and generalizability of the results of the trials involved. Interpretation and judgment are required in this science.
Indeed, systematic reviews and meta-analyses can be a powerful tool when used sparingly and appropriately; synthesizing evidence from like studies that pertain to the specific question at hand. Yet the availability of the necessary software and relative ease with which they can be conducted can result in their mis or overuse.
LeLorier et al propose that a well-conducted meta-analysis should involve a priori determination of strict standards to ensure that the criteria used for the inclusion of patients, the administration of the principal treatment, and the ascertainment of outcome events are similar in all the trials selected.[ii] Any heterogeneity among studies included in a systematic review that is not accounted for in the analysis can contribute to a biased recommendation.
Meaningful meta-analysis also requires appropriate effect size/endpoint selection and the inclusion of trials that actually address the specific question the meta-analysis is seeking to answer. The importance of appropriate endpoint selection is well illustrated by a Cochrane systematic review and a meta-analysis of the effectiveness of evidence from trials evaluating the role of fractional exhaled nitric oxide (FeNO) in guided asthma management strategies.[iii],[iv] Both the review and meta-analysis concluded that FeNO measurement should not be recommended on the basis of exacerbation reduction. However, both used proportion of participants who experienced at least one asthma exacerbation during follow-up as the primary endpoint rather than the National Institute for Health (NIH)’s recommended endpoint of asthma exacerbation rate. A repeat meta-analysis using the NIH endpoint found significantly lower exacerbation rates in patients receiving FeNO-guided asthma management.[v]
In conclusion, meta-analyses (like other statistical tools) have their merit when used appropriately. The key to interpreting the clinical implications of systematic reviews is understanding whether the evidence they synthesize is pertinent to the question at hand.
[i] Siontis KC, Hernandez-Boussard T, Ioannidis JP. Overlapping meta-analyses on the same topic: survey of published studies. BMJ. 2013;347:f4501
[ii] LeLorier J, Grégoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med. 1997;337:536-42.
[iii] Petsky HL, Cates CJ, Li A, Lynaston JA, Turner C, Chang AB: Tailored interventions based on exhaled nitric oxide versus clinical symptoms for asthma in children and adults. Cochrane Database Syst Rev 2009, 2, CD006340.
[iv] Petsky HL, Cates CJ, Lasserson TJ, Li AM, Turner C, Kynaston JA, Chang AB: A systematic review and meta-analysis: tailoring asthma treatment on eosinophilic markers (exhaled nitric oxide or sputum eosinophils). Thorax 2012, 67:199–208.
[v] Donohue JF, Jain N: Exhaled nitric oxide to predict corticosteroid responsiveness and reduced asthma exacerbation rates. Respir Med 2013, 107:943–952.
Doctor Jonathan D. Campbell1,2, David Price2,3, Alison Chisholm2
- University of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences, Aurora, Colorado, USA
- Respiratory Effectiveness Group, Cambridge, UK
- The University of Aberdeen, Aberdeen, UK