Meta-analysis allows us to synthesize the results of separate studies. The goal is to assess the mean effect size and also heterogeneity – how much the effect size varies across studies.
The heterogeneity is a critically important element in any meta-analysis. If the effect size is consistent across studies, we can say that the treatment will have essentially the same effect in all comparable populations. On the other hand, if the effect size varies substantially across studies, it becomes important to quantify the extent of the variation.
The vast majority of published meta-analyses do not address heterogeneity in any meaningful way. Researchers typically report statistics such as the Q-value, p-value and I-squared. None of these tells us how much the effect size varies. Often researchers use the I-squared statistic to quantify heterogeneity as being low, moderate or high. While this practice is ubiquitous, it is nevertheless fundamentally incorrect.
In this training, we will present the prediction interval, which actually tells us how much the effect size varies. We can then use that information to properly understand the clinical or substantive utility of the intervention being studied.
In this training you will learn:
- Why quantifying heterogeneity in a meta-analysis is important
- How the prediction interval addresses heterogeneity and how the I-squared does not
- How to report the results of a meta-analysis
- Common mistakes related to heterogeneity, and how to avoid them
You should have a basic understanding of meta-analysis. If you need a review, please watch Meta-analysis before watching this training.
Note: This training is an exclusive benefit to members of the Statistically Speaking Membership Program and part of the Stat’s Amore Trainings Series. Each Stat’s Amore Training is approximately 90 minutes long.About the Instructor
Michael Borenstein is the co-author (with Larry Hedges, Julian Higgins, and Hannah Rothstein) of the text Introduction to Meta-Analysis and the author of the text Common Mistakes in Meta-Analysis and How to Avoid Them. Dr. Borenstein previously served as Director of Biostatistics at Hillside Hospital, a Division of Long Island Jewish Medical Center in New York, and as Associate Professor at the Albert Einstein College of Medicine in New York. Beginning in 2000 Dr. Borenstein has worked exclusively in the field of meta-analysis. He has taught meta-analysis at the CDC, FDA, and NIH. He offers workshops in-person and online at www.Meta-Analysis-Workshops.com