Post-hoc tests, pairwise or other linear contrasts, are typical in an analysis of variance (ANOVA) setting to understand which group means differ. They incorporate p-value adjustments to avoid concluding that group means differ when they actually do not. There are several adjustments that can be considered for conducting multiple post-hoc tests, including single-step and stepwise adjustments.
In this training, we compare some common single-step multiple comparison adjustment procedures. We discuss common uses to these approaches, issues that can arise when considering the results, and how to report and interpret results. We also briefly discuss post-hoc comparisons for the non-parametric equivalent to ANOVA (Kruskal-Wallis) and other types of analyses.
Note: This training is an exclusive benefit to members of the Statistically Speaking Membership Program and part of the Stat’s Amore Trainings Series. Each Stat’s Amore Training is approximately 90 minutes long.
About the Instructor
Julia Sharp is faculty in the Department of Statistics at Colorado State University where she is also the Director of the Graybill Statistics & Data Science Laboratory. She is also the owner of Sharp Analytics LLC, where she is the lead statistical collaborator. She earned her M.S. and Ph.D. in Statistics from Montana State University.
Julia has experience collaborating with researchers in many domains, using her expertise in applied statistics to inform and advance scientific research. Her statistical toolbox is broad and includes study design, analysis of a wide-range of data types (e.g., repeated observations over time, categorical data), and knowledge of popular statistical computing software.