In statistical practice, there are many situations where best practices are clear. There are many, though, where they aren’t. The granddaddy of these practices is adjusting p-values when you make multiple comparisons. There are good reasons to do it and good reasons not to. It depends on the situation.
At the heart of the issue is a concept called Family-wise Error Rate (FWER). FWER is the probability that
The classic way to compare means in analysis of variance is examining pairwise differences in means after an F test.. It’s great for many problems, but sometimes your research question isn’t about pairwise differences.
Pairwise differences are not ideal if your research question is if, like the Sesame Street song, one of these groups is not like the others. Perhaps, you are trying to assure that groups conform to a common standard. Analysis of Means (ANOM) helps in all these settings. (more…)
One of the hardest steps in any project is learning to ask the right research question!
(more…)
Recommendations on how to analyze pre-post data can vary. Typical recommendations include regression analysis or matched pairs analysis for within subject studies and analysis of covariance (ANCOVA) or linear mixed effects model analysis for within and between subject studies.
(more…)
There is a lot of skill needed to perform good data analyses. It is not just about statistical knowledge (though more statistical knowledge is always helpful). Organizing your data analysis, and knowing how to do that, is a key skill. 
(more…)
It’s easy to develop bad habits in data analysis. When you’re new to it, you just don’t have enough experience to realize that what feels like efficiency will actually come back to make things take longer, introduce problems, and lead to more frustration. (more…)