One of those “rules” about statistics you often hear is that you can’t interpret a main effect in the presence of an interaction.
Stats professors seem particularly good at drilling this into students’ brains.
Unfortunately, it’s not true.
At least not always.
This is one of those situations where I caution researchers to think about their data and what makes sense, rather than memorize a rule.
When interactions do make main effects nonsensical
Let me start with a situation in which the rule holds, so that you can see why it is important to consider.
At baseline (Time=0) there is no difference in the mean of outcome variable Y for the two groups.
I’m just going to assume that that difference is not statistically significant, because the means are right on top of each other. But in a real data output, you’d want to verify this with a simple effects tests.
But there is clearly an interaction here–there was a large change in the mean of Y for the treatment group, but not the control.
This is a perfect example of a case where both main effects will be significant, but they’re not meaningful.
For example, the main effect for Condition will compare the height of the red dotted line to the height of the black solid line. Assuming equal sample sizes , those heights will be compared halfway between time 0 and 1.
Again, assuming adequate power, that difference probably will be significant. But it’s not a meaningful main effect.
A main effect says that there is a difference between the group means, regardless of time.
Sure, if you average it out, that may be technically true. But it’s pretty clear that’s only true on average across the time points because of the big difference at Time 1. It’s not true at each time point. So to say that the means generally differ across conditions, regardless of time, isn’t really accurate.
This is sometimes referred to the interaction driving the main effects and this particular example is why your stat teacher doesn’t want you to blindly say that the significant main effect means anything.
When interactions don’t affect main effects
But of course, meaningful main effects can exist even in the presence of an interaction.
Take a very similar situation, with one important difference.
The interaction itself in the following graph is identical to the one above. The difference in means for the red triangles is once again 13 points, whereas the control group’s mean difference is negligible.
But the placement of the means is farther apart.
What makes the main effect meaningful here, despite the interaction, is that the treatment group’s mean is always higher than the control group’s. Even at Time=0.
That’s a meaningful main effect. It’s saying regardless of what is happening over time, the treatment group really does generally have higher means, regardless of when it’s measured.
Failing to mention this main effect, assuming it’s theoretically important, would be a shame. You’re missing the fact that the treatment group not only changed more, but started higher.
That’s a different situation than the first one, and you need to communicate that.
So when you have both a significant main effect and a significant interaction, stop and examine the results. Don’t assume the main effect is meaningless. It may be important.