The way to follow up on a significant two-way interaction between two categorical variables is to check the simple effects. Most of the time the simple effects tests give a very clear picture about the interaction. Every so often, however, you have a significant interaction, but no significant simple effects. It is not a logical impossibility. They are testing two different, but related hypotheses.
Assume your two independent variables are A and B. Each has two values: 1 and 2. The interaction is testing if A1 – B1 = A2 – B2 (the null hypothesis). The simple effects are testing whether A1-B1=0 and A2-B2=0 (null) or not.
If you have a crossover interaction, you can have A1-B1 slightly positive and A2-B2 slightly negative. While neither is significantly different from 0, they are significantly different from each other.
And it is highly useful for answering many research questions to know if the differences in the means in one condition equal the differences in the means for the other. It might be true that it’s not testing a hypothesis you’re interested in, but in many studies, all the interesting effects are in the interactions.
Dear Karen,
thanks for your blog and for this particular post. You said that having a significant interaction effect with no significant simple effects “is not a logical impossibility”. However, I was wondering whether it is very plausible in practice and to what extent such a situation can occur in real life. Indeed, you illustrated your post with a situation in which there is a significant crossover interaction with no significant simple effects, A1-B1 being “slightly positive” and A2-B2 being “slightly negative”. However, in such a situation the size of the interaction would not be very large and should require a very small 95% CI to rject H0. Have you some real data to illustrate such a situation?
Thanks for your help
Best,
Nicolas
Hi Nicolas,
I’ve seen this situation very often in all kinds of data sets. I don’t have one at the ready, but the interaction can be quite large.
Hi Karen,
Is it possible to give an example of a hypothesis which requires a simple effects analysis and an example of a hypothesis which requires a contrast analysis?
Thanks in advance,
Best regards
Karina
Hi Karen,
Could you please help me with this question. I have run a 3*3 ANOVA. I have also chosen a simple planned contrast within SPSS to compare each variable to the first variable. I am not too sure if this is the results i should be reporting after looking at the main effects and interactions. Please help.
How do I interpret my results, when the interaction is significant and the means show the effect I want, but testing the simple effects only reveals a p = .80? Can I draw conclusions from my significant interaction or the ‘marginal significant’ p value (I don’t know if it is allowed to say that)?
This website regularly provides amazingly clear/useful advice. Yet again, you helped me climb out of a bind. Keep up the good work!
Hi Karen,
This website is helpful and much needed for people like me.
I am trying to determine whether I am meant to do a follow up power analysis for a one way anova and if so, how to interpret it.
After I had a run a 3 x (emotional support) by 2 (sex) analysis of variance, the data showed significant main effects and interaction, so I split the file on the and did a one- way anova on sex (male and female) to explore the difference and contrast. I found that there was no significant difference for males on the DV but there was for females and I have written a report describing the effect as well as the F-ratio for the non significant data. So my dilemma is, do I now do a power analysis on the non significant male data? If so I would get only 68% chance of detecting a moderate effect size of n2= .06. If I was to conclude this, I would have to state that there would need to be more male participants in the study, however I already have 120 male and 120 female so I cant really conclude that there this is not sufficiently powerful because there is not enough male participants.
Or do I just state in the conclusion that there is no significant evidence to suggest that males are effected by emotional support.
Hope I’m making sense. Not sure how to go about this one.