The way to follow up on a significant two-way interaction between two categorical variables is to check the simple effects. Most of the time the simple effects tests give a very clear picture about the interaction. Every so often, however, you have a significant interaction, but no significant simple effects. It is not a logical impossibility. They are testing two different, but related hypotheses.
Assume your two independent variables are A and B. Each has two values: 1 and 2. The interaction is testing if A1 – B1 = A2 – B2 (the null hypothesis). The simple effects are testing whether A1-B1=0 and A2-B2=0 (null) or not.
If you have a crossover interaction, you can have A1-B1 slightly positive and A2-B2 slightly negative. While neither is significantly different from 0, they are significantly different from each other.
And it is highly useful for answering many research questions to know if the differences in the means in one condition equal the differences in the means for the other. It might be true that it’s not testing a hypothesis you’re interested in, but in many studies, all the interesting effects are in the interactions.
{ 10 comments… read them below or add one }
Hi there,
Thanks for this information, but Im having some trouble getting my head around the interpretations that are possible. I have found no main effects but a significant interaction
A = non significant
B = non significant
A x B = significant (p. = .03, effect size = .30)
The interaction is disordinal (A1 is higher than A2, but B1 is lower than B2).
No follow up tests are significant when using t tests, and I understand why based upon your discussion above. So I proceeded to do the following
A1-A2 vs B1-B2: significant
But A1-B1 vs A2-B2 was also significant: in fact it resulted in exactly the same t value and significance level. So both of these tests suggested I was somehow testing the same vales, although I did not.
Furthermore, assuming that I focus on A1-B1 vs A2-B2 as the comparison Im interested in, how do I actually interpret and write this up in a valid manner for a scientific journal? Do you have any examples of this in journals I could cite?
Thanks for the help, it is very much appreciated
David,
I can’t think of an example in the literature (example, anyone?) — I don’t always see that end of things.
The best way to write it up is to display the means (either with a graph or table–I like graphs, personally) and report the F statistic with p-value. Describe it as a crossover interaction.
Because it’s a 2×2, you don’t technically need any simple effects tests. The only way for the F stat for the interaction to be significant is for the differences in means to be significantly different.
So in your write up, you can focus on either A1-A2 not equal to B1-B2, or on A1-B1 not equal to A2-B2. The interaction says both are true, but usually in research, one of these comparisons is more meaningful.
I think your two t-tests are both testing the exact same thing as the F.
If you had, say, a 2×3 interaction, this wouldn’t be exactly the same.
Karen
Thanks Karen
I really appreciate your help, it is very easy to understand.
Hello Karen,
I have been reading about this issue throughout today and feel I understand the situation on the 2×2 model but just cannot seem to apply this to my own model, a 2x2x3 mixed design, with A as a within-subjects factor; and B and C as both between-subjects factors.
I have this significant 3-way interaction but no significant simple effects when followed up, and although it is good to see this issue has arisen for others I can only seem to find examples/explanations on 2×2′s.
On the 2×2 as I understand I could look to see if it is a crossover interaction, and see if A1-B1 = 0 and if B1-B2 = 0; if one is positve and the other negative it is explained as a crossover interaction, and that it is known where the interaction is because there is only 2 differences in means, which were found to be significant.
On my 2x2x3 model: I have the means for each group, a total of 12. Would I need to do something like this for the mean differences:
A1-B1-C1; A1-B1-C2; A1-B1-C3
A2-B2-C1; A2-B2-C2; A2-B2-C3. Which gives me 6 mean differences scores, and then compare the paired sets, e.g:
A1-B1-C1 compared with A2-B2-C1 — to see if in each of these 3 pairs have the crossovers or not?
If this is the case and I do have crossovers, how would I then go about explaining it? In the answer above on here and on another forum, its mentioned anything more than the 2×2 (e.g. a 2×3) the case is not as simple in explaining – I think my case fits that, so would I explain that I have found the 3-way interaction, with further investigation the interaction is due to a crossover effect of the mean differences, however due to the model it is unsure excatly where it lies?!
If the crossover is not the case, can I then explain the inteaction due to sampling error or complex design, i.e the interaction getting lost in the noise of the data?
Thank you in advanced for any time you can spare on this issue,
Kind Regards,
Sam
Hi Sam,
It’s hard to explain this in the abstract, without looking at the same means together, but I’ll do my best.
First of all, I would suggest plotting your 12 means. I find interactions much easier to interpret if I can see the patterns. You have to be careful, because what looks different on a graph isn’t always significant, but even so, seeing the relationships among the means is very helpful.
The way to graph this is to have two separate graphs. Each one will be a 2×3 graph of A*C at one value of B (B1 and B2).
The three way interaction (if indeed that’s the one that’s significant) is saying the 2-way interactions in these two graphs are different. Maybe they’re *both* crossovers, but in different directions. If so, you’d have lots of significant interactions, but no main effects.
Your simple effects tests would then be comparing A1 to A2 at each value of C (C1, C2, C3) first in the B1 graph and then in B2.
I hope that helps!
Karen
Hi Karen,
I am dealing with a continuous terms interaction. My interaction (between two continuous terms) is significant, but now my professor is interested in knowing whether at a higher value of one predictor, the level (lower vs higher levels of 2nd predictor) means of response are significantly different from each other. I have no idea. Everywhere I see, I only see stuff for two categorical predictors. I don’t see any examples for two continuous interaction term.
thank you.
Hi FS,
When you have two continuous predictors that interact, the interaction is saying that the slope of one variable differs at every value of the other.
The means your professor wants are really predicted values. You could use the EMMeans command in spss glm (this will only work in syntax, not the menus) or the lsmeans with a slice command in sas proc glm to get that comparison. So I would start with reading the manual to see how to do that in your software (you don’t mention which one you use).
It’s something we cover in the Interpreting (Even Tricky) Regression Coefficients Workshop. It would take me a while to explain, but you could either look into that workshop:
http://www.theanalysisinstitute.com/workshops/IRC/index.html
or sign up for a Quick Question consultation:
http://www.theanalysisfactor.com/statistical-consulting-services/quick-question-consultations/
if you want me to walk you through it.
Karen
Hi Karen,
I have a question about a significiant interacation between 2 continuous variable as well. For my data, I found a significant interaction for 2 continuous variables and this interaction was testing a possible moderating effect. In this case, how do I conduct a follow-up test? Also, can I conclude that there is a moderating effect if the follow-up test is not significant?
Thanks!
Hi Tony,
You may not need a follow-up test. The regression coefficient for the interaction tell you the size of the difference in the slope of one variable for each one unit change in the other. That may be enough to interpret the interaction.
Karen
I loved your article.Really thank you! Much obliged.