Interpreting Interactions: When the F test and the Simple Effects disagree.

by Karen

Share

The way to follow up on a significant two-way interaction between two categorical variables is to check the simple effects.  Most of the time the simple effects tests give a very clear picture about the interaction.  Every so often, however, you have a significant interaction, but no significant simple effects.  It is not a logical impossibility. They are testing two different, but related hypotheses.

Assume your two independent variables are A and B.  Each has two values: 1 and 2.  The interaction is testing if A1 – B1 = A2 – B2 (the null hypothesis). The simple effects are testing whether A1-B1=0 and A2-B2=0 (null) or not.

If you have a crossover interaction, you can have A1-B1 slightly positive and A2-B2 slightly negative. While neither is significantly different from 0, they are significantly different from each other.

And it is highly useful for answering many research questions to know if the differences in the means in one condition equal the differences in the means for the other. It might be true that it’s not testing a hypothesis you’re interested in, but in many studies, all the interesting effects are in the interactions.

On the hunt for affordable statistical training with the best stats mentors around? Want to ask an expert all your burning stats questions? Check out Statistically Speaking, our exclusive membership program featuring monthly webinars and open Q&A sessions.


{ 3 comments… read them below or add one }

Sophie December 28, 2016 at 6:27 am

How do I interpret my results, when the interaction is significant and the means show the effect I want, but testing the simple effects only reveals a p = .80? Can I draw conclusions from my significant interaction or the ‘marginal significant’ p value (I don’t know if it is allowed to say that)?

Reply

Emil June 28, 2015 at 5:43 pm

This website regularly provides amazingly clear/useful advice. Yet again, you helped me climb out of a bind. Keep up the good work!

Reply

Nola May 14, 2014 at 9:59 pm

Hi Karen,
This website is helpful and much needed for people like me.
I am trying to determine whether I am meant to do a follow up power analysis for a one way anova and if so, how to interpret it.
After I had a run a 3 x (emotional support) by 2 (sex) analysis of variance, the data showed significant main effects and interaction, so I split the file on the and did a one- way anova on sex (male and female) to explore the difference and contrast. I found that there was no significant difference for males on the DV but there was for females and I have written a report describing the effect as well as the F-ratio for the non significant data. So my dilemma is, do I now do a power analysis on the non significant male data? If so I would get only 68% chance of detecting a moderate effect size of n2= .06. If I was to conclude this, I would have to state that there would need to be more male participants in the study, however I already have 120 male and 120 female so I cant really conclude that there this is not sufficiently powerful because there is not enough male participants.
Or do I just state in the conclusion that there is no significant evidence to suggest that males are effected by emotional support.
Hope I’m making sense. Not sure how to go about this one.

Reply

Leave a Comment

Please note that Karen receives hundreds of comments at The Analysis Factor website each week. Since Karen is also busy teaching workshops, consulting with clients, and running a membership program, she seldom has time to respond to these comments anymore. If you have a question to which you need a timely response, please check out our low-cost monthly membership program, or sign-up for a quick question consultation.

Previous post:

Next post: