• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • Home
  • Our Programs
    • Membership
    • Online Workshops
    • Free Webinars
    • Consulting Services
  • About
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Collaborate with Us
  • Statistical Resources
  • Contact
  • Blog
  • Login

Actually, you can interpret some main effects in the presence of an interaction

by Karen Grace-Martin 8 Comments

One of those “rules” about statistics you often hear is that you can’t interpret a main effect in the presence of an interaction.

Stats professors seem particularly good at drilling this into students’ brains.

Unfortunately, it’s not true.

At least not always.

This is one of those situations where I caution researchers to think about their data and what makes sense, rather than memorize a rule.

When interactions do make main effects nonsensical

Let me start with a situation in which the rule holds, so that you can see why it is important to consider.

Consider this simple interaction between two categorical predictors: Time and Condition.

interaction-no-main-effect

At baseline (Time=0) there is no difference in the mean of outcome variable Y for the two groups.

I’m just going to assume that that difference is not statistically significant, because the means are right on top of each other.  But in a real data output, you’d want to verify this with a simple effects tests.

But there is clearly an interaction here–there was a large change in the mean of Y for the treatment group, but not the control.

This is a perfect example of a case where both main effects will be significant, but they’re not meaningful.

For example, the main effect for Condition will compare the height of the red dotted line to the height of the black solid line.  Assuming equal sample sizes , those heights will be compared halfway between time 0 and 1.

Again, assuming adequate power, that difference probably will be significant.  But it’s not a meaningful main effect.

A main effect says that there is a difference between the group means, regardless of time.

Sure, if you average it out, that may be technically true.  But it’s pretty clear that’s only true on average across the time points because of the big difference at Time 1.  It’s not true at each time point.  So to say that the means generally differ across conditions, regardless of time, isn’t really accurate.

This is sometimes referred to the interaction driving the main effects and this particular example is why your stat teacher doesn’t want you to blindly say that the significant main effect means anything.

When interactions don’t affect main effects

But of course, meaningful main effects can exist even in the presence of an interaction.

Take a very similar situation, with one important difference.

The interaction itself in the following graph is identical to the one above.  The difference in means for the red triangles is once again 13 points, whereas the control group’s mean difference is negligible.

But the placement of the means is farther apart.

interaction-with-main-effect

What makes the main effect meaningful here, despite the interaction, is that the treatment group’s mean is always higher than the control group’s.  Even at Time=0.

That’s a meaningful main effect.  It’s saying regardless of what is happening over time, the treatment group really does generally have higher means, regardless of when it’s measured.

Failing to mention this main effect, assuming it’s theoretically important, would be a shame.  You’re missing the fact that the treatment group not only changed more, but started higher.

That’s a different situation than the first one, and you need to communicate that.

So when you have both a significant main effect and a significant interaction, stop and examine the results.  Don’t assume the main effect is meaningless.  It may be important.

Bookmark and Share

Interpreting Linear Regression Coefficients: A Walk Through Output
Learn the approach for understanding coefficients in that regression as we walk through output of a model that includes numerical and categorical predictors and an interaction.

Tagged With: interaction, interpreting, main effect

Related Posts

  • When Main Effects are Not Significant, But the Interaction Is
  • Spotlight Analysis for Interpreting Interactions
  • What’s in a Name? Moderation and Interaction, Independent and Predictor Variables
  • Confusing Statistical Terms #11: Confounder

Reader Interactions

Comments

  1. M says

    December 20, 2020 at 11:50 pm

    1)You don’t test for baseline as gerro mentioned above. It is pointless because at baseline randomisation ensures that any difference is likeli to due to randomness therefore your test of null hypothesis i.e., H0= there is no group differences is actually don’t have a stand

    2) When you have a study design like this, your ideal model to estimate the main effect is :

    time1 score = intercept + baseline score + treatmentgroup + error

    This gives you the main effect of intervention.

    Reply
  2. José Luiz CARVALHO DE SOUZA DIAS says

    February 28, 2020 at 6:50 pm

    Is there a statistical test that we could perform that would justify not interpreting a significant interaction, allowing us to interpret only the individual main effects?
    Thanks!

    Reply
  3. Alex Allen says

    December 5, 2018 at 1:16 am

    I still don’t understand why the main effect is more important in the second example. In the second example, the main effect of the condition would be the average of the two simple effects of condition (assuming equal sample sizes). So, the main effect of the condition would be roughly (10+25/2), or 17.5. I get why one would emphasize that the two simple effects are in the same direction. If the simple effects are both significant, that would also be important to discuss. But, in the second example, the main effect is still an average of two significantly different effects. What is the point in reporting the effect of the condition when time=.5? Again, I understand the difference in interpreting the main effect when there is a “cross-over” and when there isn’t. For example, the main effect of the condition would be 0 and pretty useless if the lines looked like a perfect X. I just still do not understand why you would interpret the main effect of one variable unless the average value of the variable it is interacting with is of theoretical importance.

    Reply
  4. Riccardo Toscano says

    January 11, 2017 at 2:16 pm

    Previously I had not understood why in the presence of interactions the main effects aren’t interesting, and the first example is quite explanatory.

    Anyway, now I think that main effects aren’t interesting even in the second example. There, Trt*1 > Trt*2 > Ctr*1 = Ctr*2. So the comparisons of the interactions already shows that all Trt > all Crt.

    Reply
    • Fawaz Mzayek says

      May 29, 2019 at 2:10 pm

      In fact, the main effects could be very interesting in the second example. They reveal that the two groups were different at baseline in regards to the outcome. Depending on the study design this could indicate a flawed randomization, or selection bias, among other things.

      Reply
      • gerro says

        September 26, 2020 at 9:11 am

        One never tests differences at baseline in randomized trials. It’s pure nonsense. Randomization ensures that all subjects come from a single population. So even, if the differences are statistically significant, it’s nothing but an artefact (it’s not impossible to sample very different observations into 2 groups from the same population). If the randomization if flawed, no test can show that. It’s procedural problem, not statistical. The opposite holds in non-randomized trials, where the differences at baseline come likely from two different populations. Then, the Lord’s paradox is almost certain.

        Reply
  5. Bryna Chrismas says

    January 12, 2015 at 12:13 pm

    Good afternoon,

    I have always been taught that if a main/interaction effect is not sig then you should not use the post hoc pairwise comparisons even if significant. Hopkins suggests that planned contrasts of this kind are acceptable and we can use them. What is the ‘correct’ answer and which resources would help in understanding this matter?

    Many thanks

    Reply
  6. Barry says

    November 26, 2014 at 10:35 am

    Nice.

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Please note that, due to the large number of comments submitted, any questions on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.

Primary Sidebar

This Month’s Statistically Speaking Live Training

  • Member Training: Introduction to SPSS Software Tutorial

Upcoming Free Webinars

Poisson and Negative Binomial Regression Models for Count Data

Upcoming Workshops

  • Analyzing Count Data: Poisson, Negative Binomial, and Other Essential Models (Jul 2022)
  • Introduction to Generalized Linear Mixed Models (Jul 2022)

Copyright © 2008–2022 The Analysis Factor, LLC. All rights reserved.
877-272-8096   Contact Us

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT