Using Pairwise Comparisons to Help you Interpret Interactions in Linear Regression

In a previous post we discussed using marginal means to explain an interaction to a non-statistical audience. The output from a linear regression model can be a bit confusing. This is the model that was shown.

In this model, BMI is the outcome variable and there are three predictors:

Age, which is continuous

Gender, which has two categories.

Frame, which has three categories.

The model also contains an interaction between the two categorical predictors. The problem with interactions between two categorical predictors is their regression coefficients are not intuitive.

The categorical interaction is measuring the difference in mean differences among specific categories. Let’s start with the coefficient for male/medium frame. Its value is -1.29. What does that actually mean?

Using the information from the table of marginal means will help us understand the difference in mean differences.

We begin by determining which category is the base index (a.k.a. reference category) for each of the two categorical variables in the interaction. Looking at the table we see that the base index for gender is female and the base index for frame is small.

(Luckily Stata actually labelled that, but if you’re using another software, the base is generally either the category that has a 0.0 value for the coefficient or is the one that is not included in the table at all).

Knowing these is important because all the coefficients measure mean differences  between the labelled category and the base category.

For example, the coefficient of 5.49 tells us that the mean BMI of Medium-framed females (at the same age) is 5.49 units larger than the mean BMI of Small-Framed Females.

Table of Marginal Means

We can verify that if we just do a little subtraction looking at our marginal means table. These marginal means are already controlling for age, so we don’t have to worry about any age effects.

Indeed, the mean for Medium-framed Females (31.64) is 5.49 units larger than Small-framed Females (26.15).

You can do this subtraction yourself or you can get your software to print out all the mean differences in another table.  There will probably be more mean differences than you want, so I’ve highlighted the part we care about in blue.

Table of Differences in Marginal Means

Okay, but what about that interaction?

Remember the interaction measures the difference in the mean differences. We’ve already compared one mean difference between medium and small-framed females.

The interaction term for medium frame measures whether the mean difference between medium-framed and small-framed males is exactly the same as it was for females. Yes, it’s labelled only male#medium, but that’s why it’s important to keep your base categories in mind.

Think about this for a moment: if the difference is 0, that tells you something—the difference in mean BMI between medium and small frame is exactly the same for males and females.

The coefficient of the male#medium interaction is calculated in the following manner. First find the difference between medium frame males and small frame males. Next find the difference between medium frame females and small frame females.

Medium frame males less small frame males equal 26.72 – 22.52 =4.20.

Medium frame females less small frame females equal 31.64 – 26.15 =5.49.

Next we take the difference, 4.20 – 5.49 = -1.29. This matches the coefficient of the interaction for medium frame males.

How do we know if the difference of mean differences of 1.29 is statistically significant?

Well, that’s the beauty of the original regression coefficients table. It calculated the p-value for the null hypothesis that this difference in mean differences is 0. Where did it get that?

The statistical software package calculated a standard error of 1.46 for this parameter.  We divide the coefficient by the standard error to calculate the t score. Dividing -1.29 by 1.46 is -0.88. Using a t-score table we find that a t-score of 0.88 gives us a p-value of 0.378. Thus, the interaction of gender/frame for medium frame males is not statistically significant.

So if we need a measurement and p-value for the difference in mean differences, we get that from the regression table. It tells us whether the mean BMI difference between medium and small frame males is the same as the mean BMI difference between medium and small frame females.

And in fact, our p-value of .378 indicated we have no evidence that these two mean differences are different from each other.

The pairwise comparison is a much simpler calculation. It is simply comparing the marginal means of two groups. We do not have to take the difference of the differences as we did above.

The difference between medium frame women and small frame females is 5.49. The statistical software calculated a standard error of 0.87. Dividing 5.49 by 0.87 is 6.31. The p-value of a t-score equal to 6.31 is 0.000.

So if we need a measurement and p-value for a mean differences, we get that from the table of pairwise comparisons. It tells us whether the mean BMI difference between medium and small frame males is the same as 0.

And our p-value below .0001 indicated we do have evidence that this one mean difference of 5.49 is different from 0.

That’s testing a different hypothesis than the interaction is.

Jeff Meyer is a statistical consultant with The Analysis Factor, a stats mentor for Statistically Speaking membership, and a workshop instructor. Read more about Jeff here.

Interpreting Linear Regression Coefficients: A Walk Through Output

Learn the approach for understanding coefficients in that regression as we walk through output of a model that includes numerical and categorical predictors and an interaction.

Take Me to The Video!