Answers to the Interpreting Regression Coefficients Quiz

by

Yesterday I gave a little quiz about interpreting regression coefficients.  Today I’m giving you the answers.

If you want to try it yourself before you see the answers, go here.  (It’s truly little, but if you’re like me, you just cannot resist testing yourself).

True or False?

1. When you add an interaction to a regression model, you can still evaluate the main effects of the terms that make up the interaction, just like in ANOVA.

In an ANOVA table (even the one in the regression output), categorical variables are Effect Coded.  Because of that, the main effects remain main effects, and are evaluated independent of interactions.

But in the Regression Coefficients table, unless you are explicitly effect coding, they will be Dummy Coded.  The coefficient for what looks like a main effect IS NOT a main effect.  It’s a marginal effect–the effect of that predictor ONLY when the other predictor in the interaction =0!  I kid you not.

You can get a little more info in this post or a lot more in this video or a whole lot more in the workshop.

2. The intercept is usually meaningless in a regression model.

This statement is only true if all predictors are continuous and the data don’t contain 0.  If continuous predictors are centered and/or if there are dummy variables in the model, the intercept is meaningful and important.

3. In Analysis of Covariance, the covariate is a nuisance variable, and the real point of the analysis is to evaluate the means after controlling for the covariate.

It can be true, but it doesn’t have to be.  Covariates are often important predictors that just happen to be observed and continuous.  The only way to evaluate them is to examine their coefficients.

4. Standardized regression coefficients are meaningful for dummy-coded predictors.

This one is never ever true.  Just because your software lets you get away with it doesn’t mean it’s meaningful.

5. The only way to evaluate an interaction between two independent variables is to categorize one or both of them.

Sure, it’s tricky to interpret interactions between two continuous variables, but by no means is it impossible or theoretically incorrect.  (And centering really helps).

—————————————————————————————————–

How did you do?  (BTW, it took me years of figuring all this stuff out in a way that was really intuitive, even after many stats classes). Interpreting Linear Regression Coefficients: A Walk Through Output
Learn the approach for understanding coefficients in that regression as we walk through output of a model that includes numerical and categorical predictors and an interaction.

Related Posts

{ 5 comments… read them below or add one } Stephen

A question, or maybe point of clarification, on number four. By standardized regression coefficients, does this mean both predictors and responses? Couldn’t a dummy or dichotomiualy coded response variable for two groups (say, female =1, male = 0) be meaningfully regressed on a standardized response variable? Read then as being female having [beta coefficmet] standard deviation effect? The dummy predictor variable in this case is not standardized. Karen Grace-Martin

Hi Stephen,

Essentially yes. There is a shortcut to take a regular regression coefficient and divide it by the standard deviations of both X and Y to produce what is called a “standardized regression coefficient.” It is interpreted as the number of standard deviation difference in Y, on average, associated with a one standard deviation difference in X.

You’re correct that if you’re doing the standardizing yourself of numerical variables, it’s not a problem. But if you’re using the software-created standardized regression coefficients, it’s not leaving out the dummy coded predictors. It’s standardizing them as well. So that standardized coefficient is for each one standard deviation difference in Female, which is nonsensical. Bob Nau

Hi Karen,
Cheers,
–Bob Karen Grace-Martin

Hi Bob,

Yes, I think you’re right that it comes down to what you consider usual or meaningless. I find very few models in research where there are no categorical predictors or interactions, where the means are important to understand.

And as I think I’ve mentioned before, I can’t recommend people use excel for statistics, even with add-ins. Excel is not set up for statistical analysis or data management. Reproducibility is too important. It might be fine for playing around with data (I use it for that), but not for anything that will be published. Carlos Camacho