The Analysis Factor
Statistical Consulting, Resources, and Statistics Workshops for Researchers
One of the many decisions you have to make when model building is which form each predictor variable should take. One specific version of this decision is whether to combine categories of a categorical predictor.
The greater the number of parameter estimates in a model the greater the number of observations that are needed to keep power constant. The parameter estimates in a linear [Read more…] about Simplifying a Categorical Predictor in Regression Models
Last week I had the pleasure of teaching a webinar on Interpreting Regression Coefficients. We walked through the output of a somewhat tricky regression model—it included two dummy-coded categorical variables, a covariate, and a few interactions.
As always seems to happen, our audience asked an amazing number of great questions. (Seriously, I’ve had multiple guest instructors compliment me on our audience and their thoughtful questions.)
We had so many that although I spent about 40 minutes answering [Read more…] about Your Questions Answered from the Interpreting Regression Coefficients Webinar
Even with a few years of experience, interpreting the coefficients of interactions in a regression table can take some time to figure out. Trying to explain these coefficients to a group of non-statistically inclined people is a daunting task.
For example, say you are going to speak to a group of dieticians. They are interested [Read more…] about Using Marginal Means to Explain an Interaction to a Non-Statistical Audience
Sometimes what is most tricky about understanding your regression output is knowing exactly what your software is presenting to you.
Here’s a great example of what looks like two completely different model results from SPSS and Stata that in reality, agree.
I ran a linear model regressing “physical composite score” on education and “mental composite score”.
The outcome variable, physical composite score, is a measurement of one’s physical well-being. The predictor “education” is categorical with four categories. The other predictor, mental composite score, is continuous and measures one’s mental well-being.
I am interested in determining whether the association between physical composite score and mental composite score is different among the four levels of education. To determine this I included an interaction between mental composite score and education.
Here is the result of the regression using SPSS:
Whenever I get email questions whose answers I think would benefit others, I like to answer them here. I leave out the asker’s name for privacy, but this is a great question about dummy coding:
First of all, thanks for all those helpful information you provided! Thanks sincerely for all your efforts!Actually I am here to ask a technical question. See, I have 6 locations (let’s say A, B, C, D, E, and F), and I want to see the location effect on the outcome using OLS models.
I know that if I included 5 dummy location variables (6 locations in total, with A as the reference group) in 1 block of the regression analysis, the result would be based on the comparison with the reference location.
Then what if I put 6 dummies (for example, the 1st dummy would be “1” for A location, and “0” for otherwise) in 1 block? Will it be a bug? If not, how to interpret the result?
Thanks a lot!
Great question!
If you put in a 6th dummy code for Location A, your reference group, the model will actually blow up. (Yes, that’s a technical term).
This is one of those cases of pure multicollinearity, and the model can’t be estimated uniquely.
It’s the same situation you learned back in Algebra where you have two equations, one unknown. The problem isn’t that it can’t be solved–the problem is there are an infinite number of equally good solutions.
If an observation falls in Location A, the reference group, we’ve already gotten that information from the other 5 dummy variables. That observation would have a 0 on all of them. So we already know it’s location is A. We don’t need another dummy variable to tell the model that. It’s redundant information. And so perfectly redundant that the model will choke.
Dummy coding is one of the topics I get the most questions about. It can get especially tricky to interpret when the dummy variables are also used in interactions, so I’ve created some resources that really dig in deeply.
In a previous post, Interpreting Interactions in Regression, I said the following:
In our example, once we add the interaction term, our model looks like:
Height = 35 + 4.2*Bacteria + 9*Sun + 3.2*Bacteria*Sun
Adding the interaction term changed the values of B1 and B2. The effect of Bacteria on Height is now 4.2 + 3.2*Sun. For plants in partial sun, Sun = 0, so the effect of Bacteria is 4.2 + 3.2*0 = 4.2. So for two plants in partial sun, a plant with 1000 more bacteria/ml in the soil would be expected to be 4.2 cm taller than a [Read more…] about Clarifications on Interpreting Interactions in Regression