The Analysis Factor
Statistical Consulting, Resources, and Statistics Workshops for Researchers
Multicollinearity is one of those terms in statistics that is often defined in one of two ways:
1. Very mathematical terms that make no sense — I mean, what is a linear combination anyway?
2. Completely oversimplified in order to avoid the mathematical terms — it’s a high correlation, right?
So what is it really? In English?
[Read more…] about A Visual Description of Multicollinearity
Oh so many years ago I had my first insight into just how ridiculously confusing all the statistical terminology can be for novices.
I was TAing a two-semester applied statistics class for graduate students in biology. It started with basic hypothesis testing and went on through to multiple regression.
It was a cross-listed class, meaning there were a handful of courageous (or masochistic) undergrads in the class, and they were having trouble keeping [Read more…] about Confusing Statistical Terms #2: Alpha and Beta
Just yesterday I got a call from a researcher who was reviewing a paper. She didn’t think the authors had run their model correctly, but wanted to make sure. The authors had run the same logistic regression model separately for each sex because they expected that the effects of the predictors were different for men and women.
On the surface, there is nothing wrong with this approach. It’s completely legitimate to consider men and women as two separate populations and to model each one separately.
As often happens, the problem was not in the statistics, but what they were trying to conclude from them. The authors went on to compare the two models, and specifically compare the coefficients for the same predictors across the two models.
Uh-oh. Can’t do that.
If you’re just describing the values of the coefficients, fine. But if you want to compare the coefficients AND draw conclusions about their differences, you need a p-value for the difference.
Luckily, this is easy to get. Simply include an interaction term between Sex (male/female) and any predictor whose coefficient you want to compare. If you want to compare all of them because you believe that all predictors have different effects for men and women, then include an interaction term between sex and each predictor. If you have 6 predictors, that means 6 interaction terms.
In such a model, if Sex is a dummy variable (and it should be), two things happen:
1.the coefficient for each predictor becomes the coefficient for that variable ONLY for the reference group.
2. the interaction term between sex and each predictor represents the DIFFERENCE in the coefficients between the reference group and the comparison group. If you want to know the coefficient for the comparison group, you have to add the coefficients for the predictor alone and that predictor’s interaction with Sex.
The beauty of this approach is that the p-value for each interaction term gives you a significance test for the difference in those coefficients.
If you’ve ever been puzzled by odds ratios in a logistic regression that seem backward, stop banging your head on the desk.
Odds are (pun intended) you ran your analysis in SAS Proc Logistic.
Proc logistic has a strange (I couldn’t say odd again) little default. If your dependent variable Y is coded 0 and 1, SAS will model the probability of Y=0. Most of us are trying to model the probability that Y=1. So, yes, your results ARE backward, but only because SAS is testing a hypothesis opposite yours.
Luckily, SAS made the solution easy. Simply add the ‘Descending’ option right in the proc logisitic command line. For example:
PROC LOGISTIC DESCENDING;
MODEL Y = X1 X2;
RUN;
All of your parameter estimates (B) will reverse signs, although p-values will not be affected.
[Logistic_Regression_Workshop]
Adding interaction terms to a regression model can greatly expand understanding of the relationships among the variables in the model and allows more hypotheses to be tested.
The example from Interpreting Regression Coefficients was a model of the height of a shrub (Height) based on the amount of bacteria in the soil (Bacteria) and whether the shrub is located in partial or full sun (Sun). Height is measured in cm, Bacteria is measured in thousand per ml of soil, and Sun = 0 if the plant is in partial sun, and Sun = 1 if the plant is in full sun.
The regression equation was estimated as follows:
Height = 42 + 2.3*Bacteria + 11*Sun
It would be useful to add an interaction term to the model if we wanted to [Read more…] about Interpreting Interactions in Regression