linear regression

Interpreting Lower Order Coefficients When the Model Contains an Interaction

February 23rd, 2009 by

A Linear Regression Model with an interaction between two predictors (X1 and X2) has the form: 

Y = B0 + B1X1 + B2X2 + B3X1*X2.

It doesn’t really matter if X1 and X2 are categorical or continuous, but let’s assume they are continuous for simplicity.

One important concept is that B1 and B2 are not main effects, the way they would be if (more…)


3 Reasons Psychology Researchers should Learn Regression

February 17th, 2009 by

Stage 2Back when I was doing psychology research, I knew ANOVA pretty well.  I’d taken a number of courses on it and could run it backward and forward.  I kept hearing about ANCOVA, but in every ANOVA class that was the last topic on the syllabus, and we always ran out of time.

The other thing that drove me crazy was those stats professors kept saying “ANOVA is just a special case of Regression.”  I could not for the life of me figure out why or how.

It was only when I switched over to statistics that I finally took a regression class and figured out what ANOVA was all about. And only when I started consulting, and seeing hundreds of different ANOVA and regression models, that I finally made the connection.

But if you don’t have the driving curiosity about ANOVA and regression, why should you, as a researcher in Psychology, Education, or Agriculture, who is trained in ANOVA, want to learn regression?  There are 3 main reasons.

1. There a many, many continuous independent variables and covariates that need to be included in models.  Without the tools to analyze them as continuous, you are left forcing them into ANOVA using an arbitrary technique like median splits.  At best, you’re losing power.  At worst, you’re not publishing your article because you’re missing real effects.

2. Having a solid understanding of the General Linear Model in its various forms equips you to really understand your variables and their relationships.  It allows you to try a model different ways–not for data fishing, but for discovering the true nature of the relationships.  Having the capacity to add an interaction term or a squared term  allows you to listen to your data and makes you a better researcher.

3. The multiple linear regression model is the basis for many other statistical techniques–logistic regression, multilevel and mixed models, Poisson regression, Survival Analysis, and so on.  Each of these is a step (or small leap) beyond multiple regression.  If you’re still struggling with what it means to center variables or interpret interactions, learning one of these other techniques becomes arduous, if not painful.

Having guided thousands of researchers through their statistical analysis over the past 10 years, I am convinced that having a strong, intuitive understanding of the general linear model in its variety of forms is the key to being an effective and confident statistical analyst.  You are then free to learn and explore other methodologies as needed.

 


When NOT to Center a Predictor Variable in Regression

February 9th, 2009 by

There are two reasons to center predictor variables in any type of regression analysis–linear, logistic, multilevel, etc.

1. To lessen the correlation between a multiplicative term (interaction or polynomial term) and its component variables (the ones that were multiplied).

2. To make interpretation of parameter estimates easier.

I was recently asked when is centering NOT a good idea? (more…)


Proportions as Dependent Variable in Regression–Which Type of Model?

January 26th, 2009 by

When the dependent variable in a regression model is a proportion or a percentage, it can be tricky to decide on the appropriate way to model it.

The big problem with ordinary linear regression is that the model can predict values that aren’t possible–values below 0 or above 1.  But the other problem is that the relationship isn’t linear–it’s sigmoidal.  A sigmoidal curve looks like a flattened S–linear in the middle, but flattened on the ends.  So now what?

The simplest approach is to do a linear regression anyway.  This approach can be justified only in a few situations.

1. All your data fall in the middle, linear section of the curve.  This generally translates to all your data being between .2 and .8 (although I’ve heard that between .3-.7 is better).  If this holds, you don’t have to worry about the two objections.  You do have a linear relationship, and you won’t get predicted values much beyond those values–certainly not beyond 0 or 1.

2. It is a really complicated model that would be much harder to model another way.  If you can assume a linear model, it will be much easier to do, say, a complicated mixed model or a structural equation model.  If it’s just a single multiple regression, however, you should look into one of the other methods.

A second approach is to treat the proportion as a binary response then run a logistic or probit regression.  This will only work if the proportion can be thought of and you have the data for the number of successes and the total number of trials.  For example, the proportion of land area covered with a certain species of plant would be hard to think of this way, but the proportion of correct answers on a 20-answer assessment would.

The third approach is to treat it the proportion as a censored continuous variable.  The censoring means that you don’t have information below 0 or above 1.  For example, perhaps the plant would spread even more if it hadn’t run out of land.  If you take this approach, you would run the model as a two-limit tobit model (Long, 1997).  This approach works best if there isn’t an excessive amount of censoring (values of 0 and 1).

Reference: Long, J.S. (1997). Regression Models for Categorical and Limited Dependent Variables. Sage Publishing.

 


Interpreting Interactions in Regression

January 19th, 2009 by

Adding interaction terms to a regression model has real benefits. It greatly expands your understanding of the relationships among the variables in the model. And you can test more specific hypotheses.  But interpreting interactions in regression takes understanding of what each coefficient is telling you.

The example from Interpreting Regression Coefficients was a model of the height of a shrub (Height) based on the amount of bacteria in the soil (Bacteria) and whether the shrub is located in partial or full sun (Sun). Height is measured in cm, Bacteria is measured in thousand per ml of soil, and Sun = 0 if the plant is in partial sun, and Sun = 1 if the plant is in full sun.

(more…)


SPSS GLM: Choosing Fixed Factors and Covariates

December 30th, 2008 by

The beauty of the Univariate GLM procedure in SPSS is that it is so flexible.  You can use it to analyze regressions, ANOVAs, ANCOVAs with all sorts of interactions, dummy coding, etc.

The down side of this flexibility is it is often confusing what to put where and what it all means.

So here’s a quick breakdown.

The dependent variable I hope is pretty straightforward.  Put in your continuous dependent variable.

Fixed Factors are categorical independent variables.  It does not matter if the variable is (more…)