regression models

The Distribution of Independent Variables in Regression Models

I often hear concern about the non-normal distributions of independent variables in regression models, and I am here to ease your mind.

There are NO assumptions in any linear model about the distribution of the independent variables. Yes, you only get meaningful parameter estimates from nominal (unordered categories) or numerical (continuous or discrete) independent variables. But no, the model makes no assumptions about them. They do not need to be normally distributed or continuous.

It is useful, however, to understand the distribution of predictor variables to find influential outliers or concentrated values. A highly skewed independent variable may be made more symmetric with a transformation.

Is Multicollinearity the Bogeyman?

by Karen Grace-Martin Leave a Comment

Multicollinearity occurs when two or more predictor variables in a regression model are redundant. It is a real problem, and it can do terrible things to your results. However, the dangers of multicollinearity seem to have been so drummed into students’ minds that it created a panic.

True multicolllinearity (the kind that messes things up) is pretty uncommon. High correlations among predictor variables may indicate multicollinearity, but it is NOT a reliable indicator that it exists. It does not necessarily indicate a problem. How high is too high depends on [Read more…] about Is Multicollinearity the Bogeyman?

Interpreting the Intercept in a Regression Model

by Karen Grace-Martin 43 Comments

Interpreting the Intercept in a regression model isn’t always a straightforward as it looks. Here’s the definition: the intercept (often labeled the constant) is the expected mean value of Y when all X=0.

Start with a regression equation with one predictor, X.

If X sometimes equals 0, the intercept is simply the expected mean value of Y at that value. That’s meaningful.

If X never equals 0, then the intercept has no intrinsic meaning. Both these scenarios are common in real data. [Read more…] about Interpreting the Intercept in a Regression Model