categorical predictor

In a simple linear regression model, how the constant (a.k.a., intercept) is interpreted depends upon the type of predictor (independent) variable.

If the predictor is categorical and dummy-coded, the constant is the mean value of the outcome variable for the reference category only. If the predictor variable is continuous, the constant equals the predicted value of the outcome variable when the predictor variable equals zero.

Removing the Constant When the Predictor Is Categorical

When your predictor variable X is categorical, the results are logical. Let’s look at an example. [Read more…] about The Impact of Removing the Constant from a Regression Model: The Categorical Case

Pros and Cons of Treating Ordinal Variables as Nominal or Continuous

by Karen Grace-Martin 3 Comments

There are not a lot of statistical methods designed just for ordinal variables.

But that doesn’t mean that you’re stuck with few options. There are more than you’d think. [Read more…] about Pros and Cons of Treating Ordinal Variables as Nominal or Continuous

Confusing Statistical Term #6: Factor

by Karen Grace-Martin 6 Comments

Factor is confusing much in the same way as hierarchical and beta, because it too has different meanings in different contexts. Factor might be a little worse, though, because its meanings are related.

In both meanings, a factor is a variable. But a factor has a completely different meaning and implications for use in two different contexts. [Read more…] about Confusing Statistical Term #6: Factor

3 Situations when it makes sense to Categorize a Continuous Predictor in a Regression Model

by Karen Grace-Martin 1 Comment

In many research fields, particularly those that mostly use ANOVA, a common practice is to categorize continuous predictor variables so they work in an ANOVA. This is often done with median splits—splitting the sample into two categories—the “high” values above the median and the “low” values below the median. There are many reasons why this isn’t such a good idea:

the median varies from sample to sample, making the categories in different samples have different meanings
all values on one side of the median are considered equivalent—any variation within the category is ignored, and two values right next to each other on either side of the median are considered different
the categorization is completely arbitrary. A ‘High” score isn’t necessarily high. If the scale is skewed, as many are, even a value near the low end can end up in the “high” category.

But it can be very useful and legitimate to be able to choose whether to treat an independent variable as categorical or continuous. Knowing when it is appropriate [Read more…] about 3 Situations when it makes sense to Categorize a Continuous Predictor in a Regression Model

Continuous and Categorical Variables: The Trouble with Median Splits

by Karen Grace-Martin 10 Comments

A Median Split is one method for turning a continuous variable into a categorical one. Essentially, the idea is to find the median of the continuous variable. Any value below the median is put it the category “Low” and every value above it is labeled “High.”

This is a very common practice in many social science fields in which researchers are trained in ANOVA but not Regression. At least that was true when I was in grad school in psychology. And yes, oh so many years ago, I used all these techniques I’m going to tell you not to.

There are problems with median splits. The first is purely logical. When a continuum is categorized, every value above the median, for example, is considered equal. Does it really make sense that a value just above the median is considered the same as values way at the end? And different than values just below the median? Not so much.

So one solution is to split the sample into three groups, not two, then drop the middle group. This at least creates some separation between the two groups. The obvious problem, here though, is you’re losing a third of your sample.

The second problem with categorizing a continuous predictor, regardless of how you do it, is loss of power (Aiken & West, 1991). It’s simply harder to find effects that are really there.

So why is it common practice? Because categorizing continuous variables is the only way to stuff them into an ANOVA, which is the only statistics method researchers in many fields are trained to do.

Rather than force a method that isn’t quite appropriate, it would behoove researchers, and the quality of their research, to learn the general linear model and how ANOVA fits into it. It’s really only a short leap from ANOVA to regression but a necessary one. GLMs can include interactions among continuous and categorical predictors just as ANOVA does.

If left continuous, the GLM would fit a regression line to the effect of that continuous predictor. Categorized, the model will compare the means. It often happens that while the difference in means isn’t significant, the slope is.

Reference: Aiken & West (1991). Multiple Regression: Testing and interpreting interactions.

Interpreting Regression Coefficients

by Karen Grace-Martin 32 Comments

Linear regression is one of the most popular statistical techniques.

Despite its popularity, interpretation of the regression coefficients of any but the simplest models is sometimes, well….difficult.

So let’s interpret the coefficients of a continuous and a categorical variable. Although the example here is a linear regression model, the approach works for interpreting coefficients from any regression model without interactions, including logistic and proportional hazards models.

A linear regression model with two predictor variables can be expressed with the following equation:

Y = B₀ + B₁*X₁ + B₂*X₂ + e.

The variables in the model are:

Y, the response variable;
X₁, the first predictor variable;
X₂, the second predictor variable; and
e, the residual error, which is an unmeasured variable.

The parameters in the model are:

B₀, the Y-intercept;
B₁, the first regression coefficient; and
B₂, the second regression coefficient.

One example would be a model of the height of a shrub (Y) based on the amount of bacteria in the soil (X₁) and whether the plant is located in partial or full sun (X₂).

Height is measured in cm, bacteria is measured in thousand per ml of soil, and type of sun = 0 if the plant is in partial sun and type of sun = 1 if the plant is in full sun.

Let’s say it turned out that the regression equation was estimated as follows:

Y = 42 + 2.3*X₁ + 11*X₂

Interpreting the Intercept

B₀, the Y-intercept, can be interpreted as the value you would predict for Y if both X₁ = 0 and X₂ = 0.

We would expect an average height of 42 cm for shrubs in partial sun with no bacteria in the soil. However, this is only a meaningful interpretation if it is reasonable that both X₁ and X₂ can be 0, and if the data set actually included values for X₁ and X₂ that were near 0.

If neither of these conditions are true, then B0 really has no meaningful interpretation. It just anchors the regression line in the right place. In our case, it is easy to see that X₂ sometimes is 0, but if X₁, our bacteria level, never comes close to 0, then our intercept has no real interpretation.

Interpreting Coefficients of Continuous Predictor Variables

Since X₁ is a continuous variable, B₁ represents the difference in the predicted value of Y for each one-unit difference in X₁, if X₂ remains constant.

This means that if X₁ differed by one unit (and X₂ did not differ) Y will differ by B₁ units, on average.

In our example, shrubs with a 5000 bacteria count would, on average, be 2.3 cm taller than those with a 4000/ml bacteria count, which likewise would be about 2.3 cm taller than those with 3000/ml bacteria, as long as they were in the same type of sun.

(Don’t forget that since the bacteria count was measured in 1000 per ml of soil, 1000 bacteria represent one unit of X₁).

Interpreting Coefficients of Categorical Predictor Variables

Similarly, B₂ is interpreted as the difference in the predicted value in Y for each one-unit difference in X₂ if X₁ remains constant. However, since X₂ is a categorical variable coded as 0 or 1, a one unit difference represents switching from one category to the other.

B₂ is then the average difference in Y between the category for which X₂ = 0 (the reference group) and the category for which X₂ = 1 (the comparison group).

So compared to shrubs that were in partial sun, we would expect shrubs in full sun to be 11 cm taller, on average, at the same level of soil bacteria.

Interpreting Coefficients when Predictor Variables are Correlated

Don’t forget that each coefficient is influenced by the other variables in a regression model. Because predictor variables are nearly always associated, two or more variables may explain some of the same variation in Y.

Therefore, each coefficient does not measure the total effect on Y of its corresponding variable, as it would if it were the only variable in the model.

Rather, each coefficient represents the additional effect of adding that variable to the model, if the effects of all other variables in the model are already accounted for. (This is called Type 3 regression coefficients and is the usual way to calculate them. However, not all software uses Type 3 coefficients, so make sure you check your software manual so you know what you’re getting).

This means that each coefficient will change when other variables are added to or deleted from the model.

For a discussion of how to interpret the coefficients of models with interaction terms, see Interpreting Interactions in Regression.