Linear Regression

What It Really Means to Take an Interaction Out of a Model

When you’re model building, a key decision is which interaction terms to include.

As a general rule, the default in regression is to leave them out. Add interactions only with a solid reason. It would seem like data fishing to simply add in all possible interactions.

And yet, that’s a common practice in most ANOVA models: put in all possible interactions and only take them out if there’s a solid reason. Even many software procedures default to creating interactions among categorical predictors.

Simplifying a Categorical Predictor in Regression Models

by Jeff Meyer Leave a Comment

One of the many decisions you have to make when model building is which form each predictor variable should take. One specific version of this decision is whether to combine categories of a categorical predictor.

The greater the number of parameter estimates in a model the greater the number of observations that are needed to keep power constant. The parameter estimates in a linear [Read more…] about Simplifying a Categorical Predictor in Regression Models

A Visual Description of Multicollinearity

by Karen Grace-Martin 2 Comments

Multicollinearity is one of those terms in statistics that is often defined in one of two ways:

1. Very mathematical terms that make no sense — I mean, what is a linear combination anyway?

2. Completely oversimplified in order to avoid the mathematical terms — it’s a high correlation, right?

So what is it really? In English?

Member Training: Practical Advice for Establishing Reliability and Validity

by guest

How do you know your variables are measuring what you think they are? And how do you know they’re doing it well?

A key part of answering these questions is establishing reliability and validity of the measurements that you use in your research study. But the process of establishing reliability and validity is confusing. There are a dizzying number of choices available to you.

Linear Regression for an Outcome Variable with Boundaries

by Karen Grace-Martin 2 Comments

The following statement might surprise you, but it’s true.

To run a linear model, you don’t need an outcome variable Y that’s normally distributed. Instead, you need a dependent variable that is:

Continuous
Unbounded
Measured on an interval or ratio scale

The normality assumption is about the errors in the model, which have the same distribution as Y|X. It’s absolutely possible to have a skewed distribution of Y and a normal distribution of errors because of the effect of X. [Read more…] about Linear Regression for an Outcome Variable with Boundaries

What is a Confounding Variable?

by Karen Grace-Martin Leave a Comment

By Karen Grace-Martin

Confounding variable is one of those statistical terms that confuses a lot of people. Not because it represents a confusing concept, but because of how it’s used. (Well, it’s a bit of a confusing concept, but that’s not the worst part).

First, it has slightly different meanings to different types of researchers. The definition is essentially the same, but the research context can have specific implications for how that definition plays out.

If the person you’re talking to has a different understanding of what it means, you’re going to have a confusing conversation.

Let’s take a look at some examples to unpack this.