• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • Home
  • Our Programs
    • Membership
    • Online Workshops
    • Free Webinars
    • Consulting Services
  • About
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Collaborate with Us
  • Statistical Resources
  • Contact
  • Blog
  • Login

Interpreting Regression Coefficients in Models other than Ordinary Linear Regression

by Karen Grace-Martin 2 Comments

Someone who registered for my upcoming Interpreting (Even Tricky) Regression Models workshop asked if the content applies to logistic regression as well.

The short answer: Yes

The long-winded detailed explanation of why this is true and the one caveat:

One of the greatest things about regression models is that they all have the same set up:

The left hand side is the response, Y.  It gets an i subscript because each individual has its own value of Y.

The Xs are the predictor variables.  And the βs, the coefficients, tell you about how each X relates to Y, in the context of the presence of the other predictors.  This is the part we really want to find out.

The residuals, ε, are the part of Y that doesn’t relate to the Xs.  They’re important to the model because if we misrepresent how they behave, it means we are also misrepresenting the βs.  But as long as we get a picture of their behavior right, we can make good inferences about how Y relates to the Xs.

So this is the actual model for an ordinary least squares linear regression.  The left hand side of the equation is just Y and ε, the error term, has a normal distribution.

 

Free Webinar - Interpreting Linear Regression CoefficientsLearn how to best approach coefficients that are often times tricky to interpret.
Free Webinar - Interpreting Linear Regression Coefficients: A Walk Through Output.
 

 

For other types of regression models, like logistic regression, Poisson regression, or multilevel models, all the βs and Xs stay the same.  The only parts that can differ:

1. Instead of Y on the left, there can be a function of the mean of Y–a link function.

2. Instead of a normal distribution, Y|X can have another distribution.

So for example, in a logistic regression, the function of Y is a logit (a.k.a. log-odds) function and the distribution of Y|X is binomial.

And in a multilevel model, there is no special transformation of Y, but the residual gets split into two pieces, both of which are normally distributed.

But as I said, the βs and the Xs don’t change.

The interpretation of each coefficient (at least the trickiest parts, which have to do with the Xs) is about two things:

1. the structure of X

2. what other Xs are in the model.

So interpreting coefficients is done the same basic way regardless of how Y and ε behave.  And that’s what we cover in the majority of the workshop–centering, dummy variables, interactions, correlated predictors, etc.

The one caveat:

But logistic regression also involves that link function on the mean of Y, so there is an extra step involved in interpreting logistic regression coefficients.  So you still need to understand the centering, dummy variables, etc., but you need to understand the logit link function as well.

This is slightly different from doing a transformation of Y, which is usually done to correct non-constant variance or skewed residuals.

I had not planned to, but will briefly add interpreting transformed Y’s and link functions into the workshop.

And if you want more information about the transformation and what it does to all types of logistic regression coefficients, I did a webinar last year on  logistic regression.  You can watch the webinar here.

So you still need to understand dummy variables, centering, correlated predictors, and all that tricky stuff to interpret the odds ratios.  Which is why I recommend learning the tricky stuff in the context of the (relatively) simple linear model before tackling more complicated models.


Bookmark and Share

 

Interpreting Linear Regression Coefficients: A Walk Through Output
Learn the approach for understanding coefficients in that regression as we walk through output of a model that includes numerical and categorical predictors and an interaction.

Tagged With: count model, interpreting regression coefficients, logistic regression

Related Posts

  • Member Training: Types of Regression Models and When to Use Them
  • How to Combine Complicated Models with Tricky Effects
  • Making Dummy Codes Easy to Keep Track of
  • Interpreting the Intercept in a Regression Model

Reader Interactions

Comments

  1. C says

    June 30, 2011 at 7:39 am

    One very stupid question.

    If I make a normal regression does my response variable also have to be normal, or just the residuals?

    I am modelling a process that doesn’t seem like any known distribution, but if I model it with a normal regression, the residuals are normal..

    Thanks in advance!

    Reply
    • Karen says

      June 30, 2011 at 3:22 pm

      Not stupid at all! I see even seasoned researchers with the same question, and it took me a while to figure it out too. 🙂

      The assumption is just about the residuals. The response does NOT have to be normal.

      It happens more often than you’d think, actually, that the response is not normal, but the residuals are. It’s because the X variables are affecting the distribution of Y, but not the residuals.

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Please note that, due to the large number of comments submitted, any questions on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.

Primary Sidebar

This Month’s Statistically Speaking Live Training

  • Member Training: Analyzing Pre-Post Data

Upcoming Free Webinars

Poisson and Negative Binomial Regression Models for Count Data

Upcoming Workshops

  • Analyzing Count Data: Poisson, Negative Binomial, and Other Essential Models (Jul 2022)
  • Introduction to Generalized Linear Mixed Models (Jul 2022)

Copyright © 2008–2022 The Analysis Factor, LLC. All rights reserved.
877-272-8096   Contact Us

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT