• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • Home
  • About
    • Our Programs
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Guest Instructors
  • Membership
    • Statistically Speaking Membership Program
    • Login
  • Workshops
    • Online Workshops
    • Login
  • Consulting
    • Statistical Consulting Services
    • Login
  • Free Webinars
  • Contact
  • Login

The Difference Between Logistic and Probit Regression

by Karen Grace-Martin 12 Comments

One question that seems to come up pretty often is:

What is the difference between logistic and probit regression?

 

Well, let’s start with how they’re the same:

Both are types of generalized linear models. This means they have this form:

glm
Both can be used for modeling the relationship between one or more numerical or categorical predictor variables and a categorical outcome.

Both have versions for binary, ordinal, or multinomial categorical outcomes. And each of these requires specific coding of the outcome. For example, in both logistic and probit models, a binary outcome must be coded as 0 or 1.

So logistic and probit models can be used in the exact same situations. How do they differ?

The real difference is theoretical: they use different link functions.

In generalized linear models, instead of using Y as the outcome, we use a function of the mean of Y. This is the link function.

A logistic regression uses a logit link function:
logit_link
And a probit regression uses an inverse normal link function:

probit_link
These are not the only two link functions that can be used for categorical data, but they’re the most common.

Think about the binary case: Y can have only values of 1 or 0, and we’re really interested in how a predictor relates to the probability that Y=1. But we can’t use the probability itself as the function above. There are two big reasons:

1. Probability can only have values between 0 and 1, whereas the right hand side of the equation can vary from -∞ to ∞.

2. The relationship between probability and the predictors isn’t linear, it’s sigmoidal (a.k.a., S-shaped).

So we need a function of the probability that does two things: (1) converts a probability into a value that runs from -∞ to ∞ and (2) has a linear relationship with the Xs. Probit and Logistic functions both do that.

The difference in the overall results of the model are usually slight to non-existent, so on a practical level it doesn’t usually matter which one you use.

The choice usually comes down to interpretation and communication.

Interpretation:

Anyone who has ever struggled to interpret an odds ratio may find it difficult to believe that a logistic link leads to more intuitive coefficients. Because we can back transform those log-odds into odds ratios, we can get a somewhat intuitive way to interpret effects.

With a probit link, it’s not so easy. After all, what does that inverse normal really mean?

Remember back to intro stats when you had to look up in Z tables the area under the normal curve for a specific Z value? That area represents a cumulative probability: the probability that Z is less than or equal to the specified Z value.

 

normal_curve

When we do the inverse normal transformation, we’re going in the opposite direction: for any cumulative probability, what is the corresponding Z value?

(See how there is a direct conversion from a probability to a number line that runs from -∞ to ∞)?

So you can think of the probit function as the Z (standard normal) value that corresponds to a specific cumulative probability.

Coefficients for probit models can be interpreted as the difference in Z score associated with each one-unit difference in the predictor variable.

Not very intuitive.

Another way to interpret these coefficients is to use the model to calculate predicted probabilities at different values of X.

Remember, though, just like in logistic regression, the difference in the probability isn’t equal for each 1-unit change in the predictor. The sigmoidal relationship between a predictor and probability is nearly identical in probit and logistic regression. A 1-unit difference in X will have a bigger impact on probability in the middle than near 0 or 1.

That said, if you do enough of these, you can certainly get used the idea. Then you will start to have a better idea of the size of each Z-score difference.

Communication:

In some fields, the convention is to use a probit model. Even if you get the same results from a logistic model, it’s worth sticking to the one your audience is familiar with if you’re in one of those fields. After all, why confuse your audience when the results are the same?

Understanding Probability, Odds, and Odds Ratios in Logistic Regression
Despite the way the terms are used in common English, odds and probability are not interchangeable. Join us to see how they differ, what each one means, and how to tame that tricky beast: Odds Ratios.

Tagged With: categorical outcome, generalized linear models, inverse normal link, link function, logistic regression, logit link, probit regression

Related Posts

  • Generalized Linear Models in R, Part 3: Plotting Predicted Probabilities
  • Generalized Linear Models in R, Part 1: Calculating Predicted Probability in Binary Logistic Regression
  • How to Decide Between Multinomial and Ordinal Logistic Regression Models
  • The Difference Between Link Functions and Data Transformations

Reader Interactions

Comments

  1. Maor says

    November 17, 2020 at 6:19 am

    Thank you for the clear explanation!

    I have a question related to Propensity score analysis, are you familiar with this?

    Thank you!

    Reply
  2. Yusuf Abdulfatah says

    June 21, 2020 at 11:03 am

    Thank so much for making it simple and straight forward, please i will need more note on binary probit distribution model

    Reply
  3. Keri says

    May 4, 2020 at 11:42 am

    Thanks for this. So does it mean that you cannot get odds ratios when you are starting with probit regressions?

    Reply
    • Karen Grace-Martin says

      May 4, 2020 at 1:34 pm

      Correct.

      Reply
    • John Maindonald says

      December 8, 2020 at 3:37 am

      For values of p between 0.01 and 0.99 (or even beyond those limits, depending on how finicky you want to be),
      the straight line relationship logit(p) = 1.77 probit(p) can be used to translate to a logit scale, and hence to an odds ratio,

      Reply
  4. geng chuol says

    October 23, 2019 at 11:24 am

    i like to use these models in my research

    Reply
  5. bgal says

    January 8, 2019 at 8:41 pm

    Thanks so much.Please I need a worked examples for better clarification.Better a real life situation.Thanks.

    Reply
  6. Matthew T Brenneman says

    December 19, 2018 at 1:02 pm

    I am curious why the claim that the probit and logit are basically indistinguishable is true. Both functions do yield sigmoid curves that pass through (0.5,0) but the deviation between the functions becomes non-trivial as p goes to either 0 and 1. And even for values near 0.5, the absolute relative difference in the functions (abs{[logit(p)-probit(p)]/probit(p)}) is about 50%.
    Is the proof of this statement from an analysis of the sampling distributions for the estimators using these two models?
    Thanks!

    Reply
    • John Maindonald says

      December 9, 2020 at 2:58 am

      logit(p), for p between 0.01 and 0.99, is close to 1.77 probit(p)
      For most practical purposes, they differ only by a scale factor.
      With p=0.5, both equal 0. You should get something like NaN for (0-0)/0.

      Reply
  7. Kim says

    January 12, 2018 at 12:50 pm

    “So we need a function of the probability that does two things: (1) converts a probability into a value that runs from -∞ to ∞ and (2) has a linear relationship with the Xs. Probit and Logistic functions both do that.”

    Did you meant “non-linear relationship”? Weren’t you talking about why to use logit and probit instead of linear function?

    I might have misunderstood though! Thanks!

    Reply
    • Karen says

      April 11, 2018 at 12:21 pm

      Nope. Linear.

      While P has a non-linear relationship with the Xs, once you apply the link function, the relationship becomes linear.

      So logit(P) or probit(P) both have linear relationships with the Xs. P doesn’t.

      That’s why you get coefficients on the scale of the link function that could be interpreted just like linear regression coefficients: for each 1-unit difference in X leads to a b unit difference in the log-odds of P.

      But that’s not intuitive, so we back-transform to get an odds-ratio, which is no longer linear since we’ve taken it off the log scale.

      Reply
  8. Upendra Lele says

    December 14, 2017 at 12:54 am

    Thanks for sharing this valuable information with such clarity and simplicity.
    Regards. Upendra

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Please note that, due to the large number of comments submitted, any questions on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.

Primary Sidebar

Free Webinars

Effect Size Statistics on Tuesday, Feb 2nd

This Month’s Statistically Speaking Live Training

  • January Member Training: A Gentle Introduction To Random Slopes In Multilevel Models

Upcoming Workshops

  • Logistic Regression for Binary, Ordinal, and Multinomial Outcomes (May 2021)
  • Introduction to Generalized Linear Mixed Models (May 2021)

Read Our Book



Data Analysis with SPSS
(4th Edition)

by Stephen Sweet and
Karen Grace-Martin

Statistical Resources by Topic

  • Fundamental Statistics
  • Effect Size Statistics, Power, and Sample Size Calculations
  • Analysis of Variance and Covariance
  • Linear Regression
  • Complex Surveys & Sampling
  • Count Regression Models
  • Logistic Regression
  • Missing Data
  • Mixed and Multilevel Models
  • Principal Component Analysis and Factor Analysis
  • Structural Equation Modeling
  • Survival Analysis and Event History Analysis
  • Data Analysis Practice and Skills
  • R
  • SPSS
  • Stata

Copyright © 2008–2021 The Analysis Factor, LLC. All rights reserved.
877-272-8096   Contact Us

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled

Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.

Non-necessary

Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.