Opposite Results in Ordinal Logistic Regression—Solving a Statistical Mystery

by Karen Grace-Martin

FacebookTwitterGoogle+Share

A number of years ago when I was still working in the consulting office at Cornell, someone came in asking for help interpreting their ordinal logistic regression results.

The client was surprised because all the coefficients were backwards from what they expected, and they wanted to make sure they were interpreting them correctly.

It looked like the researcher had done everything correctly, but the results were definitely bizarre. They were using SPSS and the manual wasn’t clarifying anything for me, so I did the logical thing: I ran it in another software program. I wanted to make sure the problem was with interpretation, and not in some strange default or mistake in running the program.

I used Minitab. Lo and behold, most of the parameter estimates were identical, but with signs opposite those from SPSS.

Now, if all of them had reversed sign, it would have been a simple matter of different defaults.

But the intercept was the same in both packages. Only the rest of the parameter estimates were backwards.

Curiouser and curiouser.

Luckily, in my office, we had access to at least four different packages, and we found (by now, the whole office was in on solving the mystery) that SPSS and Stata gave us one result and Minitab and SAS gave the other.

Here’s an example of ordinal logistic regression from SPSS and SAS output. For the record, SPSS uses “Threshold” for Intercept:

You can see that indeed, all the coefficients (Estimate column) are identical, but with opposite signs. Except the intercepts, which are identical.

Mystery Solved

As it turns out, SPSS and Stata did something clever.

SAS and Minitab parameterize the model in the usual way—the same way any regression model does:

It makes interpretation difficult though, because those Fijs represent cumulative probabilities.

Fi1 is the probability that Y = 1, the lowest ordered category.
Fi2 is the probability that Y ≤ 2, the lowest two ordered categories.
Fi3 is the probability that Y ≤ 3, the lowest three ordered categories, and so on.

Each odds ratio (exp(beta)) represents the factor increase in the odds of moving into a lower ordered category for each one-unit increase in X.

In other words, as X gets bigger, a positive beta means higher odds of a lower ordered category.

That is intuitively backward.

It’s not wrong. It just takes a little bit more thought to keep track of all the directions of all these variables.

To make your life just a little easier, SPSS and Stata run the model this way instead:

See that very important minus sign? It flips the estimation of all the coefficients other than the intercept.

In this version of the model, positive values of beta indicate higher odds of moving to the next higher ordered category for higher values of X.

Which makes a little more sense.

The lessons:

  1. Know your software defaults. It was a strange output that led us to look up this one, but not all incorrect results look strange.
  2. This will also be crucial if you use these results to calculate predicted probabilities.
  3. Knowing at least two software packages is a huge advantage. It helps you figure out if strange results are a software issue or a statistical one.
  4. Wow, my job is so fun. (Really).

(Postscript: I received the following email message from a reader of this blog post.)

Bookmark and Share

{ 6 comments… read them below or add one }

abay December 1, 2016 at 2:11 am

I have a project on ordinal logistic regression using spss the how to interprete the result so send me an example with related to this

Reply

Vagner Camilotti January 30, 2016 at 6:18 pm

To obtain OR in SPSS, do we need to compute exp(b) or exp(-b)? I’ve seen these two and I’m a little confused.

Reply

Karen February 22, 2016 at 2:26 pm

Vagner, exp(b). You will still interpret it in the same direction–> higher X leads to more likely higher categories if OR>1

Reply

Melinda December 16, 2013 at 11:16 am

I remember reading about this back in grad school (I/O Psych) in Tabachnick & Fidel. They compared four different stats programs and noted this issue. Whenever I do logistic regression (not often), I always have to refresh my memory on how to interpret it in SAS.

Reply

Dharmendra Naidu December 8, 2013 at 6:50 pm

Hi,

Which version of SAS has this problem. I have a result opposite to what I have expected but I want to be 100 percent confident that this is due to internal SAS programming.

Kind Regards,
Dharmendra

Reply

Karen December 9, 2013 at 10:48 am

Hi Dharmendra,

It’s not a problem with SAS. It’s just the way they parameterize the model. As far as I know, it’s across all versions.

Reply

Leave a Comment

Please note that Karen receives hundreds of comments at The Analysis Factor website each week. Since Karen is also busy teaching workshops, consulting with clients, and running a membership program, she seldom has time to respond to these comments anymore. If you have a question to which you need a timely response, please check out our low-cost monthly membership program, or sign-up for a quick question consultation.

Previous post:

Next post: