The Difference Between a Chi-Square Test and a McNemar Test

You may have heard of McNemar tests as a repeated measures version of a chi-square test of independence. This is basically true, and I wanted to show you how these two tests differ and what exactly, each one is testing.

First of all, although Chi-Square tests can be used for larger tables, McNemar tests can only be used for a 2×2 table.  So we’re going to restrict the comparison to 2×2 tables.

The Chi-square test

Here’s an example of a contingency table that would typically be tested with a Chi-Square Test of Independence:

The Chi-Square will test whether Experiencing Joint Pain is associated with running more than 25km/week.

How is it doing that?

The chi-square statistic itself is calculated based on the counts of people in each of those four cells of the table and their subsequent row and column totals. 

But the comparison it essentially boils down to is the comparison of the two purple percentages.  You’ll notice each of these percentages is based on the row total.  In other words, the 75 non-runners answering Yes to Joint Pain represent 26% of the 290 non-runners*.

But 33% of the 1165 Runners said Yes, they’ve experienced joint pain.  A higher proportion of runners than non-runners are experiencing joint pain.

If those percentages were the same, the chi-square test statistic would be zero and it would mean that whether someone runs tells you nothing about whether they have join pain.

So if those percentages were the same, we’d conclude the two variables are  not associated.

Since our percentages aren’t the same, we conclude that running and joint pain are associated.  (Feel free to check the p-value on this example).

*As a non-runner myself, I’m being strict here in the definition of a “runner” as someone who runs at least 25k/week. All others I’m calling non-runners for simplicity.

The McNemar Test

A McNemar test does something different.

The McNemar is not testing for independence, but consistency in responses across two variables.

Here is a table with the exact same counts, but different variables.  Now we’re comparing whether someone experiences joint pain before and after some treatment.  We want to test whether the treatment worked to change people from Yes to No.

But the McNemar recognizes that some people will move from Yes to No and others from No to Yes just randomly.  If the treatment is having no effect, the number of people who move from No to Yes should be about equal to those who move in the other direction.

But if there is a direction to the movement, we’ll see it because one of those purple boxes will be different from the other.

The 215 people who said no at both time points and the 380 people who said Yes at both are actually irrelevant to this comparison.  We’re actually just interested in whether the people who change answers do so randomly or not.

In the McNemar test, we can compare counts directly, because the comparison is not based on row totals.  But if changing to percentages makes interpretation easier, that’s fine too.  Just make sure you use percentage of the total sample, not percentage of the row totals, as we did for Chi-square.

Tagged as: Chi square test, mcnemar test, Repeated Measures