• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • Home
  • About
    • Our Programs
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Guest Instructors
  • Membership
    • Statistically Speaking Membership Program
    • Login
  • Workshops
    • Online Workshops
    • Login
  • Consulting
    • Statistical Consulting Services
    • Login
  • Free Webinars
  • Contact
  • Login

The Difference Between a Chi-Square Test and a McNemar Test

by Karen Grace-Martin 20 Comments

You may have heard of McNemar tests as a repeated measures version of a chi-square test of independence. This is basically true, and I wanted to show you how these two tests differ and what exactly, each one is testing.

First of all, although Chi-Square tests can be used for larger tables, McNemar tests can only be used for a 2×2 table.  So we’re going to restrict the comparison to 2×2 tables.

The Chi-square test

Here’s an example of a contingency table that would typically be tested with a Chi-Square Test of Independence:

https://i1.wp.com/www.theanalysisfactor.com/wp-content/uploads/2014/08/newsletter-graph-0814.jpg?w=450

The Chi-Square will test whether Experiencing Joint Pain is associated with running more than 25km/week.

How is it doing that?

The chi-square statistic itself is calculated based on the counts of people in each of those four cells of the table and their subsequent row and column totals.

But the comparison it essentially boils down to is the comparison of the two purple percentages.  You’ll notice each of these percentages is based on the row total.  In other words, the 75 non-runners answering Yes to Joint Pain represent 26% of the 290 non-runners*.

But 33% of the 1165 Runners said Yes, they’ve experienced joint pain.  A higher proportion of runners than non-runners are experiencing joint pain.

If those percentages were the same, the chi-square test statistic would be zero and it would mean that whether someone runs tells you nothing about whether they have join pain.

So if those percentages were the same, we’d conclude the two variables are  not associated.

Since our percentages aren’t the same, we conclude that running and joint pain are associated.  (Feel free to check the p-value on this example).

*As a non-runner myself, I’m being strict here in the definition of a “runner” as someone who runs at least 25k/week. All others I’m calling non-runners for simplicity.

The McNemar Test

A McNemar test does something different.

The McNemar is not testing for independence, but consistency in responses across two variables.

Here is a table with the exact same counts, but different variables.  Now we’re comparing whether someone experiences joint pain before and after some treatment.  We want to test whether the treatment worked to change people from Yes to No.

https://i0.wp.com/www.theanalysisfactor.com/wp-content/uploads/2014/08/newsletter-graph-20814.jpg?w=450

But the McNemar recognizes that some people will move from Yes to No and others from No to Yes just randomly.  If the treatment is having no effect, the number of people who move from No to Yes should be about equal to those who move in the other direction.

But if there is a direction to the movement, we’ll see it because one of those purple boxes will be different from the other.

The 215 people who said no at both time points and the 380 people who said Yes at both are actually irrelevant to this comparison.  We’re actually just interested in whether the people who change answers do so randomly or not.

In the McNemar test, we can compare counts directly, because the comparison is not based on row totals.  But if changing to percentages makes interpretation easier, that’s fine too.  Just make sure you use percentage of the total sample, not percentage of the row totals, as we did for Chi-square.

Tagged With: chi-square test, mcnemar test, Repeated Measures

Related Posts

  • Member Training: Seven Fundamental Tests for Categorical Data
  • What to Do When You Can’t Run the Ideal Analysis 
  • Statistical Models for Truncated and Censored Data
  • Six Differences Between Repeated Measures ANOVA and Linear Mixed Models

Reader Interactions

Comments

  1. Show-Hong Duh says

    September 13, 2020 at 2:12 pm

    McNemar test is for paired nominal data, which is not stated in the write-up. Since the data is paired that is why the numbers for the No/No and Yes/Yes are irrelevant.

    Reply
  2. Greg Dams says

    July 16, 2020 at 3:32 pm

    I’m wondering two things:
    First, is there a Fisher’s Exact Test equivalent to the McNemar’s test?
    Second, if category membership is mixed between independent and dependent samples (e.g., some subjects could be represented across groups whereas some other subjects may not), does one use a chi-square or McNemar’s test?
    Thank you in advance for your response!

    Reply
  3. Roberta Palmer says

    November 28, 2019 at 10:10 pm

    How would I set up a contingency table if I were using McNemar test and doing a pre and post audit. Pre data 3 yes/47 no after education intervention,
    post data yes 12/21 no. Thank you

    Reply
  4. Patricia says

    September 22, 2019 at 9:41 pm

    Hi Karen,

    I’m wondering how we’re able to only compare the purple values, 785 and 75. If we’re seeing if the people are randomly going from Yes to No and No to Yes, wouldn’t going Yes to No happen more frequently just because there are more people starting off with Yes?

    Reply
  5. Patricia says

    September 20, 2019 at 8:36 pm

    Hi. Nice write up!

    I’m just wondering how the McNemar test is only comparing the changes from No to Yes and Yes to No. For example, wouldn’t you expect more changes from
    Yes to No than No to Yes if there were more people just starting off as Yes.

    Reply
  6. Priyanka says

    December 4, 2018 at 4:08 am

    Hi,
    loved your explanation. I would like to know how can I apply McNemar to compare baseline and end line data to see the shift in proportions/percentages? There is a drop out rate of 15% from the baseline to the end line (200 in the baseline and 171 in the endline).Please suggest with examples.
    Thanking you in anticipation!
    Regards,

    Reply
  7. Rebecca Martin says

    October 19, 2018 at 8:09 pm

    Thanks for the helpful explanation, Karen!

    Since the McNemar test uses paired data, is it more powerful than the chi-square test of independence, assuming the same general research question is being asked? Like how a paired t-test is more powerful than an independent t-test? Thanks!

    Reply
    • Karen Grace-Martin says

      October 23, 2018 at 4:29 pm

      Hi Rebecca,

      I don’t think it necessarily is, but honestly, I’ve never seen the comparison.

      Paired t-tests are more powerful than between subjects because although they’re essentially testing the same hypothesis about the equivalence of two means, a paired test is able to do it with a lot smaller error variance.

      But McNemar and Chi-square are really testing two different hypotheses.

      Reply
      • Rebecca Martin says

        November 1, 2018 at 3:00 am

        Ah, that makes sense. Thanks Karen!

        Reply
  8. yugoh says

    September 12, 2017 at 6:58 am

    when using McNemar’s test, what will be the sample (N)?

    Reply
  9. Timo says

    April 29, 2017 at 9:36 am

    Nice explanation of the differences between these two Tests for categorial data. I love it.

    Reply
  10. Senjuti Kabir says

    April 11, 2017 at 12:47 am

    If I want to compare any test value (eg. interferon-gamma level) among same group of people before and after treament or any intervention, which will be the ideal test?

    Reply
    • Sylvester Ilo says

      October 27, 2019 at 9:40 pm

      The nature of data determines the type of test tool. The explanation here is for categorical data where data measure is nominal in nature (Yes or no, good or bad which represents the feelings or opinions of the sample.
      For your test of comparison of interferon-gamma levels of pre and post-treatment intervention among same group, you need to use paired-sample t-test.

      Reply
  11. zulaikha says

    January 17, 2017 at 11:12 pm

    What if we have pre post but 3 group independent variable? It’s 3 by 2, can we proceed with mc nemar?

    Reply
    • Karen says

      January 18, 2017 at 11:05 am

      Nope. McNemar only works in a 2×2

      Reply
  12. larry says

    July 29, 2016 at 2:58 pm

    What about he Cochrane Q? I thought it was an extension of McNemar test for the case of tables greater than 2X2.

    Reply
  13. Sabrina says

    July 15, 2016 at 10:39 am

    Hi, I was wondering whether either of these tests would be appropriate for assessing the association between repeat questions which were inserted into a survey as measure of internal validity.

    Reply
  14. Lauren says

    May 22, 2016 at 11:03 pm

    Hi, I am wondering whether there is a chi sq repeated measures equivalent where the data is not binary? I know you can use McNemar for 2×2 but my data includes >2 categories (likert not at all = 1 to very much = 5) and >2 time points (e.g. intake, progress 1, exit, follow up 1).
    Thanks very much.

    Reply
    • Karen says

      June 3, 2016 at 9:10 am

      Hi Lauren,

      Not that I know of for nominal variables or for 3 time points. You’d have to run a GEE logistic regression model with one predictor.

      Reply
    • mayur says

      December 28, 2017 at 12:54 am

      You can use stuert Maxwell test for greater than 3 categories it is an extension of Mc neymar’s test.

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Please note that, due to the large number of comments submitted, any questions on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.

Primary Sidebar

Free Webinars

Binary, Ordinal, and Multinomial Logistic Regression for Categorical Outcomes (Signup)

This Month’s Statistically Speaking Live Training

  • April Member Training: Statistical Contrasts

Upcoming Workshops

  • Logistic Regression for Binary, Ordinal, and Multinomial Outcomes (May 2021)
  • Introduction to Generalized Linear Mixed Models (May 2021)

Read Our Book



Data Analysis with SPSS
(4th Edition)

by Stephen Sweet and
Karen Grace-Martin

Statistical Resources by Topic

  • Fundamental Statistics
  • Effect Size Statistics, Power, and Sample Size Calculations
  • Analysis of Variance and Covariance
  • Linear Regression
  • Complex Surveys & Sampling
  • Count Regression Models
  • Logistic Regression
  • Missing Data
  • Mixed and Multilevel Models
  • Principal Component Analysis and Factor Analysis
  • Structural Equation Modeling
  • Survival Analysis and Event History Analysis
  • Data Analysis Practice and Skills
  • R
  • SPSS
  • Stata

Copyright © 2008–2021 The Analysis Factor, LLC. All rights reserved.
877-272-8096   Contact Us

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled

Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.

Non-necessary

Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.

SAVE & ACCEPT