Analyzing Pre-Post Data with Repeated Measures or ANCOVA

by Karen Grace-Martin

Not too long ago, I received a call from a distressed client. Let’s call her Nancy.

Nancy had asked for advice about how to run a repeated measures analysis. The advisor told Nancy that actually, a repeated measures analysis was inappropriate for her data.

Nancy was sure repeated measures was appropriate and the response led her to fear that she had grossly misunderstood a very basic tenet in her statistical training.

The Design

Nancy had measured a response variable at two time points for two groups: an intervention group, who received a treatment, and a control group, who did not.

Both groups were measured before and after the intervention.

The Analysis

Nancy was sure that this was a classic repeated measures experiment with one between subjects factor (treatment group) and one within-subjects factor (time).

The advisor insisted that this was a classic pre-post design, and that the way to analyze pre-post designs is not with a repeated measures ANOVA, but with an ANCOVA.

In ANCOVA, the dependent variable is the post-test measure. The pre-test measure is not an outcome, but a covariate. This model assesses the differences in the post-test means after accounting for pre-test values.

The advisor said repeated measures ANOVA is only appropriate if the outcome is measured multiple times after the intervention. The more she insisted repeated measures didn’t work in Nancy’s design, the more confused Nancy got.

The Research Question

This kind of situation happens all the time, in which a colleague, a reviewer, or a statistical consultant insists that you need to do the analysis differently. Sometimes they’re right, but sometimes, as was true here, the two analyses answer different research questions.

Nancy’s research question was whether the mean change in the outcome from pre to post differed in the two groups.

This is directly measured by the time*group interaction term in the repeated measures ANOVA.

The ANCOVA approach answers a different research question: whether the post-test means, adjusted for pre-test scores, differ between the two groups.

In the ANCOVA approach, the whole focus is on whether one group has a higher mean after the treatment. It’s appropriate when the research question is not about gains, growth, or changes.

The adjustment for the pre-test score in ANCOVA has two benefits. One is to make sure that any post-test differences truly result from the treatment, and aren’t some left-over effect of (usually random) pre-test differences between the groups.

The other is to account for variation around the post-test means that comes from the variation in where the patients started at pretest.

So when the research question is about the difference in means at post-test, this is a great option. It’s very common in medical studies because the focus there is about the size of the effect of the treatment.

The Resolution

As it turned out, the right analysis to accommodate Nancy’s design and answer her research question was the Repeated Measures ANOVA. (For the record, linear mixed models also work, and had some advantages, but in this design, the results are identical).

The person she’d asked for advice was in a medical field, and had been trained on the ANCOVA approach.

Either approach works well in specific situation. The one thing that doesn’t is to combine the two approaches.

I’ve started to see situations, particularly when there is more than one post-test measurement, where data analysts attempt to use the baseline pre-test score as both a covariate and the first outcome measure in a repeated measures analysis.

That doesn’t work, because both approaches remove subject-specific variation, so it tries to remove that variation twice.

{ 3 comments… read them below or add one }

Nancy Novotny

Thank you so much for your initial post and explanation (in this long line of posts). Your initial post was so VERY clear and helped me understand how to deal with the specific data I analyzing it just when I needed it!

Reply

Richa Nautiyal

Hi,
Excellent article! It really helped me understand repeated measures a bit better. I still have a few doubts, I hope you could help me out. I have a the Pretest Posttest Follow up Design with Control Group design. I am measuring Psychological well-being among school children with 50 subjects in each experimental and control group and have given intervention only to the experimental group.

My doubt is 1) Can I use repeated measures design? 2) If I can which all output table from SPSS should be considered according to APA format. I am having confusion whether to add between-subject effects table or not. Some places they say its not essential and places they say it should be taken into consideration. So which is it? And what is the difference between within-subject effect for factor*group and between-subject effect?

Thank you in advance.

Reply

Karen Grace-Martin

Hi Richa,

I don’t have enough information to answer. We have a program set up called Statistically Speaking that is designed for situations like this. We have both live Q&A and a forum so that you can ask questions and we can ask you all the contextual clarifying questions to really understand all the details.

Reply

Leave a Comment

Please note that, due to the large number of comments submitted, any comments on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to answers and more resources 24/7.

Previous post:

Next post: