Not too long ago, I received a call from a distressed client. Let’s call her Nancy.

Nancy had asked for advice about how to run a repeated measures analysis. The advisor told Nancy that actually, a repeated measures analysis was inappropriate for her data.

Nancy was sure repeated measures *was *appropriate and the response led her to fear that she had grossly misunderstood a very basic tenet in her statistical training.

**The Design**

Nancy had measured a response variable at two time points for two groups: an intervention group, who received a treatment, and a control group, who did not.

Both groups were measured before and after the intervention.

**The Analysis**

Nancy was sure that this was a classic repeated measures experiment with one between subjects factor (treatment group) and one within-subjects factor (time).

The advisor insisted that this was a classic pre-post design, and that the way to analyze pre-post designs is not with a repeated measures ANOVA, but with an ANCOVA.

In ANCOVA, the dependent variable is the post-test measure. The pre-test measure is not an outcome, but a covariate. This model assesses the differences in the post-test means after accounting for pre-test values.

The advisor said repeated measures ANOVA is only appropriate if the outcome is measured multiple times after the intervention. The more she insisted repeated measures didn’t work in Nancy’s design, the more confused Nancy got.

**The Research Question**

This kind of situation happens all the time, in which a colleague, a reviewer, or a statistical consultant insists that you need to do the analysis differently. Sometimes they’re right, but sometimes, as was true here, *the two analyses answer different research questions*.

Nancy’s research question was whether the mean change in the outcome from pre to post differed in the two groups.

This is directly measured by the time*group interaction term in the repeated measures ANOVA.

The ANCOVA approach answers a different research question: whether the post-test means, adjusted for pre-test scores, differ between the two groups.

In the ANCOVA approach, the whole focus is on whether one group has a higher mean after the treatment. It’s appropriate when the research question is not about gains, growth, or changes.

The adjustment for the pre-test score in ANCOVA has two benefits. One is to make sure that any post-test differences truly result from the treatment, and aren’t some left-over effect of (usually random) pre-test differences between the groups.

The other is to account for variation around the post-test means that comes from the variation in where the patients started at pretest.

So when the research question is about the difference in means at post-test, this is a great option. It’s very common in medical studies because the focus there is about the size of the effect of the treatment.

**The Resolution**

As it turned out, the right analysis to accommodate Nancy’s design *and* answer her research question *was* the Repeated Measures ANOVA. (For the record, linear mixed models also work, and had some advantages, but in this design, the results are identical).

The person she’d asked for advice was in a medical field, and had been trained on the ANCOVA approach.

Either approach works well in specific situation. The one thing that doesn’t is to combine the two approaches.

I’ve started to see situations, particularly when there is more than one post-test measurement, where data analysts attempt to use the baseline pre-test score as both a covariate and the first outcome measure in a repeated measures analysis.

That doesn’t work, because both approaches remove subject-specific variation, so it tries to remove that variation twice.

Natnael Gietu says

I have three treatment groups and pre and post-test data. which method of analysis is compatible with my research design?

Karen Grace-Martin says

Hi Natnael,

Both approaches I discussed in the article could work here. It all depends on your research question.

Suleyman Bulut says

Our research is 4 week fish oil supplementation study and following supplementation we applied 1 bout acute resistance exercise. It is placebo controlled, crossover study. We’ve collected blood at baseline, after 4 weeks of supplementation after 1 week wash out and before-after resistance exercise bout. I have two questions: 1) Should I analyze the data with 2 way ANOVA (treatments [fish oil or placebo and acute resistance exercise] and time). 2) Should I normalize the data to baseline values, if so how? Following normalization can I do statistics with normalized data? Many thanks in advance

Pamela Rollins says

I have a pre-post data with randomization from an intervention study. The groups do not have the same number of people in them. I have been trying to figure out the best way to analyze it. In the resolution you say a linear mixed model would be is another way to go. Is that the same as hierarchical linear modeling (HLM)? I considered using HLM but didnt think you could use it with two waves of data. I was also thinking about using a multiple regression model with Time2 of the variable of interest as the outcome and time 1, covariates and group as predictors. Do eigher of these make sense to use or should I use a traditional ANCOVA model.

Vincent says

I think it does not matter you use repeated measure model or ANCOVA as long as it is a randomized design. i.e., you randomly assign subjects into two groups. Eventually two methods will estimate the same thing- difference in mean post-treatment scores between two groups, because the baseline mean scores are equal under randomization. Also, ANCOVA is more efficient than regular repeated measure model (including time, group and time*group) because repeated measure model inherently assumes the baseline means are different between two groups and need to estimate one more parameter. Instead, if you really want to model both pre- and post-treatment scores, you can use a constrained repeated measure model (time, time*group) by forcing the intercept (or difference in baseline score between two groups) equal to 0. This constrained repeated measure model performs comparably to ANCOVA model.

If it is not randomized study, the story will be different. Both models require different assumptions and are not really comparable.

Here is the reference: Statistical analysis of two arm randomized pre-post design with one post-treatment measurement

https://www.researchgate.net/publication/342975051_Statistical_analysis_of_two_arm_randomized_pre-post_design_with_one_post-treatment_measurement

filip says

Very helpful ! I was looking for this! So grateful for finding it!

Kaeli says

Greetings, Karen!

I have a general question related to this statement: “In the ANCOVA approach, the whole focus is on whether one group has a higher mean after the treatment. It’s appropriate when the research question is not about gains, growth, or changes.”

If you used the change score as the outcome in an ANCOVA, where the coefficient for the group variable would be the same as the model which used the post score as the outcome, would that not address a research question about group differences in changes from baseline, adjusting for the pre value?

Thanks,

Kaeli

Karen Grace-Martin says

Hi Kaeli,

Yes, change scores as outcomes work to measure growth. But then you lose the ability to test which group had a higher mean at the beginning or end. It’s only about the change, regardless of where the groups start or end. This can be fine, depending on what you’re interested in testing. It all comes down to the research question.

Kaeli says

Thank you, Karen! I really appreciate your response and your wonderful blog posts – they are always insightful.

Covariate says

There is another case, typical in clinical research – change adjusted for baseline, that is to say, (post-pre)~pre + time. This is also modelled by ANCOVA (GLS), mixed model (if we play with subject-level random effects) or GEE (for popularion-averaged effects).

Prasanth Chandrasekaran says

Hi, This is Prasanth

In my research I have the Pre, Post & Follow up Design with Experimental group and Control Group design. I am investigating risk factors among school children with 65 subjects in each experimental and control group and have given intervention only to the experimental group.

My doubt is 1) Can I use Mixed model ANOVA? 2) If I can, what are all the output table from SPSS should be considered according to APA format. There re lots of table in the output. How to interpret the Mixed model ANOVA

Thank you.

Nancy Novotny says

Thank you so much for your initial post and explanation (in this long line of posts). Your initial post was so VERY clear and helped me understand how to deal with the specific data I analyzing it just when I needed it!

Richa Nautiyal says

Hi,

Excellent article! It really helped me understand repeated measures a bit better. I still have a few doubts, I hope you could help me out. I have a the Pretest Posttest Follow up Design with Control Group design. I am measuring Psychological well-being among school children with 50 subjects in each experimental and control group and have given intervention only to the experimental group.

My doubt is 1) Can I use repeated measures design? 2) If I can which all output table from SPSS should be considered according to APA format. I am having confusion whether to add between-subject effects table or not. Some places they say its not essential and places they say it should be taken into consideration. So which is it? And what is the difference between within-subject effect for factor*group and between-subject effect?

Thank you in advance.

Karen Grace-Martin says

Hi Richa,

I don’t have enough information to answer. We have a program set up called Statistically Speaking that is designed for situations like this. We have both live Q&A and a forum so that you can ask questions and we can ask you all the contextual clarifying questions to really understand all the details.