One area in statistics where I see conflicting advice is how to analyze pre-post data. I’ve seen this myself in consulting. A few years ago, I received a call from a distressed client. Let’s call her Nancy.
Nancy had asked for advice about how to run a repeated measures analysis. The advisor told Nancy that actually, a repeated measures analysis was inappropriate for her data.
Nancy was sure repeated measures was appropriate. This advice led her to fear that she had grossly misunderstood a very basic tenet in her statistical training.
The Study Design
Nancy had measured a response variable at two time points for two groups. The intervention group received a treatment and a control group did not. Participants were randomly assigned to one of the two groups.
The researcher measured each participant before and after the intervention.
Analyzing the Pre-Post Data
Nancy was sure that this was a classic repeated measures experiment. It has one between subjects factor (treatment group) and one within-subjects factor (time).
The advisor insisted that this was a classic pre-post design, and that the way to analyze pre-post data is not with a repeated measures ANOVA, but with an ANCOVA.
In ANCOVA, the dependent variable is the post-test measure. The pre-test measure is not an outcome, but a covariate. This model assesses the differences in the post-test means after accounting for pre-test values.
The advisor said repeated measures ANOVA is only appropriate if the outcome is measured multiple times after the intervention. The more she insisted repeated measures didn’t work in Nancy’s design, the more confused Nancy got.
The Research Question
This kind of situation happens all the time, in which a colleague, a reviewer, or a statistical consultant insists that you need to do the analysis differently. Sometimes they’re right, but sometimes, as was true here, the two analyses answer different research questions.
Nancy’s research question was whether the mean change in the outcome from pre to post differed in the two groups.
This is directly measured by the time*group interaction term in the repeated measures ANOVA.
The ANCOVA approach answers a different research question: whether the post-test means, adjusted for pre-test scores, differ between the two groups.
In the ANCOVA approach, the whole focus is on whether one group has a higher mean after the treatment. It’s appropriate when the research question is about the mean value at the end. Not about gains, growth, or changes.
The adjustment for the pre-test score in ANCOVA has two benefits. One is to make sure that any post-test differences truly result from the treatment, and aren’t some left-over effect of (usually random) pre-test differences between the groups.
The other is to account for variation around the post-test means that comes from the variation in where the patients started at pretest.
So when the research question is about the difference in means at post-test, this is a great option. It’s very common in medical studies because the focus there is about the size of the effect of the treatment.
The Resolution
As it turned out, the right analysis to accommodate Nancy’s design and answer her research question was the Repeated Measures ANOVA. (For the record, linear mixed models also work. It would have some advantages, but in this design, the results are identical).
The person she’d asked for advice was in a medical field, and had been trained on the ANCOVA approach.
Either approach works well in specific situation. The one thing that doesn’t is to combine the two approaches.
I’ve started to see data analysts attempt to use the baseline pre-test score as both a covariate and the first outcome measure in a repeated measures analysis. Particularly when there is more than one post-test measurement.
That doesn’t work, because both approaches remove subject-specific variation, so it tries to remove that variation twice.
ANCOVA then is the chocie.
One of the assumptions underlying RM ANOVA is that factor levels are randomised within subjects. In the case above, the factor levels are pre and post tests, which is unidirectional. So the factor levels were not randomised with subjects. Therefore, RM ANOVA should NOT be applied.
see reference here: Pat Dugard & John Todman (1995) Analysis of Pre‐test‐Post‐test Control
Group Designs in Educational Research, Educational Psychology, 15:2, 181-198, DOI:
10.1080/0144341950150207
If pretest of some parameters show significant between experiment and control group. How should I do?
I have three treatment groups and pre and post-test data. which method of analysis is compatible with my research design?
Hi Natnael,
Both approaches I discussed in the article could work here. It all depends on your research question.
Our research is 4 week fish oil supplementation study and following supplementation we applied 1 bout acute resistance exercise. It is placebo controlled, crossover study. We’ve collected blood at baseline, after 4 weeks of supplementation after 1 week wash out and before-after resistance exercise bout. I have two questions: 1) Should I analyze the data with 2 way ANOVA (treatments [fish oil or placebo and acute resistance exercise] and time). 2) Should I normalize the data to baseline values, if so how? Following normalization can I do statistics with normalized data? Many thanks in advance
I have a pre-post data with randomization from an intervention study. The groups do not have the same number of people in them. I have been trying to figure out the best way to analyze it. In the resolution you say a linear mixed model would be is another way to go. Is that the same as hierarchical linear modeling (HLM)? I considered using HLM but didnt think you could use it with two waves of data. I was also thinking about using a multiple regression model with Time2 of the variable of interest as the outcome and time 1, covariates and group as predictors. Do eigher of these make sense to use or should I use a traditional ANCOVA model.
I think it does not matter you use repeated measure model or ANCOVA as long as it is a randomized design. i.e., you randomly assign subjects into two groups. Eventually two methods will estimate the same thing- difference in mean post-treatment scores between two groups, because the baseline mean scores are equal under randomization. Also, ANCOVA is more efficient than regular repeated measure model (including time, group and time*group) because repeated measure model inherently assumes the baseline means are different between two groups and need to estimate one more parameter. Instead, if you really want to model both pre- and post-treatment scores, you can use a constrained repeated measure model (time, time*group) by forcing the intercept (or difference in baseline score between two groups) equal to 0. This constrained repeated measure model performs comparably to ANCOVA model.
If it is not randomized study, the story will be different. Both models require different assumptions and are not really comparable.
Here is the reference: Statistical analysis of two arm randomized pre-post design with one post-treatment measurement
https://www.researchgate.net/publication/342975051_Statistical_analysis_of_two_arm_randomized_pre-post_design_with_one_post-treatment_measurement
Very helpful ! I was looking for this! So grateful for finding it!
Greetings, Karen!
I have a general question related to this statement: “In the ANCOVA approach, the whole focus is on whether one group has a higher mean after the treatment. It’s appropriate when the research question is not about gains, growth, or changes.”
If you used the change score as the outcome in an ANCOVA, where the coefficient for the group variable would be the same as the model which used the post score as the outcome, would that not address a research question about group differences in changes from baseline, adjusting for the pre value?
Thanks,
Kaeli
Hi Kaeli,
Yes, change scores as outcomes work to measure growth. But then you lose the ability to test which group had a higher mean at the beginning or end. It’s only about the change, regardless of where the groups start or end. This can be fine, depending on what you’re interested in testing. It all comes down to the research question.
Thank you, Karen! I really appreciate your response and your wonderful blog posts – they are always insightful.
There is another case, typical in clinical research – change adjusted for baseline, that is to say, (post-pre)~pre + time. This is also modelled by ANCOVA (GLS), mixed model (if we play with subject-level random effects) or GEE (for popularion-averaged effects).
Hi, This is Prasanth
In my research I have the Pre, Post & Follow up Design with Experimental group and Control Group design. I am investigating risk factors among school children with 65 subjects in each experimental and control group and have given intervention only to the experimental group.
My doubt is 1) Can I use Mixed model ANOVA? 2) If I can, what are all the output table from SPSS should be considered according to APA format. There re lots of table in the output. How to interpret the Mixed model ANOVA
Thank you.
Thank you so much for your initial post and explanation (in this long line of posts). Your initial post was so VERY clear and helped me understand how to deal with the specific data I analyzing it just when I needed it!
Hi,
Excellent article! It really helped me understand repeated measures a bit better. I still have a few doubts, I hope you could help me out. I have a the Pretest Posttest Follow up Design with Control Group design. I am measuring Psychological well-being among school children with 50 subjects in each experimental and control group and have given intervention only to the experimental group.
My doubt is 1) Can I use repeated measures design? 2) If I can which all output table from SPSS should be considered according to APA format. I am having confusion whether to add between-subject effects table or not. Some places they say its not essential and places they say it should be taken into consideration. So which is it? And what is the difference between within-subject effect for factor*group and between-subject effect?
Thank you in advance.
Hi Richa,
I don’t have enough information to answer. We have a program set up called Statistically Speaking that is designed for situations like this. We have both live Q&A and a forum so that you can ask questions and we can ask you all the contextual clarifying questions to really understand all the details.
Dear Karen,
thank you for this clear explanation.
The sentence that puzzles me is that ANCOVA is “very common in medical studies because the focus there is about the size of the effect of the treatment.”
Isn’t a change (as analysed per ANOVA) also the size of an effect of a treatment? I understand that both are effect sizes:
– In the RM ANOVA case expressed as a difference between the two groups’ change values,
– in the ANCOVA case expressed as a difference between the two groups’ post mean values?
Thanks for clarifying.
Claudia
Hi Karen,
Thanks for the nice post. Very informative.
Based on the above mentioned suggestions, we modeled our analyses as follows:
We have two groups of students: A) good TOEFL score, and (B) bad TOEFL score. At 1st round of intervention, we will teach each group one topics using a conventional teaching method. We will check their level of understanding before and after the intervention. Similarly, at 2nd round of intervention, we will use a new teaching method to teach a different topics, and evaluate them again.
We are interested to see: (1) whether our new method is better than the conventional method; and (2) whether the new method can overcome the barrier of language proficiency.
We will first perform repeated measure ANOVA separately for group A and B; focusing on time*method interaction. Second, multiple regression with categorical covariates for the TOEFL score and post test score to compare the slopes between group A and B.
We hope to get some usable findings.
I am currently completing my DNP in Healthcare Systems Leadership. My project is the implementation of computer barcode labels for single packages (peel pack) reusable medical equipment (RME).
My question is: For all users of RME will standardizing instrument identification using bar code labels versus the current method improve quality and costs?
The current method is handwritten instrument naming on indicator tape and no traceability.
I plan on collecting pre- and post- intervention data of RME (# of items and error rates) and use a statistical measure to prove pre to post. What’s the best to use?
I was think RR with CI: 95%
Any help appreciated- so overwhelmed.
Hi,
I am looking at state anxiety pre and post test, with an intervention. I have also measure trait anxiety as a covariate. I believe the correct analysis is an ANCOVA. Will my DV be the post test scores?
many thanks, Sammy JO.
Hi,
I’m bit confusing, I’d appreciate your advice about this study.
I’m having a drug group and a placebo group, and I’m measuring lab parameters on two time points; at baseline and after two month. It’s an RCT.
Data are normally distributed, so I used an independent t-test to compare between groups.
I’ve been advised to perform a 2-way ANOVA ! after searching, it was revealed that such analysis needed a 2 categorical independent variables.
I’ve thought they maybe considered a one-factor repeated measure ANOVA?!
I’ve done it with time (2 level) and a Group as a fixed variable, and there was no sig. on between groups test, although the results from both ANCOVA (with pre-tests values as covariates) and independent t-test were significant between groups.
I’ve read that ANCOVA is considered better in pre-post treatment analysis especially in RCTs.
Can I even perform a 2-way ANOVA ? and if No, what Can I argue with? I’m sending it a reviewer who asked me to use a 2-way ANOVA.
Sorry for disturbing.
Here are two article related to this question, I hope all of you can read and discuss here. Because it is a interesting question recently bother me.
Paper 1. Dimitrov, Dimiter M., and Phillip D. Rumrill Jr. “Pretest-posttest designs and measurement of change.” Work 20.2 (2003): 159-165.
This paper says: “….and one within-subjects (pretest-posttest) factor.
Unfortunately,this is not a healthy practice because previous
research [10,12] has demonstrated that the results
provided by repeated measures ANOVA for pretestposttest
data can be misleading. Specifically, the F test
for the treatment main effect (which is of primary interest)
is very conservative because the pretest scores are
not affected by the treatment. A very little known fact
is also that the F statistic for the interaction between the
treatment factor and the pretest-posttest factor is identical
to the F statistic for the treatment main effect with
a one-way ANOVA on gain scores [10]. Thus, when
using repeated measures ANOVA with pretest-posttest
data, the interaction F ratio, not the main effect F ratio,
should be used for testing the treatment main effect.
A better practice is to directly use one-way ANOVA
on gain scores or, even better, use ANCOVA with the
pretest scores as a covariate.”
And this website:
http://www.jerrydallal.com/lhsp/prepost.htm
Actually, if you use gain score (post – pre) as a outcome to conduct ANCOVA with pre score as a covariate, the p value of between-subject effect (treatment group) is the same as using only post score as a outcome.
post – pre = intercept + prescore + treatment + error
is equivalent with
post = intercept + prescore + treatment + error
for testing treatment effect
So if I use the gain score in the model, do I answer the question:
“whether the mean change in the outcome from pre to post differed in the two groups.(we need to add conditioned on the same mean pre score because prescore was used as covariate”. If yes, then can we use ANCOVA instead of mixed model to test interaction term? Which is better?
The two models are not equivalent unless the coefficient associated with prescore is 1 in the first Model.
this seems like an old post, but hopefully i can still get a response.
I have a very similar case as described here and am indeed planning to do a repeated measures ANOVA with one between subjects factor (treatment group) and one within-subjects factor (time). Can you give a hint how to set this up with linear regression? I have unequal sample sized in the groups, and I also might need to ad a third factor. Plus, I’m just curious.
Hi.
This is my topic: the effect of strategy based instruction on learner reading autonomy, reading performance, & reading motivation in tertiary education in Iran. Which kind of test is appropriate for that? I have 4 variables, strategy based instruction is independent and the rest of them are dependent. How show should I do test?
Any help will be appreciated.
Hi Karen! I am as crazily confused as the others here. But I hope you could look at my dilemma. I don’t know what statistical tool to use to test my hypotheses. I’m having one group with more than a pair of observations. I’m testing a group of 15 athletic swimmers and will subject them to blood gas analysis to get the difference of their arterial oxygenation. I will have a blood gas baseline measurement (before swim), blood gas results at 15 minute interval swim (post result), and blood gas results at 30 minute interval swim (another post result, but will be compared to the previous post result instead of comparing it to the baseline measurement (before swim). Please help me. I dont know what to do anymore. Thanks.
Hi Karen, I feel like Nancy since I am not sure which method to apply. I am applying one pre and one post tests (identical). I have control and experiment groups and working with three school. In summary, three school, one control and one experiment class in each school. One and one post test for each. I am measuring the influence of an educational intervention. Is this a repeated measure ANOVA?
I will apply Levene and t-test to check the homogeneity of variance and means at the beginning.
Hi Karen, I have a question here. Recently I conducted a school based health education on dengue. I used pre- and post-test design whereby an education material was provided during the inter-phase of the study. Focus is to evaluate the intervention (to see gain/changes in knowledge, attitude and practices). Usual analysis method for this kind of data in SPSS is Dependent-t-test, but it only applies if the data are paired. In my case, I did not paired the data (the respondents) during the pre and post-test. Also, I have unequal sample size. Do you have any suggestions on what kind of test I can use in SPSS to analyse this data? TQVM!
Hi Kareen,
I have a question.I am conducting a research study and I want to know wether the intervention leads in that clients participate more in group treatment or not? and then compare their session attendance with previous clients referred to the same center and clients referred to another center for the same problem( there are some problems that we can not have control groups). I also want to check some secondary outcomes in the group who received MI in three different time which is pre, post intervention, and post group treatment. I thought that it’s better to use repeated measure anova, after discussion with a statistician she told me to use ANCOVA for all of my research questions and outcomes. I am a bit confused. Would you please let me know which test should I use?
Hi Karen
I have carried out a mindfulness intervention with elements of qualitative a bit of a quant. I collected data pre and post intervention with 2 groups, but no control. What I did not do was pair the data (!).
I am looking to see if there are any differences in stress and levels of mindfulness pre and post intervention. Is there still an analysis that may work?
Hey Dee,
I am actually doing a very similar study with mindfulness. I would love to hear what you have done since you last asked your questions. Are you useing scales and comparing the answers on those scales from pre to post?
Best,
Francisco
I am looking at 20 companies which were applying a particular set of accounting standards in their financial reporting. After 5 years of application, they were required to change the standards and apply a different set of standards. I want to analyze the effect of the new standards on the companies’ financial performance and market value, which statistical test will be more appropriate?
Hi. I am studying a QoL questionaire before surgery and 1 month after surgery. I donot have any control group. How do i analyse results? Further if i postulate that after surgery the QoL improves by 20% than what should be my sample size?
please i want to use ancova to analyse data but i dont know how to go about it. I dont know independent, covariance and factor
Hi,
First, let me express my gratitude for the useful information provided here.
Then, I should say that I have a similar problem: I had a homogeneous sample (selected via a homogeneity test), then randomly divided them into two groups. After that, I administered a pretest (writing skills).Then, 20 sessions of treatment (in control group: I taught academic writing and in the treatment group: academic writing plus the treatment). Next, I gave them a post-test to calculate the (probable) significance of the treatment. Now, should I be using ANCOVA for the analysis or just t-test?
Please help 🙂
Thanks a lot.
Hi Karen,
I have 2 groups, 1 group that received an intervention (170 participants) and another that didn’t receive the intervention (30 participants). I am trying to assess whether the intervention has 1) an impact on 3 process variables and 2) 4 DVs.
There was no chance to randomly allocate the groups… they were 2 student cohorts measured at equivalent times in their training… one going into winter and 1 going into summer.
Problems (1) – controlling for baseline differences, (2) dealing with unequal cell sizes and (3) dealing with seasonal effects.
Given that I am looking at a total of 7 continuous variables…
I have a few questions:
1) should I do a repeated measures MANCOVA?
2) If I do repeated measures MANCOVA will this control for baseline differences?
3) I have been advised that I could do a matched subjects design…..draw a matched sample of 30 from the 170…. but others say that this reduces power.
Hi Karen,
I am completing a repeated measures ANCOVA, with one covariate and between subjects factor. I am a bit confused as to how to check the homogeneity of regression slopes assumption. Can you shed any light on this?
Any help will be greatly appreciated!
Alice Maier
The topic is “Effect of Pilates and Swiss ball training on balance, strength and sleep quality”. Herein are attached tables of the data collection. We intend to apply ANCOVA for the research question.
Questions we are searching answer for are:
Improvement in all three variables in the Pilates group,
Improvement in all three variables in the Swiss ball group,
And the final inference is to be drawn whether which group has shown better improvement on comparison for the three variables.
Please suggest which table (pre/pre, post/post or pre/post, pre/post) should be used for the statistical analysis using ANCOVA test.
Hi,
I am doing a simple pretest, education, posttest. What I want to know is if the education made a difference for the group. The group was made up of 8 with 12 questions. I tried the t-test and was told I had to have 30 participants. I explained my results and I am still told I need help explaining and the t-test was not the test to use. So, I have been reviewing the ANOVA is that what I would use. And how do I explain the difference?
Thanks, M
Hi, I really need some help with this. I am investigating whether there is any change on maternal strategies before and after an intervention.I am coding on materlan strategies in response to children’s errors. I have coded the variables pre-test and at post-test. I have divided the variables with the total number of errors at pre-test and at post test to adjust fr the scores and have a fair comparison. I have run a 2X2 repeated measures ANOVA but now I have second thoughts. Would it be wiser to run an ANCOVA and covary for the number of errors???
We have conducted an efficacy study of an intervention Program. 55 participants completed the Program and 37 took part in a control group. At baseline no diferences were found between the groups regarding the study variables or demographic variables. We used repeated measures ANOVA. We are now interested in understanding the role of some variables, for example experiential avoidance, in the decrease that occured in depressive symptoms from T1 ( baseline) to T2 (post treatment). What would you suggest?
i have 4 experimental group, each experimental group was measured as pre, mid and post score for all the groups. what type of statistics i want to use
Dear Karen,
Your answers are clear and good. Am a doctoral student and my objective of the study is the use of SMS for learning among undergraduate students
of Visual Communication and considers pre-post test design. My doubt is, is it necessary to have same questions to conduct the study. For a period of month the students will be receiving important terms, glossaries of communication.
Hi Karen,
I have almost the same design. I am trying to evaluate the effectiveness of intervention on specific performance outcomes in different conditions and I have control group. I have 2 conditions and both conditions have 2 levels( I ll use mixed anova). I am really confused about should I take pre-post test as a factor into analyses or should I get the differences between pre and post. what do u think?
Hi Karen, I am a big fan of your site. I have a related question. I have an experiment with 2 fixed factors and 1 random factor of site. The treatments were assigned randomly at each site and each combination has 4 plots. I am interested in the effects of the fixed factors (inl. interactions) on the post-treatment grass shoot counts.
Since I have a random factor and dealing with counts I have chosen the SPSS GENLINMIXED procedure with a Poisson distribution. However, the resulting model is over-dispersed. Before turning to the negative binomial, I would like to include a continuous covariate (grass shoot counts in each plot pre-treatment) to try to explain more of the variance. Is this a right move?
Also, the GENLINMIXED model seems to be taking the covariate as a fixed factor. How would I interpret the results in this case?
Many thanks,
Jo
Hi Karen,
Thank you for your post. it seems that sometimes it’s hard to know what is exactly the right analysis. In my case, an educational intervention was implemented and pre/post data survey was admistered to evaluate effectiveness on health outcomes. I would like to also know if the patients who indicated that they had arthristis expereinced reduced level of pain in pre/post intervention.
well your answer made me much more confused lol.These are my research questions:
1 What are the different effects of text-picture and audio-picture in facilitating EFL vocabulary immediate recall?
2 What are the different effects of text- picture and audio-picture in facilitaing EFL reading comprehesion? I would really appreciare tour help Karen.Thank you in davance
hi, im really confused about the kind of measueremnt i should use:ANCOVA, or repeated measure ANOVA.I would like to conduct a pretest-postest design. I would like to investigate the effects of multimedia on the vocabulary acquisition and reading comprehension. The two experimental groups will receive a pretest and then a postest for both dependent variables reading comprehension and vocabulary acquisition.
Hi Asma,
Based on what you’ve told me, it could be either. It depends on what you’re interested in testing: how much the multimedia changes vocabulary and reading comprehension OR whether the final vocabulary and reading comprehension is higher based on which multimedia exposure they got, controlling for where each person started. They’re different research questions. One is about change/growth/development. The other is about who is better off at the end.
Hi karen,
I have benefited a lot from your explanation.
I have a pre-post design, and it is a uncontrolled prospective study. We want to compare whether the seurm BDNF in patients after 8 weeks treatment is higher than those in patients before treatment . in this situation, can I use repeated measures ANOVA? and if I need to adjust for
potential confounder of factors such as duration of illness, severity before treatment ? if so, how can I adjust these confounders?
Thank you so much, and looking forword to your response!
Hi Shenhe,
Yes, you can use a repeated measures approach. I suspect, though, that you’ll be better off with a marginal model, in order to incorporate those covariates. RM anova can include covariates, but it’s limited. See this article on approaches to repeated measures.
Dear Karen,
Thanks for your answer, and it benefits a lot. However, I still have a question. In our study, there is only one antidepressant treatment patients group,and there is not a placebo treatment patients group.
my question is that, in this situation, how can I calculate the connection between the BDNF decrease and the course of treatment, especially when there were many confunding factors such as age, sex, severity before treatment?
In general, it seems that adjusting the covariates need two groups, and since our study lacked of placebo treatment group, I doubt whether I can still use repeated ANOVA inculde confunding factors as covariates to interpret the connection between BDNF decrease and the course of treatment.
Thanks very much! I am looking forward to your help and answer.
I am sorry, the right expression is connection between BDNF increase and the course of treatment.
Hi Karen,
do you offer a discount to students? I did a study and wanted to ask some questions.
could’t find a contact email hence posting here.
Thanks
Hi
I have a similar question. If I do an ANCOVA, is there anyway to compare the change in each group? When I do an ANCOVA, I get a baseline for both groups, how do I compare this baseline score with the each group mean? Or can I?
Thank you so much
Hi Anoop,
You can’t. If change from baseline is what will answer your research question, don’t use ANCOVA.
Stumbled upon this post trying to address a reviewer comment that I should do an ANCOVA instead of a RM ANOVA. Thank you so much for your commentary, as this helped me in formulating my response.
Hi Jeff,
Woo hoo! That’s why I do this. 🙂
I”m giving a pre post test questionnaire asking 10 questions with multiple answers but only one correct answer. Looking at knowledge deficit changes
in only one population. The comparator is the pre test score. A 1 hour educational module will be taken by each subject, then 6-8 weeks later they will take the same test again. Is ANCOVA the correct test to determine the difference? I really enjoy reading these online conversations. Thank you.
Hi Mimi,
It’s a possible approach, but there are many details that really affect the correct test. Start here:
https://www.theanalysisfactor.com/8-things-to-consider-in-choosing-statistical-analysis/
Hi Karen,
I conducted an intervention study where the groups were randomly assigned to experimental and control groups. The experimentals received treatment across two years and data were collected from experimentals and controls during 4 points in time. I want to evaluate the effectiveness of the intervention by comparing the changes across time between the two groups.
I initially thought I would need to do a repeated measures ANOVA but I read that the groups have to be the same. Does this mean that my control and experimental group are the same people? If I can’t use this analysis do you have another suggestion?
Thank you,
Debbie
Hi Debbie,
It sounds like you have a mixed model. One factor (time) is repeated measures but the other (group) is between subjects. So yes, you’ll need some version of a repeated measures, all of which can incorporate a between subjects factor.
Hi Karen, interesting post. Currently designing a study, confused whether I’m dealing with repeated measures analysis or not. Measuring X at 3 time points (different seasons) during the year – might be high in summer/low in winter, high in winter/low in summer, consistently high, consistently low, etc. Y (health outcome) measured at same 3 time points. Time between measures not equal between participants, since it’s the “season” that is of interest for X. Y may or may not change within the short time frame. Main research question is whether seasonal variation in X is associated with Y. ie. would someone with high-X in all seasons have healthier Y vs. someone with more variability in X, or someone with low-X in all 3 seasons? This almost seems to me a just a multiple linear/logistic regression analysis… but then I keep coming back and wondering about repeated measures or ANCOVA…? Thanks in advance for any insight.
Hi Katya,
If the three measurements are on the same individuals, then yes, it’s repeated measures. It doesn’t matter if time is equally spaced or if it really represents another concept, like season. You will definitely need a mixed model, not a repeated measures ANOVA. Mixed models can accommodate many more designs.
Hello,
I have a pre-test /post-test study design with two treatment and two control groups, all tested for attitude scores at 2 points of measurement. There are 2 additional demographic covariates that need to be controlled for. The study is not randomized and there were slight (albeit insignificant) differences in baseline scores for the 2 groups. Would an ANCOVA with attitude at t2 as DV and attitude at t1 as one of the 3 covariates be the right way to go here? I am interested in the effect of the intervention (i.e. whether the the intervention increases the attitude scores more than the expected increase with time).
Thanks in advance!
Emily
Hi Emily,
I would have said yes until I got to your last sentence. If the point is to test the INCREASE then the ANCOVA approach won’t do it. You’ll have to use a repeated measure approach.
I want to analyze effect on PSQI Score of 15 controls and 15 cases pre and post intervention (MSBR).What tests can I use to analyze ?
Hi SS,
I’d need to know how PSQI scores are measured. Are they categorical? Continuous, likert, counts?
Hi,
I have a stats related question as well…
I am testing for period effect in a crossover study that has multiple measure outcomes (i.e. test 1, test 2, test 3–but all these tests are within a similar theme, z-scored, and correlated (but not too highly)). I have run a one-way (or within group) repeated measure MANOVA but cannot quite resolve how to do the post-hoc paired t-tests…help pretty please!
Post hoc tests are tricky in a manova as you can really only do it on the univariate anovas. That may or may not come out the same as the multivariate tests.
great post – thanks a lot!
Thanks Karen.
Another question: how would I analyse my pre-test/ post-test data using the ANCOVA approach but with a hypothesis that the results are mediated by another variable.
For example, an online intervention program is effective in reducing the stigma toward mental health. However, I believe that this result is mediated by knowledge about depression.
Do I perform the same analysis but also add knowledge about depression as another covariate?
Hi Karen,
Can you please outline the assumptions I need to check for when using the ANCOVA approach you mention here?
Thanks
JCL
Hi JCL,
Yes, they’re listed here: https://www.theanalysisfactor.com/assumptions-of-linear-models/
Hi Karen, I’ve got data from a before-and-after study design with an intervention and control group. I’m want to see how my “intervention” (taking my obesity course) impacts weight-related attitudes compared to a “control” course. Attitudinal measures were done at the beginning (time 1) and end of the semester (time 2). However, there are signficant group differences in both attitudinal measures at baseline. What’s the best way to deal with this?
Option 1: Repeated measures ANOVA set up normally
Option 2: ANCOVA where anti-fat attitudes (time 2) is the DV and anti-fat attitudes (time 1) is a covariate
Option 3: Repeated measures ANOVA where the DIFFERENCE in anti-fat attitudes (time 2 – time 1) are the DV and the time 1 scores are a covariate
Or something completely different??
Hi Lori,
I think option 1 is best. Both 2 and 3 will allow you to look at what is happening at time 2, controlling for what is going on at time 1, but neither will allow you to show that it’s happening, and you do need to show that.
I’ve gain a lot with few minutes interacting with this content. I’ll very much appreciate if you’ll reply whenever I request for guide.
Andy
Best wishes
Hi Karen,
I have a situation as you described with Nancy’s example and I was wondering whether repeated measures ANOVA is appropriate to use.
I have a cohort with cardiac function assessment at two time points (t=0 and t=6 months) and blood pressure measurement at t=6.
The question that I like to answer is whether hypertension ( which is developped slowly over time but was diagnosed at t=6) is related to cardiac function depression at t=6.
For this purpose I would like to use RM ANOVA and enter cardiac function assessment as dependent with 2 levels and hypertension as “Between-subjects Factors” in SPSS.
Is this correct and to I have to model differently as a covariate?
Thank you very much!
Best regards,
Ulx
Hi Ulx,
If cardiac function *depression* is defined as how it differs from baseline, then indeed, you need a repeated measures, not ANCOVA.
Hi Karen,
Have a related question on ANCOVA
I had an experiment of 15 snakes exposed to 2 prey types. The snakes ate prey type one and then prey type two few minutes later. My covariate here is the relative prey mass (RPM; prey mass/snake mass). My measures are the time it took for the snake to capture prey one vs prey two. Is that feasible to perform ANCOVA considering that the same snake have eaten both prey types (lack of independence?). I will appreciate any hint. Does ancova works fine with unequal sample size?
Thanks,
Mil
Hi Mil,
You are correct–with two observations per snake, you need to account for non-independence. I would start here:
https://www.theanalysisfactor.com/repeated-measures-approaches/
https://www.theanalysisfactor.com/longitudinal-repeated-measures/
You can think of the two prey as repeated measures per snake.
I have a pre test – post test within subjects design. 30 questions on each test that fall into 3 different constructs. Did dependent t-tests on combined constructs and each question, plus Cohen’s d effect sizes. What other higher level analysis should I perform?
HI DK, that entirely depends on your research question. What do you want to know from these data?
Hi,
First, thanks a lot for the explanation. I have the same exactly the research design (pre- & post-test with experimental & control group) and faced the same problem. I received mixed advice and confused with the appropriate method -ANOVA or ANCOVA.
If my research objective is to evaluate the effect of an intervention program, which statistical analysis would you suggest? Both group has the same sample size, were randomly assigned and demographic characteristic for both groups are homogeneous. Please kindly help as I am very confused with too many conflicting advises. Thank you in advance.
Hi Emma,
As you can probably tell from the article, it’s hard to know what is exactly the right analysis. I’d have to ask you a lot of questions about your exact, specific research questions and design in order to really advise.
Karen
Hi I wonder if you could help me,
I am currently doing my dissertation on how mindfulness can be an effective way to reduce stress and have done a 2 way mixed anova for the analysis and written up that section. However I also have data from a follow up questionnaire asking participants how many times a week they did the training. So I need to find out if there is a relationship between time spent doing the training (3 or less days or more than 3 days) and amount stress score decreased from pre to post test.
Its not a straight forward correlation because I dont have numbers for the decrease I just have 2 scores pre and post. I have no idea what test to do on SPSS.