Five Advantages of Running Repeated Measures ANOVA as a Mixed Model

by Karen

There are two ways to run a repeated measures analysis.The traditional way is to treat it as a multivariate test–each response is considered a separate variable.The other way is to it as a mixed model.While the multivariate approach is easy to run and quite intuitive, there are a number of advantages to running a repeated measures analysis as a mixed model.

First I will explain the difference between the approaches, then briefly describe some of the advantages of using the mixed models approach.

Let’s use as an example a data set of students, who are measured at four time points across a school year.The children are given reading tests at the beginning of first grade and at three other time points evenly spaced across the school year. So each child has four observations for reading tests. Let’s assume the children are assigned to different experimental groups, and other covariates are measured.

In the multivariate approach, each child would have a single row of data in the data spreadsheet and four columns for the four reading scores. This is called the wide data form and the unit of observation is considered a child.Covariates that do not change across time, such as sex or age at time 1, would each appear in a column.

In the mixed model approach, each child would have four rows of data.One column would contain the time of measurement and another the reading score. This is called the long format, and the unit of observation is considered one time point per child.Covariates that don’t change would have repeated values across the four rows of data. A time-varying covariate would change values across the four rows, but only one column is needed for each one.

Some advantages of using a Mixed Models Approach:

  1. Missing Data: The default approach to missing data in nearly all statistical packages is Listwise Deletion, which drops any observation with any missing data on any variable involved in the analysis. If the percentage missing is small and the missing data are a random sample of the data set, this is a reasonable approach.In the multivariate approach, if a child is missing one time point, they will be dropped from the entire analysis.In the mixed approach, only that time point will be dropped. The remaining data will be retained.
  2. Post hoc tests: Because of the way the Sums of Squares are calculated in the multivariate approach, post-hoc tests are not available for repeated measures factors. They are available, however, using the mixed approach.
  3. Flexibility in treating time as continuous: Depending on the design of the study, rather than consider time as four categories, it can be more accurate to treat time as a continuous variable. This allows you to model a regression line for time, rather than estimate four means.(You need at least 3 time points to do this, but more are better).This is not possible in the multivariate approach, but simple in the mixed approach.
  4. A single dependent variable can be used in other analyses: For example, in a study I’m currently working on, a two-factor (2×4) repeated measures design was used to study whether the impact of these two factors on an outcome was mediated by a third variable. Each subject has eight values of the mediator (one for each of the conditions) and eight values on the final outcome. The mediator is both an outcome and a predictor variable in two different models in this path analysis. Therefore, it was necessary to have a single outcome variable, not eight, in order to have a single path coefficient between the mediator and the outcome.
  5. Easier to build into larger mixed models: If our school design happened to also cluster children within teachers, we would need to include teacher as another level in the mixed model. Changing a two to a three level model is simple to do (in practice, if not conceptually) if the model is already set up as a mixed model.

And a bonus reason may be the most important one of all.You, the data analyst, becomes familiar with the terminology, concepts, and programming involved with mixed models in a simple repeated measures design. Then, when you encounter something more complicated (which you most likely will), your learning curve will be a single step, not a giant leap.

Bookmark and Share
If you want to learn more about mixed models, check out the recording of my Random Intercept and Random Slope Models webinar.  These two models are the basic building blocks of all mixed models.  Get it all here.  It’s free.

rm-500Learn more about repeated measures analysis using mixed models in our most popular workshop: Analyzing Repeated Measures Data: GLM and Mixed Models Approaches.

Send to Kindle

{ 8 comments… read them below or add one }

Carsten Dahl Mørch November 26, 2009 at 5:38 pm

Thanks for the advice to shift to mixed models. I think it works like a charm. I am wondering though. Say we split the students into two groups after the first measurement we give them an intensive reading course during which measurement two may be taken. Now we know that students are forgetful so we expect the students that had the intensive course to fall back an perform similar as the control students in test 3 or 4. Now can we calculate the ‘post hoc’ on the interaction between time and student group?

Best Carsten

Reply

Karen November 30, 2009 at 10:09 am

Hi Carsten,

Theoretically, you can. You might be limited by your software, though. I know that SAS proc mixed allows for a variety of post-hoc adjustments in the interactions in the lsmeans statement. I believe SPSS has fewer options. If you’re using another program, I would suggest checking the manual to see what your options are.

Reply

Luping August 17, 2010 at 5:09 am

Hi Carsten,

Currently I am facing a problem with the mediation test in a two within-factor repeated measure model (with time-varying covariates). Could you please share more about how you deal with this problem?
Great thanks!^_^

Reply

Karen August 23, 2010 at 10:39 pm

Hi Luping,

I’m not sure exactly what the problem is, but if it’s just the general approach…

Just like in any mediation analysis, you’re going to run three models. The first tests the direct effect of the Independent Variables (your within-factors, controlling for covariates) you’re interested in on the Dependent Variable (DV). You will have to run it as a mixed model, rather than a multivariate repeated measures, to make sure that you have only one DV.

The second tests the effect of the same predictors on the Mediator.

The third is the same as the first model, but add in the Mediator as a covariate. See if the relevant effects in equation 1 go away. That’s mediation.

Is that helpful?
Karen

Reply

Gabriel December 19, 2012 at 4:43 pm

I have the same problem, I’m not sure if it’s possible to calculate mediation in within-subject designs (repeated measures). e.g.

Judd, Kenny, McClelland, 2001. pp 116.

[…] These analytic procedures (mediation), however, apply only to the situation in which the treatment variable varies between experimental units, with some units or participants receiving one treatment and others receiving another. However, treatment comparison often involves within-subject comparisons, examining every experimental unit or participant in each treatment condition […] A design in which the independent variable varies within participants rather than between them is particularly useful when the effects of the independent variable are relatively transitory and the independent variable is easily varied. Such a design can often result in a dramatic increase in statistical power relative to between-subjects designs, because variance due to stable individual differences contributes to the error variance in testing between-subjects effects but not to the error variance in within-subject analyses. As a result, within-subject designs often involve smaller samples than those typically used in between-subjects designs.

http://dionysus.psych.wisc.edu/lit/articles/juddc2001a.pdf

But !

Preacher & Hayes, 2007, pp 20-21.

[…]The causal steps approach is by far the most commonly used method for assessing mediation. The criteria have recently been extended for use in within-subject designs as well (Judd, Kenny, & McClelland, 2001). But despite its simplicity and intuitive appeal, the causal steps strategy suffers from serious limitations relative to other methods we discuss soon. First, it is possible to observe seemingly paradoxical effects using this approach. For example, a significant c and nonsignificant c′ may differ by a trivial amount in absolute terms, yet the causal steps criteria would indicate that mediation is occurring (a possible Type I error; Holmbeck, 2002) […].

http://www.sagepub.com/upm-data/23657_Chapter2.pdf

In my visual search experiment, the design is a 2 X 4 repeated measure ANOVA with 2 repeated factors of search type (Condition L / Condition X) and set-size (2, 4, 8, 12), with participants as random factor. Dependent variables: response time and an index of confidence.
All suggestions are welcome !
Thanks !
Regards
Gabriel

Reply

Karen January 4, 2013 at 1:21 pm

It’s a good point, Gabriel. The procedures for good mediation are definitely changing rapidly.

I don’t think there is a really great solution right now, unfortunately. The old Baron and Kenny approach, which I assume is what “causal steps” means, doesn’t cut it in many fields any more. But there aren’t good tests for the direct effects either in within subjects effects, like Preacher and Hayes’ bootstrapping.

This is a subject I intend to spend some time catching up on, but at this point, I don’t know of any great answers.

Karen

Reply

Kristin July 10, 2011 at 12:17 pm

Do you have a webinar/handouts for doing this in SAS?
Thanks!

Reply

Karen July 15, 2011 at 9:11 am

Hi Kristin,

I don’t, but I should. I have a webinar with handouts for doing this in SPSS.

The LOGIC of what the random and repeated statements do in SAS proc mixed and SPSS mixed is the same. Here is a nice handout with the syntax for both. http://psych.unl.edu/psycrs/944/SAS_SPSS_Mixed.pdf

Reply

Leave a Comment

{ 3 trackbacks }

Previous post:

Next post: