Crossed random effects models are a little trickier than most mixed models, but they are quite common in many fields. Recognizing when you have one and knowing how to analyze the data when you do are important statistical skills.
The Nested Multilevel Design
The most straightforward use of Mixed Models is when observations are clustered or nested in some higher group.
It’s also so common that it often has its own name: multilevel model.
Examples include studies where patients share the same doctor, plants grow in the same field, or participants respond to multiple experimental conditions.
The observations at Level 1 (patient, plant, response) are clustered at Level 2 (doctor, field, or participant). This makes the responses from the same cluster correlated.
In these models, the Level 2 cluster is often not of interest. It’s what we call a “blocking factor.” Even so, we need to control for its effects.
If the researcher would like to generalize the results to all doctors, fields, or participants, these clustering variables are random factors.
The observations of the dependent variable are always measured at Level 1 (the patient, plant, or time point). Predictor variables (fixed effects) can be measured at either Level 1 or Level 2. For example, number of years of experience of a doctor would be at Level 2, but patient age would be measured at Level 1.
We assume the observations within cluster are are correlated, but the observations between clusters are independent.
A third level is possible as well. This would happen if each doctor sees all their patients at one of four hospitals or each field has only one of 5 species.
The Crossed Multilevel Design
In one kind of 2-level model, there is not one random factor at Level 2, but two crossed factors.
Each observation at Level 1 is nested in the combination of these two random factors. These models need to be specified correctly to capture the effects of both random factors at Level 2.
Here are the same examples with crossed random effects:
Example 1:
Every patient (Level 1) sees their Doctor (Random Effect at Level 2) at one of four Hospitals (Random Effect at Level 2) for a study comparing a new drug treatment for diabetes to an old one.
Each doctor sees patients at each of the hospitals. Patient responses vary across doctors and hospitals.
Because each Patient sees a single doctor at a single hospital, patients are nested in the combination of Doctor and Hospital.
The response is measured at Level 1–the patient. Predictors can occur at Level 1 (age, diet) or either Level 2 factor (years of practice by doctor, size of hospital).
Example 2:
An agricultural study is studying plants in 6 fields.
While there are many species of plants in each field, the researcher randomly chooses 5 species to study.
Each individual plant (Level 1) lies within one combination of species and field. But since every species is in every field, Species and Field are crossed at Level 2.
The response is measured at Level 1–the plant. Predictors can occur at Level 1 (height of plant) or either Level 2 factor (fertilizers applied to the field, whether the species is native or introduced).
Example 3:
In a psychological experiment, subjects rate statements that describe behaviors done by a fictional person, Bob.
On each trial, subjects rate whether or not they find Bob friendly. The response time of the rating is recorded.
Each subject sees the same 10 friendly and 10 unfriendly behaviors. The behaviors are not in themselves of interest to the experimenter,but are representative of all friendly and unfriendly behaviors that Bob could perform.
Because responses to the same behavior tend to be similar, it is necessary to control for their effects. Each trial of the experiment (Level 1) is nested both within Subject and Behavior, which are both random effects at Level 2.
Subject and Behavior are crossed at Level 2 since every Subject rates every Behavior. The response is measured at Level 1–the trial. Predictors can occur at Level 1 (a distractor occurs on some trials) or either Level 2 factor (Behavior is friendly or not, Subject is put into positive, neutral, or negative mood).
————————————————————————————————–
Luckily, standard mixed modeling procedures such as SAS Proc Mixed, SPSS Mixed, Stat’s xtmixed, or R’s lmer can all easily run a crossed random effects model. (R’s lme can’t do it).
Use care, however, because like most mixed models, specifying a crossed random effects model correctly can be tricky.
Hi,
Thanks for this interesting thread. In example 2, if we had a case in which not every species was in every field, rather it was be partially nested and partially crossed, what would be the case? would you still consider this as crossed at level 2?
That’s a great question. There are definitely situations where neither crossed or nested is entirely true. You can call it partially crossed. Ultimately, you can try to fit an interaction and it may or may not run. Just try it.
Hi Karen,
I am doing an experiment where I have four groups of individuals, which are exposed to different temperatures:
Group 1 (12 individuals): week 1 to 4: ambient (control group)
Group 2 (12 individuals): week 1: ambient, week 2-4: +2C
Group 3 (12 individuals): week 1: ambient, week 2: +2C, week 3-4: +4C
Group 4 (12 individuals): week 1: ambient temperature week 2: +2C week 3: +4C, week 4: +6C.
I want to look at the effect of temperature on various continuous behavioural responses at the level of the four different temperatures not taking into account groups, but also at the group/individual level, i.e. taking into consideration the repeated measurements and the different temperatures the individuals are exposed to with each repeated measure.
Is a multi-level model with crossed random effects suitable for this question?
Thanks,
Beth
Hi Beth,
I really can’t give advice without asking you a lot of questions. We do have consulting services and a membership if you need advice and want to talk with someone.
Dear Karen,
I have a question related to the third example, which (I think) is similar to the experiment I did. Unfortunately, I do not know how to specify the model because I cannot find any literature related to this kind of multilevel application, as in most applications in the literature, individuals are nested in classes etc.
I tried to specify the model in MPLUS but their support told me ‘you have no multilevel situation’.
Now I found your example, which seems quite similar, so I still have hope that I am not totally wrong.
In my experiment, each participant was exposed to the same 25 stimuli that reflect five different categories of theoretical interest (5 stimuli representing each category). Hypotheses are concerned with differences between the five categories. To control for variances caused by differences in the stimuli representing each category, I aimed to use a multilevel approach. Dependent variables are a continuous response-time measure and a binary coded answer.
Is it correct that in my case trials are nested in the five categories as well as the individuals? Or is it that trials are nested in the individuals as well as in the 25 stimuli that are nested in the five categories?
Maybe you are aware of similar applications in literature and/or text books?
Thank you very much for your help!
Daniel
Hi Daniel,
Yes, these are tricky. I don’t know for sure whether MPlus can do them as your random factors are not nested into different “levels.” They’re crossed. Trials are nested in the individual*category interaction. Individuals are crossed with both stimulus and category. Stimuli are nested in category.
If you think of this as a mixed model instead of multilevel, it will be easier. Think of each variable as fixed or random.
How can I download this
Hi Karen,
Do you have example code for your first example in SAS that you can share. How would I code for controlling for both patient and physician characteristics on one model?
Thanks for the help!
Is this the same as a cross-level interaction for a multilevel model?
Hi Patricia,
It’s not. A cross-level interaction in a multilevel model is an interaction among fixed effects, one of which is measured at level 1 and one of which is at level 2. The fact that you have level 1 and 2 indicates the random effects are nested. For example: students nested within teachers because each student has only one teacher.
Crossed random effects means that your random factors themselves are crossed, not nested. So not each student having one teacher. Each student having every teacher.
Hi Karen!
First thank you, because this is the first article that gives me a clue, that what I have is called as “crossed design” in some way.
But it seems, that I have a doubled crossed design:
I got participants that reacted to 6 different alarms in 2 different alarm conditions. Every participant did the 6 alarms in both conditions (the order of condition was randomized, but it does not have an effect at the particular dependent variable).
My problem is, that one of my dependent variables is nominal: they should identify the alarm and this variable has the values “identified correctly” and “identified wrong”.
So with all other variables (on ratio scale) I just take the mean over the 6 alarms and use a repeated-measure ANOVA (hopefully this is correct, as the variance between alarms is small and the number is equal). But with this dichotomous variable I cannot just take the mean, am I right? So that I have to look at the alarms as another level in multilevel modeling?
Do you have a clue what to do or am I just thinking wrong?
Thanks a lot in advance!
Hi Eva,
Yes, you’ve got a mixed model. I would also not average 6 responses on the numerical outcomes. When you do that, you’re ignoring variability.
Mixed designs can get complicated fast.
Hi Karen!
Thanks, as always your explanation are very paedagogic!
A question: can you introduce a fixed effect, an independent variable, at the intersection of your two random effects at level 2? For instance, in your first example, if Dr A is happier at hospital 1 than at hospital 2; or if each doctor’s office differs by hospital? Or would you have to introduce a new level 2, say, doctors’ office, or a “doctor by hospital” level, which then is cross-classified by a doctors and hospitals (now being level 3)?
Thanks again!
Best wishes/Klara
Hi Klara,
Yes, you can. The doctor*hospital interaction exists as a “unit of measurement.”
Hi Karen,
Thanks so much for this very helpful page. I have a question for you if you have the time: Is there any minimum requirement for the number of observations within each cross-classification (each cell, if you’re thinking of it in terms of rows-by-columns)? Without getting into the details, I have a study in which “Raters” evaluated stimuli collected from “Donors” (and nearly every Rater evaluated stimuli from each of the Donors). Thus, stimulus ratings are cross-classifed into Rater-by-Donor cells. I plan to examine how these ratings vary as a function of Rater characteristics, Donor characteristics, and the compatibility of each Rater-Donor pair (specifically, proportion of alleles that the Rater and Donor have in common at one particular locus). So, a cross-classified model seems like the way to go. The only potential problem is that each cell will contain just one single rating… or is that not a problem? Thanks!
Hi Kelly,
One is sufficient. It means there will be some interactions you can’t test, but this is a very common design.
Hello,
My data contains product prices, observed for several stores, which belong to a particular chain.
I’m thinking of doing a variance decomposition. Thing is I’m not sure how my model should look like. I should have a Random Effect for chain, and for store. But how about products?
Thank you in advance for any suggestion.
Hi Joaquin,
It could be that chain, store, and product are all random factors or just some. I would have to know much more detail about your study.
How about this example–-does it apply? I study a personality trait that can be inferred from videos. The trait differs by sexual orientation (gay/straight). I have 60 targets (30 gay 30 straight) and 60 raters (30 gay and 30 straight; we are interested in whether rater’s sexual orientation increases accuracy). We’d like to generalize both beyond our targets and beyond our raters, so in that sense we’d like to consider them random effects. Every rater sees every target, so this seems completely crossed. I’m having trouble mapping my situation onto any of the above though.
Hi Mike,
It’s honestly hard for me to tell without really understanding every detail of the design, but it certainly sounds like it applies. Your raters are technically your “subjects” and the targets are like the “trials.”
Hi Karen —
Thank you so much for your site — it is an amazing resource, and has helped clarify my thinking about these issues a lot. I am also writing to ask about your example 3 above. In that scenario, I read distractor as being nested within both participant and behavior — as some participants may be more sensitive to distractors than others and some behaviors may be more or less prone to disruption by distractors. When modeling this in SPSS, is the idea that distractor would be added as a fixed effect, and then nested within the two random effect commands?
Many thanks,
Emily
I’m not sure what you mean by conditions or replicates. We do not have treatment groups. All images were viewed by all subjects in the study and the response is measured at the image level. I want to examine the effect of different demographic factors on the response. I also want to include whether the image was viewed first in the model. Do I have to do anything special when specifying the model if some covariates measured at the image level and others at the subject level?
I have a study that I think has crossed random effects. Participants rated 7 images on the same outcome variable. Each participant rated the same 7 images (in random order). Dependent variables are demographics and some other variables measured at the image level. I’m wondering how do you specify this with SAS? Do I do two random statements?
Also, my original thought with how to analyse this data was to use a random intercept model. Then I could include image as a covariate to see the differences in ratings for each image. Would that be valid?
Hi Sarah,
It could be. It depends on whether those 7 images represent 7 conditions or if some are replicates. If you want to compare the actual images, then don’t make it random.
But yes, two random statements in SAS. You can have a random intercept for subject and a random intercept for image.
Thanks so much for discussing this topic, I am having a hard time understanding the differences between nesting and cross-classification and this post definitely points me in the right direction!
I am trying to understand the difference between the two with respect to a standard repeated measures design (for example, measuring participants’ emotional responses to a stressor for 1 hour at 10 minute intervals). I am used to thinking of this type of design as time nested within participant, but now I am wondering if time and participant are actually crossed because each participant is measured at every time (and every time is assessed for every participant). Am I understanding this correctly?
1) IF NO: can please you elaborate on the difference between nested and cross-classified designs? I must have misunderstood something along the way.
2) IF YES: I am unclear why ‘trial’ in your third example is nested within behavior and participant. Doesn’t every participant receive every trial just like every participant receives every behavior? I am not clear as to why this design is not completely cross-classified.
Also, do you know of any resources for SPSS nested versus cross-classified syntax?
Thank you for your help!
Hi Lisa,
Great questions.
In answer to your first question, you are correct–in a standard repeated measures study where every participant is measured at every time point, participant and time point are crossed.
2) Trial, in this example, is literally the individual trial, not the behavior. If 15 subjects each see 10 behaviors, that’s 150 trials. Each of those trials occurs for only one subject and one behavior. So each subject has 10 trials. So subject and behavior are crossed, but trail is nested within both subject and trial.
Make sense?
Karen
Thanks for your quick and helpful response! I am still slightly confused. I hate to get hung up on a specific example, but this example is very relevant to my own research, so figuring out the source of my confusion will go a long way!
I think I am unclear about why we need to define ‘trial’ and how we would use ‘trial’ in an analysis. Specifically, it sounds like trial is just a combination of subject and behavior such that each trial is a unique combination of one person and one behavior. If that is the case, why do we need to identify trial in addition to subject and behavior – doesn’t specifying subject and behavior give us this information already?
I am not sure if the following metaphor will make sense, but it feels similar to specifying a particular ‘cell’ in a factorial experiment. If there are two independent variables with two levels each, there are four total ‘cells’. These cells are a combination of the other two factors and they are unique. I guess I am confused because I don’t understand why we need to define the specific ‘cell’ because (at least in an anova framework) those cells are created automatically by defining two crossed factors.
Is there any chance you have SPSS syntax written for example 3? Perhaps seeing the actual syntax will help me figure things out. Thanks again for your help!