• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • Home
  • Our Programs
    • Membership
    • Online Workshops
    • Free Webinars
    • Consulting Services
  • About
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Collaborate with Us
  • Statistical Resources
  • Contact
  • Blog
  • Login

Multilevel Models with Crossed Random Effects

by Karen Grace-Martin 27 Comments

Crossed random effects models are a little trickier than most mixed models, but they are quite common in many fields. Recognizing when you have one and knowing how to analyze the data when you do are important statistical skills.

The Nested Multilevel Design

The most straightforward use of Mixed Models is when observations are clustered or nested in some higher group.

It’s also so common that it often has its own name: multilevel model.

Examples include studies where patients share the same doctor, plants grow in the same field, or participants respond to multiple experimental conditions.

The observations at Level 1 (patient, plant, response) are clustered at Level 2 (doctor, field, or participant). This makes the responses from the same cluster correlated.

In these models, the Level 2 cluster is often not of interest. It’s what we call a “blocking factor.” Even so, we need to control for its effects.

If the researcher would like to generalize the results to all doctors, fields, or participants, these clustering variables are random factors.

The observations of the dependent variable are always measured at Level 1 (the patient, plant, or time point). Predictor variables (fixed effects) can be measured at either Level 1 or Level 2. For example, number of years of experience of a doctor would be at Level 2, but patient age would be measured at Level 1.

We assume the observations within cluster are are correlated, but the observations between clusters are independent.

A third level is possible as well. This would happen if each doctor sees all their patients at one of four hospitals or each field has only one of 5 species.

The Crossed Multilevel Design

In one kind of 2-level model, there is not one random factor at Level 2, but two crossed factors.

Each observation at Level 1 is nested in the combination of these two random factors. These models need to be specified correctly to capture the effects of both random factors at Level 2.

Here are the same examples with crossed random effects:

Example 1:

Every patient (Level 1) sees their Doctor (Random Effect at Level 2) at one of four Hospitals (Random Effect at Level 2) for a study comparing a new drug treatment for diabetes to an old one.

Each doctor sees patients at each of the hospitals. Patient responses vary across doctors and hospitals.

Because each Patient sees a single doctor at a single hospital, patients are nested in the combination of Doctor and Hospital.

The response is measured at Level 1–the patient. Predictors can occur at Level 1 (age, diet) or either Level 2 factor (years of practice by doctor, size of hospital).

Example 2:

An agricultural study is studying plants in 6 fields.

While there are many species of plants in each field, the researcher randomly chooses 5 species to study.

Each individual plant (Level 1) lies within one combination of species and field. But since every species is in every field, Species and Field are crossed at Level 2.

The response is measured at Level 1–the plant.  Predictors can occur at Level 1 (height of plant) or either Level 2 factor (fertilizers applied to the field, whether the species is native or introduced).

Example 3:

In a psychological experiment, subjects rate statements that describe behaviors done by a fictional person, Bob.

On each trial, subjects rate whether or not they find Bob friendly. The response time of the rating is recorded.

Each subject sees the same 10 friendly and 10 unfriendly behaviors. The behaviors are not in themselves of interest to the experimenter,but are representative of all friendly and unfriendly behaviors that Bob could perform.

Because responses to the same behavior tend to be similar, it is necessary to control for their effects. Each trial of the experiment (Level 1) is nested both within Subject and Behavior, which are both random effects at Level 2.

Subject and Behavior are crossed at Level 2 since every Subject rates every Behavior. The response is measured at Level 1–the trial. Predictors can occur at Level 1 (a distractor occurs on some trials) or either Level 2 factor (Behavior is friendly or not, Subject is put into positive, neutral, or negative mood).

————————————————————————————————–

Luckily, standard mixed modeling procedures such as SAS Proc Mixed, SPSS Mixed, Stat’s xtmixed, or R’s lmer can all easily run a crossed random effects model. (R’s lme can’t do it).

Use care, however, because like most mixed models, specifying a crossed random effects model correctly can be tricky.

Fixed and Random Factors in Mixed Models
One of the hardest parts of mixed models is understanding which factors to make fixed and which to make random. Learn the important criteria to help you decide.

Tagged With: Correlated response, mixed model, multilevel model, SAS Proc Mixed, SPSS Mixed

Related Posts

  • Specifying Fixed and Random Factors in Mixed Models
  • The Difference Between Random Factors and Random Effects
  • Multilevel, Hierarchical, and Mixed Models–Questions about Terminology
  • Examples for Writing up Results of Mixed Models

Reader Interactions

Comments

  1. Tahereh Dehdarirad says

    January 17, 2022 at 6:42 am

    Hi,

    Thanks for this interesting thread. In example 2, if we had a case in which not every species was in every field, rather it was be partially nested and partially crossed, what would be the case? would you still consider this as crossed at level 2?

    Reply
    • Karen Grace-Martin says

      January 28, 2022 at 9:42 am

      That’s a great question. There are definitely situations where neither crossed or nested is entirely true. You can call it partially crossed. Ultimately, you can try to fit an interaction and it may or may not run. Just try it.

      Reply
  2. Beth says

    January 16, 2020 at 11:49 pm

    Hi Karen,

    I am doing an experiment where I have four groups of individuals, which are exposed to different temperatures:
    Group 1 (12 individuals): week 1 to 4: ambient (control group)
    Group 2 (12 individuals): week 1: ambient, week 2-4: +2C
    Group 3 (12 individuals): week 1: ambient, week 2: +2C, week 3-4: +4C
    Group 4 (12 individuals): week 1: ambient temperature week 2: +2C week 3: +4C, week 4: +6C.

    I want to look at the effect of temperature on various continuous behavioural responses at the level of the four different temperatures not taking into account groups, but also at the group/individual level, i.e. taking into consideration the repeated measurements and the different temperatures the individuals are exposed to with each repeated measure.

    Is a multi-level model with crossed random effects suitable for this question?

    Thanks,

    Beth

    Reply
    • Karen Grace-Martin says

      January 24, 2020 at 10:04 am

      Hi Beth,

      I really can’t give advice without asking you a lot of questions. We do have consulting services and a membership if you need advice and want to talk with someone.

      Reply
  3. Daniel says

    November 22, 2019 at 8:01 am

    Dear Karen,

    I have a question related to the third example, which (I think) is similar to the experiment I did. Unfortunately, I do not know how to specify the model because I cannot find any literature related to this kind of multilevel application, as in most applications in the literature, individuals are nested in classes etc.

    I tried to specify the model in MPLUS but their support told me ‘you have no multilevel situation’.

    Now I found your example, which seems quite similar, so I still have hope that I am not totally wrong.
    In my experiment, each participant was exposed to the same 25 stimuli that reflect five different categories of theoretical interest (5 stimuli representing each category). Hypotheses are concerned with differences between the five categories. To control for variances caused by differences in the stimuli representing each category, I aimed to use a multilevel approach. Dependent variables are a continuous response-time measure and a binary coded answer.

    Is it correct that in my case trials are nested in the five categories as well as the individuals? Or is it that trials are nested in the individuals as well as in the 25 stimuli that are nested in the five categories?

    Maybe you are aware of similar applications in literature and/or text books?

    Thank you very much for your help!
    Daniel

    Reply
    • Karen Grace-Martin says

      December 2, 2019 at 11:02 am

      Hi Daniel,

      Yes, these are tricky. I don’t know for sure whether MPlus can do them as your random factors are not nested into different “levels.” They’re crossed. Trials are nested in the individual*category interaction. Individuals are crossed with both stimulus and category. Stimuli are nested in category.

      If you think of this as a mixed model instead of multilevel, it will be easier. Think of each variable as fixed or random.

      Reply
  4. Silesji says

    August 16, 2019 at 5:52 am

    How can I download this

    Reply
  5. Shaloo Verma says

    June 18, 2018 at 3:42 pm

    Hi Karen,

    Do you have example code for your first example in SAS that you can share. How would I code for controlling for both patient and physician characteristics on one model?

    Thanks for the help!

    Reply
  6. Patricia says

    July 31, 2017 at 9:07 am

    Is this the same as a cross-level interaction for a multilevel model?

    Reply
    • Karen says

      July 31, 2017 at 9:41 am

      Hi Patricia,

      It’s not. A cross-level interaction in a multilevel model is an interaction among fixed effects, one of which is measured at level 1 and one of which is at level 2. The fact that you have level 1 and 2 indicates the random effects are nested. For example: students nested within teachers because each student has only one teacher.

      Crossed random effects means that your random factors themselves are crossed, not nested. So not each student having one teacher. Each student having every teacher.

      Reply
  7. Eva says

    August 15, 2015 at 11:30 am

    Hi Karen!
    First thank you, because this is the first article that gives me a clue, that what I have is called as “crossed design” in some way.

    But it seems, that I have a doubled crossed design:
    I got participants that reacted to 6 different alarms in 2 different alarm conditions. Every participant did the 6 alarms in both conditions (the order of condition was randomized, but it does not have an effect at the particular dependent variable).
    My problem is, that one of my dependent variables is nominal: they should identify the alarm and this variable has the values “identified correctly” and “identified wrong”.
    So with all other variables (on ratio scale) I just take the mean over the 6 alarms and use a repeated-measure ANOVA (hopefully this is correct, as the variance between alarms is small and the number is equal). But with this dichotomous variable I cannot just take the mean, am I right? So that I have to look at the alarms as another level in multilevel modeling?

    Do you have a clue what to do or am I just thinking wrong?

    Thanks a lot in advance!

    Reply
    • Karen Grace-Martin says

      March 12, 2019 at 9:43 am

      Hi Eva,

      Yes, you’ve got a mixed model. I would also not average 6 responses on the numerical outcomes. When you do that, you’re ignoring variability.

      Mixed designs can get complicated fast.

      Reply
  8. Klara says

    May 15, 2015 at 5:22 am

    Hi Karen!
    Thanks, as always your explanation are very paedagogic!

    A question: can you introduce a fixed effect, an independent variable, at the intersection of your two random effects at level 2? For instance, in your first example, if Dr A is happier at hospital 1 than at hospital 2; or if each doctor’s office differs by hospital? Or would you have to introduce a new level 2, say, doctors’ office, or a “doctor by hospital” level, which then is cross-classified by a doctors and hospitals (now being level 3)?
    Thanks again!
    Best wishes/Klara

    Reply
    • Karen Grace-Martin says

      March 12, 2019 at 9:44 am

      Hi Klara,

      Yes, you can. The doctor*hospital interaction exists as a “unit of measurement.”

      Reply
  9. Kelly says

    February 19, 2015 at 10:16 pm

    Hi Karen,
    Thanks so much for this very helpful page. I have a question for you if you have the time: Is there any minimum requirement for the number of observations within each cross-classification (each cell, if you’re thinking of it in terms of rows-by-columns)? Without getting into the details, I have a study in which “Raters” evaluated stimuli collected from “Donors” (and nearly every Rater evaluated stimuli from each of the Donors). Thus, stimulus ratings are cross-classifed into Rater-by-Donor cells. I plan to examine how these ratings vary as a function of Rater characteristics, Donor characteristics, and the compatibility of each Rater-Donor pair (specifically, proportion of alleles that the Rater and Donor have in common at one particular locus). So, a cross-classified model seems like the way to go. The only potential problem is that each cell will contain just one single rating… or is that not a problem? Thanks!

    Reply
    • Karen Grace-Martin says

      March 12, 2019 at 9:45 am

      Hi Kelly,

      One is sufficient. It means there will be some interactions you can’t test, but this is a very common design.

      Reply
  10. Joaquín says

    September 22, 2014 at 3:43 pm

    Hello,

    My data contains product prices, observed for several stores, which belong to a particular chain.

    I’m thinking of doing a variance decomposition. Thing is I’m not sure how my model should look like. I should have a Random Effect for chain, and for store. But how about products?

    Thank you in advance for any suggestion.

    Reply
    • Karen Grace-Martin says

      March 12, 2019 at 9:46 am

      Hi Joaquin,

      It could be that chain, store, and product are all random factors or just some. I would have to know much more detail about your study.

      Reply
  11. Mike Bailey says

    June 2, 2014 at 12:19 am

    How about this example–-does it apply? I study a personality trait that can be inferred from videos. The trait differs by sexual orientation (gay/straight). I have 60 targets (30 gay 30 straight) and 60 raters (30 gay and 30 straight; we are interested in whether rater’s sexual orientation increases accuracy). We’d like to generalize both beyond our targets and beyond our raters, so in that sense we’d like to consider them random effects. Every rater sees every target, so this seems completely crossed. I’m having trouble mapping my situation onto any of the above though.

    Reply
    • Karen says

      June 3, 2014 at 10:58 am

      Hi Mike,

      It’s honestly hard for me to tell without really understanding every detail of the design, but it certainly sounds like it applies. Your raters are technically your “subjects” and the targets are like the “trials.”

      Reply
  12. Emily says

    May 30, 2014 at 2:21 pm

    Hi Karen —

    Thank you so much for your site — it is an amazing resource, and has helped clarify my thinking about these issues a lot. I am also writing to ask about your example 3 above. In that scenario, I read distractor as being nested within both participant and behavior — as some participants may be more sensitive to distractors than others and some behaviors may be more or less prone to disruption by distractors. When modeling this in SPSS, is the idea that distractor would be added as a fixed effect, and then nested within the two random effect commands?
    Many thanks,
    Emily

    Reply
  13. Sarah says

    April 18, 2014 at 10:56 am

    I’m not sure what you mean by conditions or replicates. We do not have treatment groups. All images were viewed by all subjects in the study and the response is measured at the image level. I want to examine the effect of different demographic factors on the response. I also want to include whether the image was viewed first in the model. Do I have to do anything special when specifying the model if some covariates measured at the image level and others at the subject level?

    Reply
  14. Sarah says

    April 7, 2014 at 1:10 pm

    I have a study that I think has crossed random effects. Participants rated 7 images on the same outcome variable. Each participant rated the same 7 images (in random order). Dependent variables are demographics and some other variables measured at the image level. I’m wondering how do you specify this with SAS? Do I do two random statements?

    Also, my original thought with how to analyse this data was to use a random intercept model. Then I could include image as a covariate to see the differences in ratings for each image. Would that be valid?

    Reply
    • Karen says

      April 7, 2014 at 1:50 pm

      Hi Sarah,

      It could be. It depends on whether those 7 images represent 7 conditions or if some are replicates. If you want to compare the actual images, then don’t make it random.

      But yes, two random statements in SAS. You can have a random intercept for subject and a random intercept for image.

      Reply
  15. Lisa says

    February 28, 2012 at 12:21 pm

    Thanks so much for discussing this topic, I am having a hard time understanding the differences between nesting and cross-classification and this post definitely points me in the right direction!

    I am trying to understand the difference between the two with respect to a standard repeated measures design (for example, measuring participants’ emotional responses to a stressor for 1 hour at 10 minute intervals). I am used to thinking of this type of design as time nested within participant, but now I am wondering if time and participant are actually crossed because each participant is measured at every time (and every time is assessed for every participant). Am I understanding this correctly?

    1) IF NO: can please you elaborate on the difference between nested and cross-classified designs? I must have misunderstood something along the way.

    2) IF YES: I am unclear why ‘trial’ in your third example is nested within behavior and participant. Doesn’t every participant receive every trial just like every participant receives every behavior? I am not clear as to why this design is not completely cross-classified.

    Also, do you know of any resources for SPSS nested versus cross-classified syntax?

    Thank you for your help!

    Reply
    • Karen says

      February 28, 2012 at 3:34 pm

      Hi Lisa,

      Great questions.

      In answer to your first question, you are correct–in a standard repeated measures study where every participant is measured at every time point, participant and time point are crossed.

      2) Trial, in this example, is literally the individual trial, not the behavior. If 15 subjects each see 10 behaviors, that’s 150 trials. Each of those trials occurs for only one subject and one behavior. So each subject has 10 trials. So subject and behavior are crossed, but trail is nested within both subject and trial.

      Make sense?
      Karen

      Reply
      • Lisa says

        February 28, 2012 at 4:05 pm

        Thanks for your quick and helpful response! I am still slightly confused. I hate to get hung up on a specific example, but this example is very relevant to my own research, so figuring out the source of my confusion will go a long way!

        I think I am unclear about why we need to define ‘trial’ and how we would use ‘trial’ in an analysis. Specifically, it sounds like trial is just a combination of subject and behavior such that each trial is a unique combination of one person and one behavior. If that is the case, why do we need to identify trial in addition to subject and behavior – doesn’t specifying subject and behavior give us this information already?

        I am not sure if the following metaphor will make sense, but it feels similar to specifying a particular ‘cell’ in a factorial experiment. If there are two independent variables with two levels each, there are four total ‘cells’. These cells are a combination of the other two factors and they are unique. I guess I am confused because I don’t understand why we need to define the specific ‘cell’ because (at least in an anova framework) those cells are created automatically by defining two crossed factors.

        Is there any chance you have SPSS syntax written for example 3? Perhaps seeing the actual syntax will help me figure things out. Thanks again for your help!

        Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Please note that, due to the large number of comments submitted, any questions on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.

Primary Sidebar

This Month’s Statistically Speaking Live Training

  • Member Training: Analyzing Pre-Post Data

Upcoming Free Webinars

Poisson and Negative Binomial Regression Models for Count Data

Upcoming Workshops

  • Analyzing Count Data: Poisson, Negative Binomial, and Other Essential Models (Jul 2022)
  • Introduction to Generalized Linear Mixed Models (Jul 2022)

Copyright © 2008–2022 The Analysis Factor, LLC. All rights reserved.
877-272-8096   Contact Us

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT