Mixed and Multilevel Models

online workshops

Analyzing Repeated Measures Data: ANOVA and Mixed Model Approaches

When it comes to analyzing repeated measures data, there are three main approaches: Multivariate GLM (aka, repeated measures ANOVA), Marginal Model, and Mixed Model. Each has its own advantages and disadvantages. Depending on what repeated measures designs you’re dealing with, some approaches work better than others. learn more

Introduction to Generalized Linear Mixed Models

GLMM is a set of powerful and versatile models that allow data analysts to deal with scenarios that other models cannot. You need a GLMM if you have data that aren’t independent because observations are clustered andy our outcome variable has some distribution that doesn’t result in normally distributed errors. learn more

the craft of statistical analysis free webinars

Random Intercept and Random Slope Models

Mixed modeling is rarely, if ever, covered in even upper-level statistics courses. Trying to learn it on your own is like prepping to perform your own appendectomy — and just about as painful. But there IS an easier path to learning mixed models, one that researchers without a Ph.D. in statistics can understand. learn more

Fixed and Random Factors in Mixed Models

What is the difference between fixed and random factors in Mixed Models? How do you figure out which one you have? learn more

What is GLMM, and When Should You Use It?

You’ve probably heard of GLMM (generalized linear mixed models) — or maybe you’ve heard of one of its popular software implementations, such as glmer or GLIMMIX. Are you wondering if this is something that you need to use when analyzing your data? learn more

statistically speaking member trainings

Crossed and Nested Factors

We often talk about nested factors in mixed models — students nested in classes, observations nested within subject. But in all but the simplest designs, it’s not that straightforward. learn more

Latent Growth Curve Models

What statistical model would you use for longitudinal data to analyze between-subject differences with within-subject change? Most analysts would respond, “a mixed model,” but have you ever heard of latent growth curves? Latent Growth Curve models use a Structural Equation Modeling approach to model change over time, which introduces quite a bit of flexibility. learn more

Matrix Algebra for Data Analysts: A Primer

If you’ve been doing data analysis for very long, you’ve certainly come across terms, concepts, and processes of matrix algebra. Not just matrices, but matrix addition and multiplication, traces and determinants, eigenvalues and eigenvectors, inverting and transposing, and positive and negative definite. learn more

Power Analysis and Sample Size Determination Using Simulation

We discuss strategies and steps for using simulations to estimate sample size and power. We explore basic concepts of statistical power and effect size, a simulation-based approach to power analysis, and how to implement simulations in various popular software programs. learn more

articles at the analysis factor

The Unstructured Covariance Matrix: When It Does and Doesn’t Work

If you’ve ever done any sort of repeated measures analysis or mixed models, you’ve probably heard of the unstructured covariance matrix. They can be extremely useful, but they can also blow up a model if not used appropriately. In this article I will investigate some situations when they work well and some when they don’t work at all. learn more

Covariance Matrices, Covariance Structures, and Bears, Oh My!

Of all the concepts I see researchers struggle with as they start to learn high-level statistics, the one that seems to most often elicit the blank stare of incomprehension is the Covariance Matrix, and its friend, the Covariance Structure. And since understanding them is fundamental to a number of statistical analyses, it’s an incomprehension you can’t afford. learn more

Approaches to Repeated Measures Data: Repeated Measures ANOVA, Marginal, and Mixed Models

In a recent post, I discussed the differences between repeated measures and longitudinal data, and some of the issues that come up in each one. I want to expand on that discussion, and discuss the three approaches you can take to analyze repeated measures data. learn more

Sample Size Estimates for Multilevel Randomized Trials

If you learned much about calculating power or sample sizes in your statistics classes, chances are, it was on something very, very simple, like a z-test. But there are many design issues that affect power in a study that go way beyond a z-test, like repeated measures, clustering of individuals, blocking, and including covariates in a model. learn more

Five Extensions of the General Linear Model

Generalized linear models, linear mixed models, generalized linear mixed models, marginal models, GEE models. You’ve probably heard of more than one of them and you’ve probably also heard that each one is an extension of our old friend, the general linear model. learn more

The Repeated and Random Statements in Mixed Models for Repeated Measures

Linear Mixed Models, as implemented in SAS’s Proc Mixed, SPSS Mixed, R’s LMER, and Stata’s xtmixed, are an extension of the general linear model. They use more sophisticated techniques for estimation of parameters (means, variances, regression coefficients, and standard errors), and as the quotation says, are much more flexible. learn more

How Simple Should a Model Be? The Case of Insignificant Controls, Interactions, and Covariance Structures

“Everything should be made as simple as possible, but no simpler,” a quote attributed to Albert Einstein, is the general goal of model selection. You really do want the model to be as simple as possible, but still able to answer the research question of interest. This applies to many areas of model selection. learn more

The Difference Between Clustered, Longitudinal, and Repeated Measures Data

What is the difference between Clustered, Longitudinal, and Repeated Measures Data? You can use mixed models to analyze all of them. But the issues involved and some of the specifications you choose will differ. It’s a common question, and there is a lot of overlap in both the study design and in how you will analyze the data from these designs. learn more

When the Hessian Matrix Goes Wacky

If you have run mixed models much at all, you have undoubtedly been haunted by some version of this very obtuse warning: “The Hessian (or G or D) Matrix is not positive definite. Convergence has stopped.” Or “The Model has not Converged. Parameter Estimates from the last iteration are displayed.” What on earth does that mean? learn more

Five Advantages of Running Repeated Measures ANOVA as a Mixed Model

There are two ways to run a repeated measures analysis. The traditional way is to treat it as a multivariate test–each response is considered a separate variable. The other way is to it as a mixed model. While the multivariate approach is easy to run and quite intuitive, there are a number of advantages to running a repeated measures analysis as a mixed model. learn more

Concepts in Linear Regression You Need to Know Before Learning Multilevel Models

It seems very many researchers are needing to learn multilevel and mixed models, and I have to say, it’s not so easy on your own. I went to graduate school before it was taught in classes – we did learn mixed models as in Split Plot designs, but things have progressed a bit since then. So I too have had to learn them without benefit of a class, or teacher. learn more

Multilevel Models with Crossed Random Effects

Crossed random effects models are a little trickier than most mixed models, but they are quite common in many fields. Recognizing when you have one and knowing how to analyze the data when you do are important statistical skills. learn more

Confusing Statistical Terms #3: Level

Level is a term in statistics that is confusing because it has multiple meanings in different contexts (much like alpha and beta). There are three different uses of the term Level in statistics that mean completely different things. What makes this especially confusing is that all three of them can be used in the exact same research context. learn more

Measures of Model Fit for Linear Regression Models

A well-fitting regression model results in predicted values close to the observed data values. The mean model, which uses the mean for every predicted value, generally would be used if there were no useful predictor variables. The fit of a proposed regression model should therefore be better than the fit of the mean model. learn more

Mixed Up Mixed Models

Anyone doing mixed modeling in SAS should read this paper, originally presented at SUGI: SAS Users Group International conference. It compares the output from Proc Mixed and Proc GLM when specified different ways. learn more

Specifying Fixed and Random Factors in Mixed Models

One of the difficult decisions in mixed modeling is deciding which factors are fixed and which are random. And as difficult as it is, it’s also very important. Correctly specifying the fixed and random factors of the model is vital to obtain accurate analyses. learn more

recommended books

SAS for Mixed Models, Second Edition
by Ramon Littell, George Miliken, Walter Stroup, Russell Wolfinger, & Oliver Schabenberger

This is a pretty technical book, and is not for the statistically feeble. But if you’re doing mixed models, you’re not statistically feeble. That said, if you are doing Mixed Modeling in SAS, it’s a must-have book. Back in the Cornell Statistical Consulting office, we actually wore out our copy. learn more

Quantitative Methods in Population Health: Extensions of Ordinary Regression
by Mari Palta

Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence
by Judith D. Singer & John B. Willett

Multilevel Analysis for Applied Research: It’s Just Regression! (Methodology In The Social Sciences)
by Robert Bickel

Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling
by Tom A.B. Snijders and Roel Boskeran

Linear Mixed Models: A Practical Guide Using Statistical Software
by Brady West, Kathleen B. Welch and Andrzej T Galecki

journal articles and other resources

Arnold, Carolyn L. (1992) An Introduction to Hierarchical Linear Models. Measurement and Evaluation in Counseling and Development, 25, 58-90.

Singer, Judith D. (1998) Using SAS PROC MIXED to Fit Multilevel Models, Hierarchial Models, and Individual Growth Models. Journal of Educational and Behavioral Statistics, 24, 323-355.

Plewis, Ian. (1998) Multilevel Models. Social Research Update, 23.

web sources


Dr. Lesa Hoffman, of the University of Nebraska, has put on her website her entire lecture slides and podcasts for all her classes and workshops, including:

It includes MPlus, SAS, and SPSS examples.