Examples for Writing up Results of Mixed Models

by Karen Grace-Martin

One question I always get in my Repeated Measures Workshop is:

“Okay, now that I understand how to run a linear mixed model for my study, how do I write up the results?”

This is a great question.

There are many pieces of the linear mixed models output that are identical to those of any linear model–regression coefficients, F tests, means.

But there is also a lot that is new, like intraclass correlations and information criteria.  And a lot of output we’re used to seeing, like R squared, isn’t there anymore.

And because the model is more complicated, you may need to include in your paper more information about how you set up the model.  For example, you usually need to say whether you included a random intercept or slope (and at which level) and which covariance structure you chose for the residuals.

The problem I have in answering this is how you write it up is very much dependent on who you’re writing for.

Writing for Journals and Committees

The first thing to consider is your field and how familiar readers from your field will be with mixed models.  I’ve worked with clients whose reviewers had never heard of them.

If you’re in a field like this, one of two things will happen.

1. reviewers will be suspicious that you were making up some hocus-pocus statistics to get significant p-values.  Or

2. reviewers will have no idea what you’re talking about and can’t evaluate what you’ve done.  Furthermore, they’ll insist you report statistics that aren’t available in mixed models, like eta-squared.

If that’s your situation, you’re going to have to write it up with a bit more detail than you otherwise would.  Confused reviewers won’t be inclined to accept your paper.  Educate your readers about the methods.  Explain not just what you did, but why it was necessary.

Be generous with citations of not only papers that used mixed models but also those that explain what they are.

If you’re in a field where mixed models are more familiar and most readers will understand them, you’ll need to give enough detail that someone who understands mixed models could evaluate the approach.  This means you will need to say which random effects you included and which covariance structure you chose.  But you won’t have to explain what a random effect does.

Writing for non-statistical audiences

What if your audience isn’t a research audience, but your company’s marketing managers or your agency’s clinical staff?  They likely never needed statistics classes and have no understanding at all of what you’re doing. They just want to understand whether the intervention worked and they’re counting on you to know what you’re doing statistically.

In that case, you want to eliminate as much statistical jargon as possible.  Do everything you can to explain anything you can in English.  You can always put the statistical details in an appendix in case some future researcher comes across it.

They may understand “I used a linear mixed model because it accounts for the fact that multiple responses from the same person are more similar than responses from other people.”  But they won’t want to know how or why this is true.

Use a model

The ideal situation is to use as a guide a published paper that used the same type of mixed model in the journal you’re submitting to.

I can’t usually supply that to researchers, because I work with so many in different fields.

So I thought I’d try this.  Here is a list of a few papers I’ve worked on personally that used mixed models.

Feel free to look them up, in case it helps.

  • J Tee Todd, Susan G Butler, Drew P Plonk, Karen Grace-Martin, Cathy A Pelletier. (2012). Effects of chemesthetic stimuli mixtures with barium on swallowing apnea duration. The Laryngoscope, 122(10):2248-51.
  • Catriona M Steele, Gemma L Bailey, Sonja M Molfenter, Erin M Yeates, Karen Grace-Martin. (2010). Pressure profile similarities between tongue resistance training tasks and liquid swallows.  The Journal of Rehabilitation Research and Development, 47(7):651-60.
  • Susan Butler, Karen Grace-Martin. (2010). The Effect of Chemesthesis on Swallowing Apnea Duration.  Otolaryngology-Head and Neck Surgery, 143(2).

One other suggestion I’ve found helpful.  Try googling:

“used a linear mixed model” .pdf field

Type that in exactly, with the quotes, but replace the word field with whatever your field is: nursing, sociology, etc.  You will be surprised what you may find.

My request to you

If you have worked on or know of a paper that used mixed models, please give us the reference in the comments.  Links to online versions are great too, if you have one.

Trust me, many people in your field are looking for an example and will be happy to cite it.

Fixed and Random Factors in Mixed Models
One of the hardest parts of mixed models is understanding which factors to make fixed and which to make random. Learn the important criteria to help you decide.

{ 14 comments… read them below or add one }

Sarah

Thank you so much!!
Here are two papers in linguistics
Lukyanenko, C., & Fisher, C. (2016). Where are the cookies? Two-and three-year-olds use number-marked verbs to anticipate upcoming nouns. Cognition, 146, 349-370.
Kwon, H. (2017). Language experience, speech perception and loanword adaptation: Variable adaptation of English word-final plosives into Korean. Journal of Phonetics, 60, 1-19.

Reply

Candace Loy

Found a marine biology and ecology paper that has a useful way of reporting mixed models. https://www.nature.com/articles/srep28875#t2

Reply

nadia

thank you Candace

Reply

Emily

Thanks so much for this, Karen. You made the work of researching how to report this specific for my audience seems much less daunting!

Here is a behavioral ecology paper that reports LMM

http://homepages.abdn.ac.uk/julienmartin/pages/content/uploads/Martin-Réale-2008-AB.pdf

Reply

Emily

Forgot to include a link to helpful tutorials I found (which include a little bit about how to report LMM) by Bodo Winter:

http://www.bodowinter.com/tutorials.html

Reply

Kristina Mathiasen

Thank you very much for this. I have just been introduced to R to analyse my data for my final year BSc research project so this is very helpful. Thank you.

Reply

Christina

Thank you for the clear explanation, and for links to related topics. And, I appreciate the citations listed by other readers.

Reply

K

Another example Relationship Between Drug Dreams, Affect, and Craving
During Treatment for Substance Dependence
Hel´ ene Tanguay, MSc, Antonio Zadra, PhD, Daniel Good, BA, and Francesco Leri, PhD

Reply

Juliane Burghardt

These authors now explain mixed models to social psychologists:
Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of personality and social psychology, 103(1), 54-69.

Reply

Erin Snook

For the ecology field, the following paper uses linear mixed models:

XU, C., LETCHER, B. H. and NISLOW, K. H. (2010), Context-specific influence of water temperature on brook trout growth rates in the field. Freshwater Biology, 55: 2253–2264. doi:10.1111/j.1365-2427.2010.02430.x

Reply

C

Saviour!

Reply

Karen

Here is another one, from one of my clients:
http://www.tandfonline.com/doi/full/10.1080/1754730X.2015.1110495

Reply

Gavin Northey

Here’s a couple of articles using linear mixed models:

Schiefer, J. and Fischer C. (2008). The gap between wine expert ratings and consumer preferences: Measures, determinants and marketing implications. International Journal of Wine Business Research, Vol. 20, No. 4, 335 – 351.

Tainsky, S. (2009). “Television broadcast demand for National Football League contests.” Journal of Sports Economics 11(6): 629-640.

Sulmont-Rossé, C., et al. (2008). “Impact of the arousal potential of uncommon drinks on the repeated exposure effect.” Food Quality and Preference 19(4): 412-420.

Reply

Catherine Ortner

Here are a couple of examples of mixed models used in articles in an APA journal, Emotion:
Denny, B. T., & Ochsner, K. N. (2014). Behavioral effects of longitudinal training in cognitive reappraisal. Emotion, 14(2), 425–33. doi:10.1037/a0035276
Crane, C. A., & Testa, M. (2014). Daily Associations Among Anger Experience and Intimate Partner Aggression Within Aggressive and Nonaggressive Community Couples. Emotion, 14(5), 985–994.

Reply

Leave a Comment

Please note that, due to the large number of comments submitted, any comments on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.

Previous post:

Next post: