• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • our programs
    • Membership
    • Online Workshops
    • Free Webinars
    • Consulting Services
  • statistical resources
  • blog
  • about
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Collaborate with Us
  • contact
  • login

Confusing Statistical Terms

Confusing Statistical Term #13: Missing at Random and Missing Completely at Random

by Karen Grace-Martin  5 Comments

Stage 2One of the important issues with missing data is the missing data mechanism. You may have heard of these: Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR).

The mechanism is important because it affects how much the missing data bias your results. This has a big impact on what is a reasonable approach to dealing with the missing data.  So you have to take it into account in choosing an approach.

The concepts of these mechanisms can be a bit abstract.missing data

And to top it off, two of these mechanisms have really confusing names: Missing Completely at Random and Missing at Random.

Missing Completely at Random (MCAR)

Missing Completely at Random is pretty straightforward.  What it means is what is [Read more…] about Confusing Statistical Term #13: Missing at Random and Missing Completely at Random

Tagged With: MAR, MCAR, missing at random, missing completely at random, Missing Data

Related Posts

  • Missing Data Mechanisms: A Primer
  • When Listwise Deletion works for Missing Data
  • How to Diagnose the Missing Data Mechanism
  • Multiple Imputation in a Nutshell

Six terms that mean something different statistically and colloquially

by guest contributer  Leave a Comment

by Kim Love and Karen Grace-Martin

Statistics terminology is confusing.

Sometimes different terms are used to mean the same thing, often in different fields of application. Sometimes the same term is used to mean different things. And sometimes very similar terms are used to describe related but distinct statistical concepts.

[Read more…] about Six terms that mean something different statistically and colloquially

Tagged With: bias, confusing statistical terms, Correlation, Error, odds, random, significance, terminology

Related Posts

  • Member Training: Confusing Statistical Terms
  • Why Statistics Terminology is Especially Confusing
  • Confusing Statistical Term #8: Odds
  • What is a Dunnett’s Test?

Confusing Statistical Term #10: Mixed and Multilevel Models

by Karen Grace-Martin  5 Comments

What’s the difference between Mixed and Multilevel Models? What about Hierarchical Models or Random Effects models?

I get this question a lot.

The answer: very little.

[Read more…] about Confusing Statistical Term #10: Mixed and Multilevel Models

Tagged With: crossed random effects, hierarchical linear model, individual growth curve model, mixed effects model, mixed model, multilevel model, random coefficient model, random effect, random intercept, Random Slope Model

Related Posts

  • Multilevel, Hierarchical, and Mixed Models–Questions about Terminology
  • The Difference Between Random Factors and Random Effects
  • Is there a fix if the data is not normally distributed?
  • What packages allow you to deal with random intercept and random slope models in R?

Why Statistics Terminology is Especially Confusing

by Karen Grace-Martin  4 Comments

The field of statistics has a terminology problem.

It affects students’ ability to learn statistics. It affects researchers’ ability to communicate with statisticians; with collaborators in different fields; and of course, with the general public.

It’s easy to think the real issue is that statistical concepts are difficult. That is true. It’s not the whole truth, though. [Read more…] about Why Statistics Terminology is Especially Confusing

Tagged With: statistical terminology, statistics, terminology

Related Posts

  • Six terms that mean something different statistically and colloquially
  • The Wisdom of Asking Silly Statistics Questions
  • What is a Dunnett’s Test?
  • Best Practices for Organizing your Data Analysis

Confusing Statistical Term #9: Multiple Regression Model and Multivariate Regression Model

by Karen Grace-Martin  25 Comments

First Published 4/29/09;Stage 2
Updated 2/23/21 to give more detail.

Much like General Linear Model and Generalized Linear Model in #7, there are many examples in statistics of terms with (ridiculously) similar names, but nuanced meanings.

Today I talk about the difference between multivariate and multiple, as they relate to regression.

Multiple Regression

A regression analysis with one dependent variable and eight independent variables is NOT a multivariate regression model.  It’s a multiple regression model.

And believe it or not, it’s considered a univariate model.

This is uniquely important to remember if you’re an SPSS user. Choose Univariate GLM (General Linear Model) for this model, not multivariate.

I know this sounds crazy and misleading because why would a model that contains nine variables (eight Xs and one Y) be considered a univariate model?

It’s because of the fundamental idea in regression that Xs and Ys aren’t the same. We’re using the Xs to understand the mean and variance of Y. This is why the residuals in a linear regression are differences between predicted and actual values of Y. Not X.

(And of course, there is an exception, called Type II or Major Axis linear regression, where X and Y are not distinct. But in most regression models, Y has a different role than X).

It’s the number of Ys that tell you whether it’s a univariate or multivariate model. That said, other than SPSS, I haven’t seen anyone use the term univariate to refer to this model in practice. Instead, the assumed default is that indeed, regression models have one Y, so let’s focus on how many Xs the model has. This leads us to…

Simple Regression: A regression model with one Y (dependent variable) and one X (independent variable).

Multiple Regression: A regression model with one Y (dependent variable) and more than one X (independent variables).

References below.

Multivariate Regression

Multivariate analysis ALWAYS describes a situation with multiple dependent variables.

So a multivariate regression model is one with multiple Y variables. It may have one or more than one X variables. It is equivalent to a MANOVA: Multivariate Analysis of Variance.

Other examples of Multivariate Analysis include:

  • Principal Component Analysis
  • Factor Analysis
  • Canonical Correlation Analysis
  • Linear Discriminant Analysis
  • Cluster Analysis

But wait. Multivariate analyses like cluster analysis and factor analysis have no dependent variable, per se. Why is it about dependent variables?

Well,  it’s not really about dependency.  It’s about which variables’ mean and variance is being analyzed.  In a multivariate regression, we have multiple dependent variables, whose joint mean is being predicted by the one or more Xs. It’s the variance and covariance in the set of Ys that we’re modeling (and estimating in the Variance-Covariance matrix).

Note: this is actually a situation where the subtle differences in what we call that Y variable can help.  Calling it the outcome or response variable, rather than dependent, is more applicable to something like factor analysis.

So when to choose multivariate GLM?  When you’re jointly modeling the variation in multiple response variables.

References

In response to many requests in the comments, I suggest the following references.  I give the caveat, though, that neither reference compares the two terms directly. They simply define each one. So rather than just list references, I’m going to explain them a little.

  1. Neter, Kutner, Nachtsheim, Wasserman’s Applied Linear Regression Models, 3rd ed. There are, incidentally, never editions with slight changes in authorship. But I’m citing the one on my shelf.

Chapter 1, Linear Regression with One Independent Variable, includes:

“Regression model 1.1 … is “simple” in that there is only one predictor variable.”

Chapter 6 is titled Multiple Regression – I, and section 6.1 is “Multiple Regression Models: Need for Several Predictor Variables.” Interestingly enough, there is no direct quotable definition of the term “multiple regression.” Even so, it’s pretty clear. Go read the chapter to see.

There is no mention of the term “Multivariate Regression” in this book.

2. Johnson & Wichern’s Applied Multivariate Statistical Analysis, 3rd ed.

Chapter 7, Multivariate Linear Regression Models, section 7.1 Introduction. Here it says:

“In this chapter we first discuss the multiple regression model for the prediction of a single response. This model is then generalized to handle the prediction of several dependent variables.” (Emphasis theirs).

They finally get to Multivariate Multiple Regression in Section 7.7. Here they “consider the problem of modeling the relationship between m responses, Y1, Y2, …,Ym, and a single set of predictor variables.”

Misuses of the Terms

I’d be shocked, however, if there aren’t some books or articles out there where the terms are not used or defined  the way I’ve described them here, according to these references. It’s very easy to confuse these terms, even for those of us who should know better.

And honestly, it’s not that hard to just describe the model instead of naming it. “Regression model with four predictors and one outcome” doesn’t take a lot more words and is much less confusing.

If you’re ever confused about the type of model someone is describing to you, just ask.

Read More Explanations of Confusing Statistical Terms.

Tagged With: Multiple Regression, multivariate analysis, SPSS Multivariate GLM, SPSS Univariate GLM

Related Posts

  • The Difference Between R-squared and Adjusted R-squared
  • Same Statistical Models, Different (and Confusing) Output Terms
  • What is Multicollinearity? A Visual Description
  • Member Training: The Link Between ANOVA and Regression

Member Training: Confusing Statistical Terms

by guest contributer 

Learning statistics is difficult enough; throw in some especially confusing terminology and it can feel impossible! There are many ways that statistical language can be confusing.

Some terms mean one thing in the English language, but have another (usually more specific) meaning in statistics.  [Read more…] about Member Training: Confusing Statistical Terms

Tagged With: ancova, association, confounding variable, confusing statistical terms, Correlation, Covariate, dependent variable, Error, factor, General Linear Model, generalized linear models, independent variable, learning statistics, levels, listwise deletion, multivariate, odds, pairwise deletion, random error, selection bias, significant

Related Posts

  • Series on Confusing Statistical Terms
  • Six terms that mean something different statistically and colloquially
  • Confusing Statistical Term #8: Odds
  • The Difference Between Association and Correlation

  • Go to page 1
  • Go to page 2
  • Go to page 3
  • Go to Next Page »

Primary Sidebar

This Month’s Statistically Speaking Live Training

  • Member Training: The Link Between ANOVA and Regression

Upcoming Workshops

    No Events

Upcoming Free Webinars

TBA

Quick links

Our Programs Statistical Resources Blog/News About Contact Log in

Contact

Upcoming

Free Webinars Membership Trainings Workshops

Privacy Policy

Search

Copyright © 2008–2023 The Analysis Factor, LLC.
All rights reserved.

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT