• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • Home
  • About
    • Our Programs
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Guest Instructors
  • Membership
    • Statistically Speaking Membership Program
    • Login
  • Workshops
    • Online Workshops
    • Login
  • Consulting
    • Statistical Consulting Services
    • Login
  • Free Webinars
  • Contact
  • Login

Correlation

Member Training: Confusing Statistical Terms

by guest

Learning statistics is difficult enough; throw in some especially confusing terminology and it can feel impossible! There are many ways that statistical language can be confusing.

Some terms mean one thing in the English language, but have another (usually more specific) meaning in statistics.  [Read more…] about Member Training: Confusing Statistical Terms

Tagged With: ancova, association, confounding variable, confusing statistical terms, Correlation, Covariate, dependent variable, Error, factor, General Linear Model, generalized linear models, independent variable, learning statistics, levels, listwise deletion, multivariate, odds, pairwise deletion, random error, selection bias, significant

Related Posts

  • Series on Confusing Statistical Terms
  • Confusing Statistical Term #8: Odds
  • The Difference Between Association and Correlation
  • Member Training: Interpretation of Effect Size Statistics

The Difference Between Association and Correlation

by Karen Grace-Martin Leave a Comment

What does it mean for two variables to be correlated?

Is that the same or different than if they’re associated or related?

This is the kind of question that can feel silly, but shouldn’t. It’s just a reflection of the confusing terminology used in statistics. In this case, the technical statistical term looks like, but is not exactly the same as, the way we mean it in everyday English. [Read more…] about The Difference Between Association and Correlation

Tagged With: association, Bivariate Statistics, Correlation, Cramer's V, Kendall's tau-b, point-biserial, Polychoric correlations, rank-biserial, Somer's D, Spearman correlation, Stuart's tau-c, tetrachoric

Related Posts

  • Member Training: Confusing Statistical Terms
  • How to Interpret the Width of a Confidence Interval
  • Effect Size Statistics: How to Calculate the Odds Ratio from a Chi-Square Cross-tabulation Table
  • How Does the Distribution of a Population Impact the Confidence Interval?

Member Training: Interpretation of Effect Size Statistics

by guest

Effect size statistics are required by most journals and committees these days ⁠— for good reason. 

They communicate just how big the effects are in your statistical results ⁠— something p-values can’t do.

But they’re only useful if you can choose the most appropriate one and if you can interpret it.

This can be hard in even simple statistical tests. But once you get into  complicated models, it’s a whole new story. [Read more…] about Member Training: Interpretation of Effect Size Statistics

Tagged With: Cohen's d, Correlation, correlation indexes, effect size, effect size statistics, empirically derived, Glass, Hedges, interpreting, null hypothesis, probability of superiority, Proportion, strength association, superiority, variance

Related Posts

  • Member Training: Statistical Rules of Thumb: Essential Practices or Urban Myths?
  • Member Training: An Overview of Effect Size Statistics and Why They are So Important
  • September Member Training: Inference and p-values and Statistical Significance, Oh My!
  • Member Training: Confusing Statistical Terms

Member Training: Those Darn Ratios!

by TAF Support

Ratios are everywhere in statistics—coefficient of variation, hazard ratio, odds ratio, the list goes on. You see them reported in the literature and in your output.

You comment on them in your reports. You even (kinda) understand them. Or, maybe, not quite?

 

Please join Elaine Eisenbeisz as she presents an overview of the how and why of various ratios we use often in statistical practice.

[Read more…] about Member Training: Those Darn Ratios!

Tagged With: Coefficient of determination, Correlation, Hazard ratio, Likelihood ratio, odds ratio, ratio, relative risk, Variance inflation factor, variation

Related Posts

  • Member Training: Confusing Statistical Terms
  • Member Training: Interpretation of Effect Size Statistics
  • Member Training: Determining Levels of Measurement: What Lies Beneath the Surface
  • February Member Training: Choosing the Best Statistical Analysis

The Difference Between Interaction and Association

by Karen Grace-Martin 16 Comments

It’s really easy to mix up the concepts of association (a.k.a. correlation) and interaction.  Or to assume if two variables interact, they must be associated.  But it’s not actually true.

In statistics, they have different implications for the relationships among your variables, especially when the variables you’re talking about are predictors in a regression or ANOVA model.

Association

Association between two variables means the values of one variable relate in some way to the values of the other.  Association is usually measured by correlation for two continuous variables and by cross tabulation and a Chi-square test for two categorical variables.

Unfortunately, there is no nice, descriptive measure for association between one [Read more…] about The Difference Between Interaction and Association

Tagged With: Correlation, interaction

Related Posts

  • Using Pairwise Comparisons to Help you Interpret Interactions in Linear Regression
  • Interpreting Interactions Between Two Effect-Coded Categorical Predictors
  • Centering a Covariate to Improve Interpretability
  • What is a Confounding Variable?

Covariance Matrices, Covariance Structures, and Bears, Oh My!

by Karen Grace-Martin 33 Comments

Of all the concepts I see researchers struggle with as they start to learn high-level statistics, the one that seems to most often elicit the blank stare of incomprehension is the Covariance Matrix, and its friend, Covariance Structures.

And since understanding them is fundamental to a number of statistical analyses, particularly Mixed Models and Structural Equation Modeling, it’s an incomprehension you can’t afford.

So I’m going to explain what they are and how they’re not so different from what you’re used to.  I hope you’ll see that once you get to know them, they aren’t so scary after all.

What is a Covariance Matrix?

There are two concepts inherent in a covariance matrix–covariance and matrix.  Either one can throw you off.

Let’s start with matrix.  If you never took linear algebra, the idea of matrices can be frightening.  (And if you still are in school, I highly recommend you take it.  Highly).  And there are a lot of very complicated, mathematical things you can do with matrices.

But you, a researcher and data analyst, don’t need to be able to do all those complicated processes to your matrices.  You do need to understand what a matrix is, be able to follow the notation, and understand a few simple matrix processes, like multiplication of a matrix by a constant.

The thing to keep in mind when it all gets overwhelming is a matrix is just a table.  That’s it.

A Covariance Matrix, like many matrices used in statistics, is symmetric.  That means that the table has the same headings across the top as it does along the side.

Start with a Correlation Matrix

The simplest example, and a cousin of a covariance matrix, is a correlation matrix.  It’s just a table in which each variable is listed in both the column headings and row headings, and each cell of the table (i.e. matrix) is the correlation between the variables that make up the column and row headings.  Here is a simple example from a data set on 62 species of mammal:

From this table, you can see that the correlation between Weight in kg and Hours of Sleep, highlighted in purple, is -.307. Smaller mammals tend to sleep more.

You’ll notice that this is the same above and below the diagonal. The correlation of Hours of Sleep with Weight in kg is the same as the correlation between Weight in kg and Hours of Sleep.

Likewise, all correlations on the diagonal equal 1, because they’re the correlation of each variable with itself.

If this table were written as a matrix, you’d only see the numbers, without the column headings.

Now, the Covariance Matrix

A Covariance Matrix is very similar. There are really two differences between it and the Correlation Matrix. It has this form:

First, we have substituted the correlation values with covariances.

Covariance is just an unstandardized version of correlation.  To compute any correlation, we divide the covariance by the standard deviation of both variables to remove units of measurement.  So a covariance is just a correlation measured in the units of the original variables.

Covariance, unlike  correlation, is not constrained to being between -1 and 1. But the covariance’s sign will always be the same as the corresponding correlation’s. And a covariance=0 has the exact same meaning as a correlation=0: no linear relationship.

Because covariance is in the original units of the variables, variables on scales with bigger numbers and with wider distributions will necessarily have bigger covariances. So for example, Life Span has similar correlations to Weight and Exposure while sleeping, both around .3.

But values of Weight vary a lot (this data set contains both Elephants and Shrews), whereas Exposure is an index variable that ranges from only 1 to 5. So Life Span’s covariance with Weight (5113.27) is much larger than than with Exposure (10.66).

Second, the diagonal cells of the matrix contain the variances of each variable. A covariance of a variable with itself is simply the variance. So you have a context for interpreting these covariance values.

Once again, a covariance matrix is just the table without the row and column headings.

What about Covariance Structures?

Covariance Structures are just patterns in covariance matrices.  Some of these patterns occur often enough in some statistical procedures that they have names.

You may have heard of some of these names–Compound Symmetry, Variance Components, Unstructured, for example.  They sound strange because they’re often thrown about without any explanation.

But they’re just descriptions of patterns.

For example, the Compound Symmetry structure just means that all the variances are equal to each other and all the covariances are equal to each other. That’s it.

It wouldn’t make sense with our animal data set because each variable is measured on a different scale. But if all four variables were measured on the same scale, or better yet, if they were all the same variable measured under four experimental conditions, it’s a very plausible pattern.

Variance Components just means that each variance is different, and all covariances=0. So if all four variables were completely independent of each other and measured on different scales, that would be a reasonable pattern.

Unstructured just means there is no pattern at all.  Each variance and each covariance is completely different and has no relation to the others.

There are many, many covariance structures.  And each one makes sense in certain statistical situations.  Until you’ve encountered those situations, they look crazy.  But each one is just describing a pattern that makes sense in some situations.

 

Tagged With: Correlation, correlation matrix, Covariance Matrix, Covariance Structure, linear mixed model, mixed model, multilevel model, Structural Equation Modeling

Related Posts

  • March Member Training: Goodness of Fit Statistics
  • Multilevel, Hierarchical, and Mixed Models–Questions about Terminology
  • The Difference Between Random Factors and Random Effects
  • Six Differences Between Repeated Measures ANOVA and Linear Mixed Models

  • Go to page 1
  • Go to page 2
  • Go to Next Page »

Primary Sidebar

This Month’s Statistically Speaking Live Training

  • April Member Training: Statistical Contrasts

Upcoming Workshops

  • Logistic Regression for Binary, Ordinal, and Multinomial Outcomes (May 2021)
  • Introduction to Generalized Linear Mixed Models (May 2021)

Read Our Book



Data Analysis with SPSS
(4th Edition)

by Stephen Sweet and
Karen Grace-Martin

Statistical Resources by Topic

  • Fundamental Statistics
  • Effect Size Statistics, Power, and Sample Size Calculations
  • Analysis of Variance and Covariance
  • Linear Regression
  • Complex Surveys & Sampling
  • Count Regression Models
  • Logistic Regression
  • Missing Data
  • Mixed and Multilevel Models
  • Principal Component Analysis and Factor Analysis
  • Structural Equation Modeling
  • Survival Analysis and Event History Analysis
  • Data Analysis Practice and Skills
  • R
  • SPSS
  • Stata

Copyright © 2008–2021 The Analysis Factor, LLC. All rights reserved.
877-272-8096   Contact Us

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled

Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.

Non-necessary

Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.

SAVE & ACCEPT