• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • our programs
    • Membership
    • Online Workshops
    • Free Webinars
    • Consulting Services
  • statistical resources
  • blog
  • about
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Collaborate with Us
  • contact
  • login

PCA

Four Common Misconceptions in Exploratory Factor Analysis

by guest contributer  Leave a Comment

by Christos Giannoulis, PhD

Today, I would like to briefly describe four misconceptions that I feel are commonly perceived by novice researchers in Exploratory Factor Analysis:

Misconception 1: The choice between component and common factor extraction procedures is not so important.

In Principal Component Analysis, a set of variables is transformed into a smaller set of linear composites known as components. This method of analysis is essentially a method for data reduction.

[Read more…] about Four Common Misconceptions in Exploratory Factor Analysis

Tagged With: common factor analysis, communality, EFA, eigenvalue, Exploratory Factor Analysis, oblique rotation, orthogonal rotation, PCA, principal axis factor analysis, principal component analysis, rotation, sample size, simple structure

Related Posts

  • In Factor Analysis, How Do We Decide Whether to Have Rotated or Unrotated Factors?
  • Can You Use Principal Component Analysis with a Training Set Test Set Model?
  • In Principal Component Analysis, Can Loadings Be Negative?
  • How To Calculate an Index Score from a Factor Analysis

In Factor Analysis, How Do We Decide Whether to Have Rotated or Unrotated Factors?

by Karen Grace-Martin  1 Comment

I recently gave a free webinar on Principal Component Analysis. We had almost 300 researchers attend and didn’t get through all the questions. This is part of a series of answers to those questions.

If you missed it, you can get the webinar recording here.

Question: How do we decide whether to have rotated or unrotated factors?

Answer:

Great question. Of course, the answer depends on your situation.

When you retain only one factor in a solution, then rotation is irrelevant. In fact, most software won’t even print out rotated coefficients and they’re pretty meaningless in that situation.

But if you retain two or more factors, you need to rotate.

Unrotated factors are pretty difficult to interpret in that situation. [Read more…] about In Factor Analysis, How Do We Decide Whether to Have Rotated or Unrotated Factors?

Tagged With: coefficients, factor, PCA, principal component analysis, rotated, rotation, unrotated

Related Posts

  • Four Common Misconceptions in Exploratory Factor Analysis
  • In Principal Component Analysis, Can Loadings Be Negative?
  • How To Calculate an Index Score from a Factor Analysis
  • Can You Use Principal Component Analysis with a Training Set Test Set Model?

In Principal Component Analysis, Can Loadings Be Negative?

by Karen Grace-Martin  1 Comment

Here’s a question I get pretty often: In Principal Component Analysis, can loadings be negative and positive?

Answer: Yes.

Recall that in PCA, we are creating one index variable (or a few) from a set of variables. You can think of this index variable as a weighted average of the original variables.

The loadings are the correlations between the variables and the component. We compute the weights in the weighted average from these loadings.

The goal of the PCA is to come up with optimal weights. “Optimal” means we’re capturing as much information in the original variables as possible, based on the correlations among those variables.

So if all the variables in a component are positively correlated with each other, all the loadings will be positive.

But if there are some negative correlations among the variables, some of the loadings will be negative too.

An Example of Negative Loadings in Principal Component Analysis

Here’s a simple example that we used in our Principal Component Analysis webinar. We want to combine four variables about mammal species into a single component.

The variables are weight, a predation rating, amount of exposure while sleeping, and the total number of hours an animal sleeps each day.

If you look at the correlation matrix, total hours of sleep correlates negatively with the other 3 variables. Those other three are all positively correlated.

It makes sense — species that sleep more tend to be smaller, less exposed while sleeping, and less prone to predation. Species that are high on these three variables must not be able to afford much sleep.

Think bats vs. zebras.

Likewise, the PCA with one component has positive loadings for three of the variables and a negative loading for hours of sleep.

 

 

 

 

 

Species with a high component score will be those with high weight, high predation rating, high sleep exposure, and low hours of sleep.

Tagged With: Interpreting Loadings, Negative Loadings, PCA, principal component analysis

Related Posts

  • Four Common Misconceptions in Exploratory Factor Analysis
  • In Factor Analysis, How Do We Decide Whether to Have Rotated or Unrotated Factors?
  • How To Calculate an Index Score from a Factor Analysis
  • Can You Use Principal Component Analysis with a Training Set Test Set Model?

How To Calculate an Index Score from a Factor Analysis

by Karen Grace-Martin  15 Comments

One common reason for running Principal Component Analysis (PCA) or Factor Analysis (FA) is variable reduction.

In other words, you may start with a 10-item scale meant to measure something like Anxiety, which is difficult to accurately measure with a single question.

You could use all 10 items as individual variables in an analysis–perhaps as predictors in a regression model.

But you’d end up with a mess.

Not only would you have trouble interpreting all those coefficients, but you’re likely to have multicollinearity problems.

And most importantly, you’re not interested in the effect of each of those individual 10 items on your [Read more…] about How To Calculate an Index Score from a Factor Analysis

Tagged With: Factor Analysis, Factor Score, index variable, PCA, principal component analysis

Related Posts

  • Four Common Misconceptions in Exploratory Factor Analysis
  • In Factor Analysis, How Do We Decide Whether to Have Rotated or Unrotated Factors?
  • Can We Use PCA for Reducing Both Predictors and Response Variables?
  • The Fundamental Difference Between Principal Component Analysis and Factor Analysis

Primary Sidebar

This Month’s Statistically Speaking Live Training

  • Member Training: Multinomial Logistic Regression

Upcoming Workshops

    No Events

Upcoming Free Webinars

TBA

Quick links

Our Programs Statistical Resources Blog/News About Contact Log in

Contact

Upcoming

Free Webinars Membership Trainings Workshops

Privacy Policy

Search

Copyright © 2008–2023 The Analysis Factor, LLC.
All rights reserved.

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT