I recently gave a free webinar on Principal Component Analysis. We had almost 300 researchers attend and didn’t get through all the questions. This is part of a series of answers to those questions.
If you missed it, you can get the webinar recording here.
Question: Can we use PCA for reducing both predictors and response variables?
In fact, there were a few related but separate questions about using and interpreting the resulting component scores, so I’ll answer them together here.
How could you use the component scores?
A lot of times PCAs are used for further analysis — say, regression. How can we interpret the results of regression?
Let’s say I would like to interpret my regression results in terms of original data, but they are hiding under PCAs. What is the best interpretation that we can do in this case?
Answer:
So yes, the point of PCA is to reduce variables — create an index score variable that is an optimally weighted combination of a group of correlated variables.
And yes, you can use this index variable as either a predictor or response variable.
It is often used as a solution for multicollinearity among predictor variables in a regression model. Rather than include multiple correlated predictors, none of which is significant, if you can combine them using PCA, then use that.
It’s also used as a solution to avoid inflated familywise Type I error caused by running the same analysis on multiple correlated outcome variables. Combine the correlated outcomes using PCA, then use that as the single outcome variable. (This is, incidentally, what MANOVA does).
In both cases, you can no longer interpret the individual variables.
You may want to, but you can’t.
Let’s use the example we used in the webinar. In this example, the ultimate research question was about predicting the expected life span of different mammal species. We found that we had a set of correlated predictor variables: weight, exposure while sleeping, hours of sleep per day, and a rating of how vulnerable the animal is to predation.
These four variables are clearly very distinct concepts. We may want to be able to understand and interpret the relationship between weight and life span.
And we may want to separately understand the relationship between exposure during sleep and lifespan.
They’re conceptually different.
Even so, in this data set, you can’t entirely distinguish between them. You can’t entirely isolate the effect of weight on lifespan if they’re too correlated.
Think about it — if all the zebras and bison sleep out in the open and weigh a lot and the bats and shrews sleep in enclosed spaces and weigh little, then you can’t separate out weight from sleep exposure in your data set.
And in our PCA we said, it’s really not possible to separate out the effects of these four variables. We explain most of the information in these four variables in just one index.
So our combined index variable is what we have to interpret. If it turns out that being high on this combined variable predicts longer lifespan, you have to interpret your regression output that way.
