One of the most confusing things about statistical analysis is the different vocabulary used for the same, or nearly-but-not-quite-the-same, concepts.
Sometimes this happens just because the same analysis was developed separately within different fields and named twice.
So people in different fields use different terms for the same statistical concept. Try to collaborate with a colleague in a different field and you may find yourself awed by the crazy statistics they’re insisting on.
Other times, there is a level of detail that is implied by one term that isn’t true of the wider, more generic term. This level of detail is often about how the role of variables or effects affects the interpretation of output. (more…)
In a statistical model–any statistical model–there is generally one way that a predictor X and a response Y can relate:
This relationship can take on different forms, of course, like a line or a curve, but there’s really only one relationship here to measure.
Usually the point is to model the predictive ability, the effect, of X on Y.
In other words, there is a clear response variable*, although not necessarily a causal relationship. We could have switched the direction of the arrow to indicate that Y predicts X or used a two-headed arrow to show a correlation, with no direction, but that’s a whole other story.
For our purposes, Y is the response variable and X the predictor.
But a third variable–another predictor–can relate to X and Y in a number of different ways. How this predictor relates to X and Y changes how we interpret the relationship between X and Y. (more…)
It’s really easy to mix up the concepts of association (as measured by correlation) and interaction. Or to assume if two variables interact, they must be associated. But it’s not actually true.
In statistics, they have different implications for the relationships among your variables. This is especially true when the variables you’re talking about are predictors in a regression or ANOVA model.
Association
Association between two variables means the values of one variable relate in some way to the values of the other. It is usually measured by correlation for two continuous variables and by cross tabulation and a Chi-square test for two categorical variables.
Unfortunately, there is no nice, descriptive measure for association between one (more…)
I recently received this great question:
Question:
Hi Karen, ive purchased a lot of your material and read a lot of your pdf documents w.r.t. regression and interaction terms. Its, now, my general understanding that interaction for two or more categorical variables is best done with effects coding, and interactions cont v. categorical variables is usually handled via dummy coding. Further, i may mess this up a little but hopefully you’ll get my point and more importantly my question, i understand that
1) given a fitted line Y = b0 + b1 x1 + b2 x2 + b3 x1*x2, the interpretation for b3 is the diff of the effect of x1 on Y, when x2 changes one unit, if x1 and x2 are cont. ( also interpretation can be reversed in terms of x1 and x2). (more…)
“Everything should be made as simple as possible, but no simpler” – Albert Einstein*
For some reason, I’ve heard this quotation 3 times in the past 3 days. Maybe I hear it everyday, but only noticed because I’ve been working with a few clients on model selection, and deciding how much to simplify a model.
And when the quotation fits, use it. (That’s the saying, right?)
*For the record, a quick web search indicated this may be a paraphrase, but it still applies.
The quotation is the general goal of model selection. You really do want the model to be as simple as possible, but still able to answer the research question of interest.
This applies to many areas of model selection. Here are a few examples: (more…)
Need to dummy code in a Cox regression model?
Interpret interactions in a logistic regression?
Add a quadratic term to a multilevel model?
This is where statistical analysis starts to feel really hard. You’re combining two difficult issues into one.
You’re dealing with both a complicated modeling technique at Stage 3 (survival analysis, logistic regression, multilevel modeling) and tricky effects in the model (dummy coding, interactions, and quadratic terms).
The only way to figure it all out in a situation like that is to break it down into parts. (more…)
