Post-hoc tests, pairwise or other linear contrasts, are typical in an analysis of variance (ANOVA) setting to understand which group means differ. They incorporate p-value adjustments to avoid concluding that group means differ when they actually do not. There are several adjustments that can be considered for conducting multiple post-hoc tests, including single-step and stepwise adjustments. [Read more…] about Member Training: ANOVA Post-hoc Tests: Practical Considerations
Statistical analysis
Member Training: Writing Study Design and Statistical Analysis Plans
One component often overlooked in the ‘Define & Design’ phase of a study, is writing the analysis plan. The statistical analysis plan integrates a lot of information about the study including the research question, study design, variables and data used, and the type of statistical analysis that will be conducted.
[Read more…] about Member Training: Writing Study Design and Statistical Analysis Plans
Member Training: Choosing the Best Statistical Analysis
Before you can write a data analysis plan, you have to choose the best statistical test or model. You have to integrate a lot of information about your research question, your design, your variables, and the data itself.
[Read more…] about Member Training: Choosing the Best Statistical Analysis
The Four Stages of Statistical Skill
At The Analysis Factor, we are on a mission to help researchers improve their statistical skills so they can do amazing research.
We all tend to think of “Statistical Analysis” as one big skill, but it’s not.
Over the years of training, coaching, and mentoring data analysts at all stages, I’ve realized there are four fundamental stages of statistical skill:
Stage 1: The Fundamentals
Stage 2: Linear Models
Stage 3: Extensions of Linear Models
Stage 4: Advanced Models
There is also a stage beyond these where the mathematical statisticians dwell. But that stage is required for such a tiny fraction of data analysis projects, we’re going to ignore that one for now.
If you try to master the skill of “statistical analysis” as a whole, it’s going to be overwhelming.
And honestly, you’ll never finish. It’s too big of a field.
But if you can work through these stages, you’ll find you can learn and do just about any statistical analysis you need to. [Read more…] about The Four Stages of Statistical Skill
Member Training: A Quick Introduction to Weighting in Complex Samples
A few years back the winning t-shirt design in a contest for the American Association of Public Opinion Research read “Weighting is the Hardest Part.” And I don’t think the t-shirt was referring to 
anything about patience!
Most statistical methods assume that every individual in the sample has the same chance of selection.
Complex Sample Surveys are different. They use multistage sampling designs that include stratification and cluster sampling. As a result, the assumption that every selected unit has the same chance of selection is not true.
To get statistical estimates that accurately reflect the population, cases in these samples need to be weighted. If not, all statistical estimates and their standard errors will be biased.
But selection probabilities are only part of weighting. [Read more…] about Member Training: A Quick Introduction to Weighting in Complex Samples
Member Training: Statistical Rules of Thumb: Essential Practices or Urban Myths?
There are many rules of thumb in statistical analysis that make decision making and understanding results much easier.
Have you ever stopped to wonder where these rules came from, let alone if there is any scientific basis for them? Is there logic behind these rules, or is it propagation of urban legends?
In this webinar, we’ll explore and question the origins, justifications, and some of the most common rules of thumb in statistical analysis, like:
[Read more…] about Member Training: Statistical Rules of Thumb: Essential Practices or Urban Myths?