• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • Home
  • Our Programs
    • Membership
    • Online Workshops
    • Free Webinars
    • Consulting Services
  • About
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Collaborate with Us
  • Statistical Resources
  • Contact
  • Blog
  • Login

Six Easy Ways to Complicate Your Analysis

by Karen Grace-Martin Leave a Comment

It’s easy to make things complex without meaning to. Especially in statistical analysis.

Sometimes that complexity is unavoidable. You have ethical and practical constraints on your study design and variable measurement. Or the data just don’t behave as you expected. Or the only research question of interest is one that demands many variables.

But sometimes it isn’t. Seemingly innocuous decisions lead to complicated analyses. These decisions occur early in the design, research questions, or variable choice.

If you’re prepared to do complex analyses, no problem. If you have the time and background to learn, great. Or if you have the resources to get help, you’re in great shape.

But if you have tight deadlines, no resources to hire someone to run things for you, no one to guide you, and there is a large gap between your current skills and what you’ll need for your complex analysis? You really want to keep things simple where you can.

Necessary or not, here are a few (of many) easy ways to make your analysis more complex. The more of these you include, the more complex your analysis will be.

Ways to make your analysis more complex

1.    Adding covariates

One of the great advantages of a regression model is its ability to include covariates. These are predictor variables whose effects you can adjust for.

But if you really need things simple, keep them out. The simplest statistical tests don’t accommodate covariates.  They have one independent and one dependent variable. No more.

Include even one covariate and you have to use some sort of regression model. There is a lot more complexity to these models, compared to tests with a single independent variable.

2.    Adding interactions

Interactions allow you to test hypotheses that cannot be tested any other way. They’re extremely useful.

But interactions do the same thing that covariates do — require a model instead of a simple test.

Interaction effects are even harder to understand and communicate, so only include them if you need them.

3.    Using an outcome variable that is not continuous, unbounded, and numerical

There is a reason that the first regression model you learn is a linear regression model. It’s the  simplest regression model. It forms the backbone of Stage 2 statistical skill. All other regression models are at Stage 3.

But not all outcome variables work for linear regression. It has an important assumption of normally distributed error terms, and you’ll rarely, if ever, meet that assumption if the outcome variable is categorical, ordinal, bounded, or discrete.

The good news is there are many other models that work for these outcome variables. The bad news: they’re more complicated.

4.    Adding blocking or other restrictions on randomization

Matching, pairing, counterbalancing, blocking, and a host of other design elements all need to be accounted for in the analysis. Yes, they have advantages. Stop and consider whether the advantages outweigh the complications to the analysis.

5.    Using scales instead of single variables

Many important variables cannot be directly measured with a single variable. But could a single variable work as a reliable and valid proxy?

Any time you combine variables into an scale, you can’t just use the scale without first establishing reliability and validity. This is an onerous process and requires different samples than the one you’re using in your analysis. If you’re using a well-established and previously tested scale, it’ll be simpler than if you create your own.

6.    Adding repeated measures

There are great reasons in statistics to use repeated measures. They allow tests of research questions about change or growth. They reduce standard errors, shrinking confidence intervals and increasing power.

But they also add complexity.  Repeated measures acts a bit like a multiplier. If you can keep everything else simple — no covariates, no other restrictions on randomization, normally distributed outcomes — then there are still simple tests for pre/post designs. But add it to any other complications and the complexity is magnified.

How to keep an analysis simple

One of the early steps of data analysis is writing a data analysis plan. The choice of analysis brings together the research question, the design, the types of variables, and potential data issues you are likely to encounter.

The more you plan ahead, the better equipped you will be to avoid unnecessary complications. There will be enough complications you can’t avoid.

But think long and hard about whether each of those six complications listed above is necessary. If it’s not, and you’re not in a place to uplevel your skills or hire help, do what you can to keep it simple.

Get trusted feedback on your plan if you’re not sure about it. It’s a lot easier to redo the plan than the analysis.

Tagged With: complex analysis, simple analysis, Study design

Related Posts

  • What to Do When You Can’t Run the Ideal Analysis 
  • What is a Chi-Square Test?
  • Member Training: Writing Study Design and Statistical Analysis Plans
  • Types of Study Designs in Health Research: The Evidence Hierarchy

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Please note that, due to the large number of comments submitted, any questions on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.

Primary Sidebar

This Month’s Statistically Speaking Live Training

  • Member Training: Analyzing Pre-Post Data

Upcoming Free Webinars

Poisson and Negative Binomial Regression Models for Count Data

Upcoming Workshops

  • Analyzing Count Data: Poisson, Negative Binomial, and Other Essential Models (Jul 2022)
  • Introduction to Generalized Linear Mixed Models (Jul 2022)

Copyright © 2008–2022 The Analysis Factor, LLC. All rights reserved.
877-272-8096   Contact Us

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT