• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
The Analysis Factor

The Analysis Factor

Statistical Consulting, Resources, and Statistics Workshops for Researchers

  • Home
  • About
    • Our Programs
    • Our Team
    • Our Core Values
    • Our Privacy Policy
    • Employment
    • Guest Instructors
  • Membership
    • Statistically Speaking Membership Program
    • Login
  • Workshops
    • Online Workshops
    • Login
  • Consulting
    • Statistical Consulting Services
    • Login
  • Free Webinars
  • Contact
  • Login

5 Reasons to Run Sample Size Calculations Before Collecting Data

by Karen Grace-Martin 3 Comments

Most of us run sample size calculations when a granting agency or committee requires it.  That’s reason 1.

That is a very good reason.  But there are others, and it can be helpful to keep these in mind when you’re tempted to skip this step or are grumbling through the calculations you’re required to do.

It’s easy to base your sample size on what is customary in your field (“I’ll use 20 subjects per condition”) or to just use the number of subjects in a similar study (“They used 150, so I will too”).

Sometimes you can get away with doing that.

However, there really are some good reasons beyond funding to do some sample size estimates. And since they’re not especially time-consuming, it’s worth doing them.

Often the most time consuming part is figuring out and writing the data analysis plan to base the calculations on, but that’s another step you should do anyway.

Reason 2:  Many, many published studies have very low power, and are bad sources for basing your sample size on.

As reported in Keppel, Cohen calculated the power of every study in a psychology journal for a year. The average power was just under 50%.

If power is 50% for a study, it basically means that that study had a 50% chance of finding significance for a real effect, given the sample size, the effect size, and the statistical test.  Because these were published studies, they must have had significant results.  But there were probably a lot of other studies (just as many) that never got published because they didn’t have adequate power.

If you now attempt to build on that study and you use the same sample size, you only have a 50% change of replicating it with significant results. Do your own power calculation and raise the sample size, if needed.

Reason 3: A power calculation estimates not only how many participants you need, but how many you don’t need.

You don’t want to spend any more  resources–time, money, and energy–collecting more data than you need.  Save those resources for a follow-up study.

Especially if your study creates any risk, or even inconvenience, for your human or animal participants, you don’t want to oversize your study either. You don’t want to expose more participants than necessary to the risk.

Reason 4: When sample size calculations tell you you’re close, but have not quite enough subjects, you can make adjustments to the study that will increase the power in other ways.

Maybe you can adjust the way you’re measuring some of your variables to add precision or switch your design to something that will give you a little more power. Or make sure you include some controls that will control some of the random error.  All of these increase power without increasing sample size.

Reason 5: The biggest benefit of doing these calculations is to not waste years and thousands of dollars in grants or tuition pursuing an impossible analysis.

If sample size calculations indicate you need a thousand subjects to find  significant results but time, money, or ethical constraints limit you to 50, don’t do that study.

I know it’s painful to go back to square 1, but it’s much better to do it now than after 3 years of work.

Effect Size Statistics
Statistical software doesn't always give us the effect sizes we need. Learn some of the common effect size statistics and the ways to calculate them yourself.

Tagged With: Calculating Sample Size, Sample Size Calculations, Statistical power

Related Posts

  • 5 Steps for Calculating Sample Size
  • How Does the Distribution of a Population Impact the Confidence Interval?
  • How Confident Are You About Confidence Intervals?
  • How to Interpret the Width of a Confidence Interval

Reader Interactions

Comments

  1. Sigrid Gibson says

    October 26, 2020 at 12:38 pm

    Thank Karen for a great FREE tutorial on effect sizes. Very helpful in understanding the relationship between various measures and what they are describing, without an “over-heavy” mathematical explanation.

    Reply
  2. Bashir says

    May 27, 2012 at 2:12 am

    I am glad with it. However, I am more interested in “second generation of multivariate analysis”, thus STRUCTURAL EQUATION MODELLING(SEM).
    If you could delve into this, it enhance the ANALYSIS FACTOR as an organization

    Reply
    • Karen says

      June 1, 2012 at 2:13 pm

      Hi Bashir,

      Thanks for the input. I agree, SEM is a great topic that would benefit a lot of people. We’ll see what we can do.

      Karen

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Please note that, due to the large number of comments submitted, any questions on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.

Primary Sidebar

This Month’s Statistically Speaking Live Training

  • February Member Training: Choosing the Best Statistical Analysis

Upcoming Workshops

  • Logistic Regression for Binary, Ordinal, and Multinomial Outcomes (May 2021)
  • Introduction to Generalized Linear Mixed Models (May 2021)

Read Our Book



Data Analysis with SPSS
(4th Edition)

by Stephen Sweet and
Karen Grace-Martin

Statistical Resources by Topic

  • Fundamental Statistics
  • Effect Size Statistics, Power, and Sample Size Calculations
  • Analysis of Variance and Covariance
  • Linear Regression
  • Complex Surveys & Sampling
  • Count Regression Models
  • Logistic Regression
  • Missing Data
  • Mixed and Multilevel Models
  • Principal Component Analysis and Factor Analysis
  • Structural Equation Modeling
  • Survival Analysis and Event History Analysis
  • Data Analysis Practice and Skills
  • R
  • SPSS
  • Stata

Copyright © 2008–2021 The Analysis Factor, LLC. All rights reserved.
877-272-8096   Contact Us

The Analysis Factor uses cookies to ensure that we give you the best experience of our website. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor.
Continue Privacy Policy
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled

Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.

Non-necessary

Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.

SAVE & ACCEPT