When Unequal Sample Sizes Are and Are NOT a Problem in ANOVA

by Karen

In your statistics class, your professor made a big deal about unequal sample sizes in one-way Analysis of Variance (ANOVA) for two reasons.

1. Because she was making you calculate everything by hand.  Sums of squares require a different formula if sample sizes are unequal, but SPSS (and other statistical software) will automatically use the right formula.

2. Nice properties in ANOVA such as the Grand Mean being the intercept in an effect-coded regression model don’t hold when data are unbalanced.  Instead of the grand mean, you need to use a weighted mean.  That’s not a big deal if you’re aware of it.

The only practical issue in one-way ANOVA is that very unequal sample sizes can affect the homogeneity of variance assumption.  ANOVA is considered robust to moderate departures from this assumption, but the departure needs to stay smaller when the sample sizes are very different.  According to Keppel (1993), there isn’t a good rule of thumb for the point at which unequal sample sizes make heterogeneity of variance a problem.

Real issues with unequal sample sizes do occur in factorial ANOVA, if the sample sizes are confounded in the two (or more) factors.  For example, in a two-way ANOVA, let’s say that your two independent variables (factors) are age (young vs. old) and marital status (married vs. not).  If there are twice as many young people as old and the young group has a much larger percentage of singles than the older group, the effect of marital status cannot be distinguished from the effect of age.

Power is based on the smallest sample size, so while it doesn’t hurt power to have more observations in the larger group, it doesn’t help either.

tn_ircNeed some statistical training that fits your busy schedule? Take a look at our On Demand Online Workshops. Our On Demand workshops are available 24/7 from wherever you have a computer and an Internet connection.

Send to Kindle

{ 192 comments… read them below or add one }

Olivia March 22, 2014 at 11:37 pm

Hi I was wondering what the full reference is for Keppel (1993). I’m interested in looking at that paper. Thanks

Reply

Karen April 4, 2014 at 9:40 am

Hi Olivia,

It’s a book, not a paper. “Design and Analysis: A Researcher’s Handbook.”

Reply

Josh April 6, 2014 at 12:51 am

Hi Karen,

I’m doing an analysis on mechanical properties with one factor. I have 3 groups, group 1 (n=5), group 2 (n=9) and group 3 (n=8). I have read the comment people asked and the replied you have given. So am I right to say that for one way ANOVA, is alright to analysis different sample size per group.

Reply

Karen April 7, 2014 at 5:00 pm

Yes.

Reply

TW April 11, 2014 at 10:00 am

Hi, I found your website very helpful and have a few of questions:

1) I have data of a entire population and am comparing the means of three groups. Do I still need to do significance testing since this isn’t really sampling?

2) The 3 groups have very different size (1200, 12000, 40000). I found the data not normal, so can I just use Kruskall-Wallis test?

3) I understand ANOVA is popular but I never found any data set that is normal. i.e. shapiro wilks test or kolmogrv test always have sig. <0 so I kept on using Kruskall walli test. Is that ok?

Many thanks!

Reply

TJ April 13, 2014 at 5:46 pm

Hi Karen,

I am working with a data set that has n~200, n~13, n~20. I would like to do an ANOVA but I am not sure how to approach this. What was the sample 20/200 you mentioned? Would a weighted mean account for these differences? The number of samples is also related to the number of interesting components for that group (not due to poor sampling).

Reply

Somayyeh April 22, 2014 at 6:44 am

Dear Karen

Thanks for the information that you provided here. I have the same issue. I have a caregiver group of 96 and 42 control participants that I compare them on one variables. I checked for the variance and there were no significant differences in the variance. so I guess that refer to that. However, do you know any published book that I can cite?

thanks

Reply

trish April 23, 2014 at 10:58 am

Hi,

I have data of 2 years 2004 and 2008 and I realized the sample size is not equal for both of the year how can i do data cleaning in stata in this case..

Reply

Muluken Tigistu April 24, 2014 at 12:13 pm

in my paper i am comparing the psychological well being of orphan and non-orphan children.sample size of n1=166,n2=333.is there a problem in computing independent t-test?

Reply

Kalyani April 30, 2014 at 2:53 pm

Hi Karen,

I hope you can help me. I’m trying to finish a paper for this term. I’ve just run two ANCOVAs. There were no problems with outliers, some problems with normality (skew and kurtosis <|3| although formal tests were significant), no problems with collinearity, correlation between covariates or homogeneity of slopes. Levene's test was significant for both ANCOVAS. The cell sizes and SDs are as follows:

ANCOVA with DV "A"
N=30, SD=1.31
N=78, SD=1.16
N=55, SD=.88
N=171, SD =1.21

ANCOVA with DV "B"
N=30, SD=0.91
N=78, SD=0.89
N=55, SD=.74
N=171, SD =.72

I realize that the smallest group has the highest variance in both cases. I hate to transform variables since it makes interpretation so complicated. What other options do I have?

Help!

-Kalyani

Reply

Yang May 1, 2014 at 2:08 pm

Hi Karen,

My experiment model have two factors – temperature and different time points. I performed 2-Way GLM for the unequal sample size I have. However, it seems that there is no effect from the interaction of two factors and the temperature itself. My question is that will the result of comparison between two temp at different time points be valid if I perform them using one-way GLM after the no significant finding in the initial 2-way GLM?

Thank,

Reply

Vicki May 2, 2014 at 5:55 am

Hi Karen,

I’m looking at the spatial variation of fish parasites for my Bachelor of Science thesis. I want to compare mean parasite abundance between male (n=71) and female (n=105) fish. I log transformed the parasite data and it has a normal distribution and equal variances, I was just wondering if I can use a One-way ANOVA to compare the mean abundance between sexes or would it be safer to just apply a non parametric Mann-Whitney U or Kruskal Wallis Test. Hope to hear from you.

Reply

Mon May 16, 2014 at 9:50 am

Hi, I’m doing study for me Bachelor of Science thesis too. Currently, I’m having problem with data analysis. My experiment design is 3×3 factorial design which consists of two independent variables (frying temperature and frying duration). However, for the duration factor is abit special which it has different duration). The setting is fried at 140C for 4, 5, 6 minutes while 160 and 180C fried for 1,2 and 3mins). Shall I use one way ANOVA or two way to analyse the effect on my sample?

thanks

Reply

manisha June 10, 2014 at 3:50 pm

Hello Sir,
Sir in my research study, i had done work in three groups Group A( n=50), Group B( n=50) and Group C(n=25), i have used one way anova. is there any problem for selection of uneven sample size of Group C or it may affect statistical analysis. Please sir advice mee.
Thanking u

Reply

Helena August 14, 2014 at 3:15 am

Hi Karen,

I have a question related to unequal sample sizes. I have a 2 (language background first language speaker (L1)/second language (L2) speaker) x3 (visual status: early blind/late blind/sighted) design. I investigate whether it is an advantage to have become blind as a child when it comes to second language acquisition.

In total I have N80: 40 L1 speakers and 40 L2 speakers (equal sizes), and each of these two groups have 11 early blind, 9 late blind and 20 sighted participants. Are these unequal sample sizes related to visual status a problem when using a 2×3 Anova? What do you suggest?

Many many thanks!
Helena

Reply

Leave a Comment

Previous post:

Next post: