What Is an Exact Test?

March 26th, 2018 by

Most of the p-values we calculate are based on an assumption that our test statistic meets some distribution. These distributions are generally a good way to calculate p-values as long as assumptions are met.

But it’s not the only way to calculate a p-value.

Rather than come up with a theoretical probability based on a distribution, exact tests calculate a p-value empirically.

The simplest (and most common) exact test is a Fisher’s exact for a 2×2 table.

Remember calculating empirical probabilities from your intro stats course? All those red and white balls in urns? (more…)