R² is such a lovely statistic, isn’t it?  Unlike so many of the others, it makes sense–the percentage of variance in Y accounted for by a model.

I mean, you can actually understand that.  So can your grandmother.  And the clinical audience you’re writing the report for.

A big R² is always good and a small one is always bad, right?

Well, maybe.

I’ve seen a lot of people get upset about small R² values, or any small effect size, for that matter.  I recently heard a comment that no regression model with an R² smaller than .7 should even be interpreted.

Now, there may be a context in which that rule makes sense, but as a general rule, no.

Just because effect size is small doesn’t mean it’s bad, unworthy of being interpreted, or useless.  It’s just small.  Even small effect sizes can have scientific or clinical significance.  It depends on your field.

For example, in a dissertation I helped a client with many years ago, the research question was about whether religiosity predicts physical health.  (If you’ve been in any of my workshops, you’ll recognize this example–it’s a great data set.  The model used frequency of religious attendance as an indicator of religiosity, and included a few personal and demographic control variables, including gender, poverty status, and depression levels, and a few others.

The model R² was about .04, although the model was significant.

It’s easy to dismiss the model as being useless.  You’re only explaining 4% of the variation?  Why bother?

But think about this.  If you think about all of the things that might affect someone’s health, do you really expect religious attendance to be a major contributor?

Even though I’m not a health researcher, I can think of quite a few variables that I would expect to be much better predictors of health.  Things like age, disease history, stress levels, family history of disease, job conditions.

And putting all of them into the model would indeed give better predicted values.  If the only point of the model was prediction, my client’s model would do a pretty bad job. (Perhaps the 70% comment came from someone who only runs prediction models).

But it wasn’t.  The point was to see if there was a small, but reliable relationship.  And there was.

Do small effect sizes require larger samples to find significance?  Sure.  But this data set had over 5000 people.  Not a problem.

Many researchers turned to using effect sizes because evaluating effects using p-values alone can be misleading.  But effect sizes can be misleading too if you don’t think about what they mean within the research context.

Sometimes being able to easily improve an outcome by 4% is clinically or scientifically important.  Sometimes it’s not even close enough.   Sometimes it depends on how much time, effort, or money would be required to get a 4% improvement.

As much as we’d all love to have straight answers to what’s big enough, that’s not the job of any statistic.  You’ve got to think about it and interpret accordingly.

If you learned much about calculating power or sample sizes in your statistics classes, chances are, it was on something very, very simple, like a z-test.

But there are many design issues that affect power in a study that go way beyond a z-test.  Like:

• repeated measures
• clustering of individuals
• blocking
• including covariates in a model

Regular sample size software can accommodate some of these issues, but not all.  And there is just something wonderful about finding a tool that does just what you need it to.

Especially when it’s free.

Enter Optimal Design Plus Empirical Evidence software.

Optimal Design is software for power calculations on individual and group randomized trials.  It was developed by a group of statistical researchers, headed by Jessaca Spybrook at Western Michigan University.  It was funded by a grant from the William T. Grant Foundation, so it is available for download free of charge.  (Link below).

I found it recently when working with a client who is planning a study for the effectiveness of an educational intervention.  The design options in the software were exactly the design issues we needed to consider for developing the best design to maximize power while accounting for data collection limitations and the budget.

There are many design options for a randomized trial.  If randomized trial isn’t a term used in your field, this is the basic idea.

The outcome is measured at the individual level, and repeated measures on individuals are possible.  There is some sort of randomization to treatment groups, and this randomization can occur at the individual level or at a cluster level.  The point of the study is to compare the mean of the outcome in the treatment groups.

For example, educational studies are often conducted on students clustered within classrooms.  In a study comparing two teaching formats for effectiveness, usually an entire classroom of students is assigned to one format condition.  It’s just not possible to randomly assign individuals within classrooms.  So the clusters (classrooms) are randomly assigned to treatment, not the individual student.

So both Person-Randomized and Group-Randomized trials are possible, and the level of randomization affects power.

It is also possible to include a third level of grouping, if classrooms are nested within schools, and a fourth, if schools are blocked within districts.

The randomization to treatments and the measurement of covariates can occur at any level.

If this is starting to get overwhelming, it’s not as bad as it sounds.  The software comes with one of the best written statistical software manuals I’ve seen.

The manual explains in great detail, with excellent examples, what each design criteria means, so that you’ll be able to recognize it in your own design.

Here is an excerpt from a table in the manual that explains some of the available designs:

Beyond the very simplest, all of these designs will require using multilevel or mixed models to analyze the data, once you’ve got it.  And to run the prospective power analyses, you will have to have estimates for some of the design effects–an intraclass correlation for the clustering, blocking effects, correlations between covariates and the outcome variable–in addition to the usual standard deviation estimates that you need for any power analysis.

But it doesn’t cover the sample size estimates for any multilevel analysis.  The effect size it requires is a mean difference for a comparison of two treatments.

This works great if you really are doing this type of intervention study–it’s exactly what you need. And within that context, the design options are plentiful.

Like any specialized tool, it works very, very well for what it’s designed for, and not much else.  It’s also very easy to use and well documented.

You can download the Optimal Design Plus software and documentation here.

If you’d like to learn more about power and sample size estimates, take a look at our online workshop:  Calculating Power and Sample Size.  We’ll go over the logic, the info you need, where to get it, how to do the steps, and how to use power software to get good estimates. We’ll also go over what these estimates really tell you, and what they don’t.

Factor is confusing much in the same way as hierarchical and beta, because it too has different meanings in different contexts.  Factor might be a little worse, though, because its meanings are related.

In both meanings, a factor is a variable.  But a factor has a completely different meaning and implications for use in two different contexts.

Factor in Factor Analysis

In factor analysis, a factor is an latent (unmeasured) variable that expresses itself through its relationship with other measured variables.

Take for example a variable like leadership. We may want to measure a person’s or an organization’s leadership style, but this is the kind of construct that would be impossible to measure using a single variable. It’s just too abstract and multifaceted, although it does represent a single concept.

So instead, you may have to develop the scale with many items, each of which measures some more measurable part of leadership. The idea would be that there is an underlying unmeasurable factor, leadership, that causes people to respond in certain patterns on the many items on the scale.

The purpose of factor analysis is to analyze these patterns of response as a way of getting at this underlying factor. Factor analysis also allows you to use the weighted item responses to create what are called factor scores.  These represent a single score for each person on the factor.

Factor scores are nice because they allow you to use a single variable as a measure of the factor in the other analyses, rather than a set of items.

Factor as a Categorical Predictor Variable

Contrast that to the use of a factor in a linear model or a linear mixed model.  In this context, a factor is still a variable, but it refers to a categorical independent variable. So you may have heard of fixed factors and random factors. In both cases, those are referring to a categorical independent variable.

Like covariates, factors in a linear model can be either control variables or important independent variables. The model uses them the same way in either case. The only difference is how you are going to interpret the results.

This all gets especially tricky when the continuous factor scores from a factor analysis are used as predictors in a linear model.  Technically, since they are continuous, they wouldn’t be factors in the model, in the second definition. They would be covariates.

———————————————————————————-

Read other posts in the series on Confusing Statistical Terms.

Sign up for free updates from The Analysis Factor to get more clarity on what statistics mean and how to use them.

Generalized linear models, linear mixed models, generalized linear mixed models, marginal models, GEE models.  You’ve probably heard of more than one of them and you’ve probably also heard that each one is an extension of our old friend, the general linear model.

This is true, and they extend our old friend in different ways, particularly in regard to the measurement level of the dependent variable and the independence of the measurements.  So while the names are similar (and confusing), the distinctions are important.

It’s important to note here that I am glossing over many, many details in order to give you a basic overview of some important distinctions.  These are complicated models, but I hope this overview gives you a starting place from which to explore more.

General Linear Models

The general linear model has this basic form:

And has these assumptions (among others)

1. the residuals are independent of each other
2. the residuals are normally distributed
3. the relationship between Y and the model parameters is linear

So let’s see how some of the different model types extend this model in different ways.

Generalized linear models

Generalized linear models extend the last two assumptions. They generalize the possible distributions of the residuals to a family of distributions called the exponential family.  This family includes the normal as well as the binomial, Poisson, negative binomial, and gamma distributions, among others. You are probably familiar with common examples like logistic, Poisson, and probit models.

When you change the distribution of the residuals, it turns out that the relationship between Y and the model parameters is no longer linear. However, for each distribution in the exponential family, there exists at least one function of the mean of Y whose relationship with the model parameters is linear. This function is called the link function.

The link function you choose will depend on which distribution you are choosing for the outcome variable. For example, a binomial residual can use a probit or a logit link function. A Poisson residual uses a log link function.

Marginal Models

Marginal models are a type of linear model that accounts for repeated response measures on the same subject. They extend the general linear model by allowing and accounting for non-independence among the observations of a single subject.

They do this by estimating one or more parameters that capture the covariance among the residuals.  So rather than having a single constant variance and zero covariance for all residuals, observations from the same subject are allowed to have different variances and nonzero covariances. The pattern of variances and covariances is known as the covariance structure of the R matrix.

They still assume that observations from different subjects are independent, and linear marginal models still assume residuals are normally distributed.

GEE Models

Generalized estimating equation models are generalized linear marginal models.  That is, they combine the generalized linear model for a non-normal residual with the repeated measures of a marginal model. You would use these when you have repeated measures on each subject and need to run a logistic, multinomial, Poisson or other generalized linear regression model.

Linear Mixed Models

Like marginal models, linear mixed models account for non-independence among clustered observations, but they do it in a different way.

Instead of estimating nonzero correlations among residuals, linear mixed models account for the fact that clustered observations are similar by estimating the variance among cluster means and among observations within a cluster.  It literally partitions the variance in Y into cluster-level and observation-level parts.

Because of the way they account for variation among subjects, linear mixed models are much more flexible than marginal models.

For example, they can accommodate three levels of repeat or clustering, like repeated measurements on patients clustered within hospitals, and can be used to estimate more precise subject effects beyond variation among means.

They can accomplish these feats because they include parameters to measure the random effects of the clusters–by treating the variation among clusters as another sort of residual variation.  The Mixed in the name comes from the fact that they estimate both fixed and random effects.

Like all linear models, linear mixed models assume residuals are normally distributed and the relationship between Y and the model parameters is linear.

Generalized Linear Mixed Models

You probably know by now where this one is going.

Generalized Linear Mixed Models are mixed models in which the residuals follow a distribution from the same exponential family.  They require the same link functions as generalized linear modelsand at least one random effect.

Both generalized linear models and linear mixed models can be computationally intensive, especially as the number of random effects to be estimated goes beyond one or two.  Putting them together can be especially so.  I’ve run GLMMs that took hours to run on not very large data sets.  They require special care and should not be undertaken lightly.

How all the models are the same

I’ve focused on how these models differ, but they also have underlying similarities.

1. The structure is the same: they all are models of the relationship between a single response variable Y, and one or more predictor variables X.  The variation around the model is estimated in the residual.
2. They generally all use some form of maximum likelihood estimation.  Even OLS estimation, used in the general linear model, is a special case of maximum likelihood.
3. Fixed effects work the same in all these models.  The function of Y may differ, and the residual structure may differ, but the X variables work the same in every one of these models.  Dummy and effect coding, continuous predictors, interactions, quadratic terms have the same inherent meaning and can be used in any of these models.
4. The General Linear Model is a subset of each of these other models.  You could, if you really wanted to, run a GLM model in a software procedure designed for any of these other models by choosing the right options.  The reverse is not true.

 If you want to learn more about linear mixed models, check out the recording of my Random Intercept and Random Slope Models webinar. These two models are the basic building blocks of all mixed models.

Get it all here. It’s free.

You may have noticed conflicting advice about whether to leave insignificant effects in a model or take them out in order to simplify the model.

One effect of leaving in insignificant predictors is on p-values–they use up precious df in small samples. But if your sample isn’t small, the effect is negligible.

The bigger effect is  on interpretation, and really the above cases are about whether it aids interpretation to leave them in. Models do get so cluttered it’s hard to figure out what’s going on, and it makes sense to eliminate effects that aren’t serving a purpose, but even insignificant effects can have a purpose.

So these are three situations where there is a purpose in showing that specific predictors were not significant and to measure their coefficient anyway:

1. Expected control variables.  You need to show that you’ve controlled for them.

In many fields, there are control variables that everyone expects to see.

• Age in medical studies
• Race, income, education in sociological studies
• Socioeconomic status in education studies

The examples go on and on.

If you take these expected controls out, you will just get criticism for not including them.  And it may be interesting to show that in this sample and with these variables, these controls weren’t significant.

2. Predictors you have specific hypotheses about.

Another example is if the point of a model is to specifically test a predictor–you have a hypothesis about a predictor and it’s meaningful to show that it’s not significant. In that case, I would leave it in, even if not significant.

3. Items involved in higher-order terms

When you take out a term that is involved in something higher, like a two-way interaction that is part of a three-way interaction, you actually change the meaning of the higher order term.  The sums of squares for each higher-order term is based on comparisons to specific means and represents variation around that mean.

If you take out the lower order term, that variation has to be covered somewhere, and it’s usually not where you expect it.  For example, a two-way interaction represents the variation in cell means around the main effect means.  But if the variation between the main effect means isn’t measured with a main effect term, it ends up in the interaction, and that interaction doesn’t reflect the variation it did if the main effect were in the model.

So it’s not that it’s wrong, but it changes the meaning of the interaction.  For that reason, most people recommend leaving those lower-order effects in.

The main point here is there are often good reasons to leave insignificant effects in a model. The p-values are just one piece of information. You may be losing important information by automatically removing everything that isn’t significant.

Data Mining methods lie at the center of the constellation of techniques under the umbrella of “business analytics.”  These techniques deal with analysis of large existing datasets (as opposed to controlled experiments, or sample surveys).

This webinar will give an overview of data mining techniques, which include:

• In predictive modeling, we build a model to predict the known value of a variable of interest using “training” data, then apply the model to data where the value is unknown.
• In clustering, we seek to identify groups of similar customers, records, etc. Clustering is the statistical component in customer segmentation.
• A recommender system identifies, statistically, “what goes with what.”  These systems lie behind the notices advising you that “customers who bought X also bought Y.”
• Additional methods include graphical techniques and text analytics (the most rapid growth is in text – Twitter feeds, Facebook contents, emails, etc.).

Date: May 30, 2012

Time: 1pm Eastern Time UTC -4 (12pm Central, 11am Mountain, 10am Pacific)

Where: Anywhere you have a fast internet connection

Length of Program: An Hour

Cost: Always FREE

Space is limited.

About Our Guest

Mr. Peter Bruce is President of The Institute for Statistics Education at Statistics.com. He is the developer of Resampling Stats software (originated by Julian Simon in the 1970′s), and has taught resampling statistics at the University of Maryland and in a variety of short courses. He is the co-author of Data Mining for Business Intelligence (Wiley, 2006, 2nd ed. 2010), as well as a number of journal articles.

It’s really easy to mix up the concepts of association (a.k.a. correlation) and interaction.  Or to assume if two variables interact, they must be associated.  But it’s not actually true.

In statistics, they have different implications for the relationships among your variables, especially when the variables you’re talking about are predictors in a regression or ANOVA model.

Association

Association between two variables means the values of one variable relate in some way to the values of the other.  Association is usually measured by correlation for two continuous variables and by cross tabulation and a Chi-square test for two categorical variables.

Unfortunately, there is no nice, descriptive measure for association between one categorical and one continuous variable, but either one-way analysis of variance or logistic regression can test an association (depending upon whether you think of the categorical variable as the independent or the dependent variable).

Essentially, association means the values of one variable generally co-occur with certain values of the other.

Interaction

Interaction is different.  Whether two variables are associated says nothing about whether they interact in their effect on a third variable.  Likewise, if two variables interact, they may or may not be associated.

An interaction between two variables means the effect of one of those variables on a third variable is not constant—the effect differs at different values of the other.

What Association and Interaction Describe in a Model

The following examples show three situations for three variables: X1, X2, and Y. X1 is a continuous independent variable, X2 is a categorical independent variable, and Y is the dependent variable.  I chose these types of variables to make the plots easy to read, but any of these variables could be either categorical or continuous.

In scenario 1, X1 and X2 are associated.  If you ignore Y, you can see the mean of X1 is lower when X2=0 than when X2=1.  But they do not interact in how they affect Y—the regression lines are parallel.  X1 has the same effect on Y (the slope) for both X2=1 and X2=0.

A simple example is the relationship between height (X1) and weight (Y) in male (X2=0) and female (X2=1) teenagers.  There is a relationship between height (X1) and gender (X2), but for both genders, the relationship between height and weight is the same.

This is the situation you’re trying to take care of by including control variables.  If you didn’t include gender as a control, a regression would fit a single line to all these points and attribute all variation in weights to differences in heights.  This line would also be steeper, as it tried to fit all the points using one line, and it would overestimate the size of the unique effect of height on weight.

In a second scenario, X1 and X2 are not associated—the mean of X1 is the same for both categories of X2.  But how X1 affects Y differs for the two values of X2—the definition of an interaction.  The slope of X1 on Y is greater for X2=1 than it is for X2=0, in which it is nearly flat.

An example of this would be an experiment in which X1 was a pretest score and Y a posttest score.  Imagine participants were randomly assigned to a control (X2=1) or a training (X2=0) condition.

If randomization is done well, the assigned condition (X2) should be unrelated to the pretest score (X1).  But they do interact—the relationship between pretest and posttest differs in the two conditions.

In the control condition, without training, the pretest and posttest scores would be highly correlated, but in the training condition, if the training worked well, pretest scores would have less effect on posttest scores.

In the third scenario, we’ve got both an association and an interaction.   X1 and X2 are associated—once again the mean of X1 is lower when X2=0 than when X2=1.  They also interact with Y—the slopes of the relationship between X1 and Y are different when X2=0 and X2=1.  So X2 affects the relationship between X1 and Y.

A good example here would be if Y is the number of jobs in a county, X1 is the percentage of the workforce that holds a college degree, and X2 is whether the county is rural (X2=0) or metropolitan (X1=0).

It’s clear rural counties have, on average, lower percentages of college-educated citizens than metropolitan counties.  They also have fewer jobs.

It’s also clear that the workforce’s education level in metropolitan counties is related to how many jobs there are.  But in rural counties, it doesn’t matter at all.

This situation is also what you would see if the randomization in the last example did not go well or if randomization was not possible.

The differences between Interaction and Association will become clearer as you analyze more data. It’s always a good idea to stop and explore your data through graphs or by trying different terms in your model to figure out exactly what’s going on with your variables.

If you want more information on using and interpreting interactions, get the recording from my webinar: Interpreting Regression Coefficients: A Walk Through Output. It’s free.

In the world of data analysis, there’s not always one clearly appropriate analysis for every research question. There are so many factors to take into account, including the research question to be answered, the measurement of the variables, the study design, data limitations and issues, the audience, practical constraints like software availability, and the purpose of the data analysis.

So what do you do when a reviewer rejects your choice of data analysis? This reviewer can be your boss, your dissertation committee, a co-author, or journal reviewer or editor. What do you do?

There are ultimately only two choices: You can redo the analysis their way. Or you can fight for your analysis. How do you choose?

The one absolute in this choice is that you have to honor the integrity of your data analysis and yourself. Do not be persuaded to do an analysis that will produce inaccurate or misleading results, especially when readers will actually make decisions based on these results. (If no one will ever read your report, this is less crucial).

But even within that absolute, there are often choices. Keep in mind the two goals in data analysis:

1. The analysis needs to accurately reflect the limits of the design and the data, while still answering the research question.
2. The analysis needs to communicate the results to the audience.

So first and foremost, if your reviewer is asking you to do an analysis that does not appropriately take into account the design or the variables, you need to fight.

For example, a few years ago I worked with a researcher who had a study with repeated measurements on the same individuals. It had a small sample size and an unequal number of observations on each individual. It was clear that to take into account the design and the unbalanced data, the appropriate analysis was a linear mixed model.

The researcher’s co-author questioned the use of the linear mixed model, mainly because he wasn’t familiar with it, and thought the researcher was attempting something fishy. His suggestion was to use an ad hoc technique of averaging over the multiple observations for each subject.

This was a situation where fighting was worth it. Unnecessarily simplifying the analysis to please people who were unfamiliar with an appropriate method was not an option because the simpler model would have violated assumptions. This was particularly important because the research was being submitted to a high-level journal.

So it was the researcher’s job to educate not only his coauthor, but the readers, in the form of explaining the analysis and its advantages, with citations, right in the paper.

In contrast, often the reviewer is not really asking for a completely different analysis, but a different way of running the same analysis, or reporting different specific statistics.

For example a confirmatory factor analysis can be run either in standard statistical software like SAS, SPSS, or Stata, or it can be run it in structural equation modeling software like Amos or MPlus. The analysis is essentially the same, but the two types of software will report different statistics.

If your committee members are familiar with structural equation modeling, they probably want to see the type of statistics that structural equation modeling software will report, including overall model fit statistics like RMSEA or model chi-squares.

This is a situation where it may be easier, and produces no ill-effects, to jump through the hoop. Running the factor analysis in the software they prefer, assuming you have access to the software, won’t violate any assumptions or produce inaccurate results. Especially if the reviewer is someone who has the ability to stop your research in its tracks, it may be worth it to rerun the analysis to get the statistics they want to see reported.

Now you do have to decide whether the cost of jumping through the hoop, in terms of time, money, and emotional energy, is worth it. If the request is relatively minor, it usually is. If it’s a matter of rerunning every analysis you’ve done to indulge a committee member’s pickiness, it may be worth standing up for yourself and your analysis.

When you’re dealing with anonymous reviewers, the situation can get sticky.  You cannot ask them to clarify their concerns and you have limited ability to explain the reasons for choosing your analysis. It may be harder to discern if they are being overly picky, don’t understand the statistics themselves, or have a valid point.

If you choose to stand up for yourself, be well armed. Research the issue until you are absolutely confident in your approach (or until you’re convinced that you were missing something). A few hours in the library or talking with a trusted expert is never a wasted investment. Compare that to running an unpublishable analysis to please a committee member or coauthor.

Often, the problem is actually not in the analysis you did, but in the way you explained it. It’s your job to explain why the analysis is appropriate and, if it’s unfamiliar to readers, what it does. Rewrite that section, making it very clear. Ask colleagues to review it. Cite other research that uses or explains that statistical method.

Whatever you choose, be confident that you made the right decision, then move on.

Odds ratios are the bane of many data analysts. Interpreting them can be like learning a whole new language. This webinar will go over an example to show how to interpret the odds ratios in binary logistic regression. You will learn:

• how probability and odds both measure the same thing on different scales
• the meaning of odds
• how to interpret an odds ratio for continuous and categorical predictors in logistic regression

[webinar-recording-optin]

Last month I did a webinar on Poisson and negative binomial models for count data. With a few hundred participants, we ran out of time to get through all the questions, so I’m answering some of them here on the blog.

This set of questions are all related to when it’s appropriate to treat count data as continuous and run the more familiar and simpler linear model.

Q: Do you have any guidelines or rules of thumb as far as how many discrete values an outcome variable can take on before it makes more sense to just treat it as continuous?

The issue usually isn’t a matter of how many values there are.  I see what you mean in that a discrete scale that goes from 0 to 8 for example feels more discrete because there are only nine possible values, compared to a discrete scale that goes from 0 to 200.  201 values just feels more continuous.

But that’s not really the issue in most count models. The issue with count variables is that they bounded at zero. This wreaks havoc on the assumptions of a linear model, which require continuous data.

If none of your data are near zero, it would be less of an issue.  Treating that count variable as continuous would give you predicted values that are non-integers, but perhaps that’s not a big issue in your particular data set.

Q: How high does the count scale have to be before you can consider it continuous?

I suspect you’re getting at the same issue as in the last question. It’s certainly true that when you get into very large numbers, many of the issues with count variables aren’t issues anymore.

For example, most incomes are not measured using decimals, just whole numbers. You could consider them a count of the number of dollars. Likewise, demographic variables like the number of children vaccinated in a state over the course of the year are truly counts, but the smallest values are likely to be in the hundreds of thousands, or even millions.

As long as there are no data along the bound of zero, and you don’t mind predicted values that include decimals, there’s no problem treating it as continuous.

Q: For count data distributing not skewed, but in a symmetric/or even normal shape, are poisson and NB still the best choice?

Sometimes.  The Poisson distribution is only skewed when the mean is very small. When the mean gets up to only 10, the distribution will become symmetric and bell shaped.

Depending on the effects of the predictors, and actual range of the data, i.e. whether there are actual 0 pounds or not, you may get identical results from running a linear model compare to a Poisson or negative binomial model.

If you do run a linear model, it will be possible to get predictive values below zero, and you need to consider whether that’s problematic in your situation. If the point of your model is prediction, it may be more of an issue.

Q: If count data can be normalized by log transformation, will you recommend using poisson or linear regression?

It’s never wrong to run a Poisson model, so what you’re asking is if the increased accuracy is worth the trouble of running the more complicated model.  There are certainly cases where running a linear model simplifies things a lot and still gives you the same results.  (You just won’t know you have the same results unless you run both).

When the mean count is very small, and zero is the most common value in the data set, it will be impossible to normalize using a log transformation.  It just won’t work. The mode will always be at the lowest value.  In that situation, you have no choice.

However, if the mean count is a little bit larger, zero may not be the most common value. When the mode is not the lower bound, it will be possible to use a log transformation to normalize the data. In fact, not too many years ago, when Poisson and negative binomial models were not readily available in software, textbooks did suggest this approach.  You may still have some on your shelf that do so.

It’s not necessarily a bad approach.  You may very well get the exact same results. If so, and if, for example, you are writing a report for an audience with out the statistical sophistication to understand the Poisson model, it may be a better choice.

However, it’s not exactly the same thing. A Poisson model uses a log link function, which applies the log to the mean–not each individual data point. So it’s harder to back transform coefficients when you’ve got a log transformation.  So if there are not major advantages to running a linear model, you are usually better off with a more sophisticated and more accurate Poisson model, or one of its derivatives.

If you’d like to learn more about the different models available for Count data, you can download a recording of the webinar: Poisson and Negative Binomial Regression for Count Data. It’s free.