EM Imputation and Missing Data: Is Mean Imputation Really so Terrible?

I’m sure I don’t need to explain to you all the problems that occur as a result of missing data.  Anyone who has dealt with missing data—that means everyone who has ever worked with real data—knows about the loss of power and sample size, and the potential bias in your data that comes with listwise deletion.

stage-3

Listwise deletion is the default method for dealing with missing data in most statistical software packages.  It simply means excluding from the analysis any cases with data missing on any variables involved in the analysis.

A very simple, and in many ways appealing, method devised to overcome these problems is mean imputation. Once again, I’m sure you’ve heard of it–just plug in the mean for that variable for all the missing values. The nice part is the mean isn’t affected, and you don’t lose that case from the analysis.

And it’s so easy! SPSS even has a little button to click to just impute all those means.

But there are new problems.

While it’s true the mean doesn’t change, the relationships with other variables do. And that’s usually what you’re interested in, right? Well, now they’re biased.

And while the sample size remains at its full value, the standard error of that variable will be vastly underestimated–and this underestimation gets bigger the more missing data there are. Too-small standard errors lead to too-small p-values, so now you’re reporting results that should not be there.

There are other options. Multiple Imputation and Maximum Likelihood both solve these problems. But while Multiple Imputation is not available in all the major stats packages, it is very labor-intensive to do well. And Maximum Likelihood isn’t hard or labor intensive, but usually requires using structural equation modeling software, such as AMOS or MPlus.

The good news is there are other imputation techniques that are still quite simple, and don’t cause bias in some situations. And sometimes (although rarely) it really is okay to use mean imputation. When?

If your rate of missing data is very, very small, it honestly doesn’t matter what technique you use. I’m talking very, very, very small (2-3%).

There is another, better method for imputing single values, however, that is only slightly more difficult than mean imputation. It uses the E-M Algorithm, which stands for Expectation-Maximization. It is an iterative procedure in which it uses other variables to impute a value (Expectation), then checks whether that is the value most likely (Maximization). If not, it re-imputes a more likely value. This goes on until it reaches the most likely value.

EM imputations are better than mean imputations because they preserve the relationship with other variables, which is vital if you go on to use something like Factor Analysis or Linear Regression.

EM Imputations still underestimate standard error, however. Once again, this approach is only reasonable if the standard error of individual items is not vital, like in Factor Analysis, which doesn’t have p-values.

The heavy hitters like Multiple Imputation and Maximum Likelihood are still superior methods of dealing with missing data and are in most situations the only viable approach. But you need to fit the right tool to the size of the problem.

It may be true that backhoes are better at digging holes than trowels, but trowels are just right for digging small holes. It’s better to use a small tool like EM when it fits than to ignore the problem altogether.

EM Imputation is available in SAS, Stata, R, and SPSS Missing Values Analysis module.

 

Approaches to Missing Data: the Good, the Bad, and the Unthinkable
Learn the different methods for dealing with missing data and how they work in different missing data situations.

Reader Interactions

Comments

  1. Gianfranco says

    Thank you for the post.
    In a dataset with both continuos and categorical missings I can’t use EM.
    Is a good strategy:
    1)a logistic imputation of the categoricals;
    2)Expect. Maxim. for the continuos.
    3) Analysis (i.e. linear regression)

    Thanks a lot!

    • Karen Grace-Martin says

      Hi Gianfranco,

      I can’t advise a strategy without digging in to all the details, but I can say that you cannot mix multiple imputation and EM. And generally speaking all imputation should be multiple.

  2. Charlotte Paterson says

    Hi Karen,
    Is it appropriate to use multiple imputation for entire outcomes (i.e. entire questionnaires). I’m only trying to produce descriptive stats for a feasibility trial so I have produced completer descriptive analyses (listwise deletion), however there is a large portion of participants with missing follow-up questionnaires. Therefore I wanted to use MI to impute these missing outcomes and compare the descriptive stats (and effect sizes) produced from an imputed data set to a unimputed data set. What do you think?

  3. Hassan Qutub says

    Hi Karen, thanks for the valuable information about missing data. You mentioned
    “If your rate of missing data is very, very small, it honestly doesn’t matter what technique you use. I’m talking very, very, very small (2-3%)”
    Do you have a reference for that? It will really be usefull.

    Regards

    Hassan

  4. Sushant says

    Hi,
    I want some datasets with missing data (I just can’t remove data by myself it has to be random) can you suggest some ? or can you suggest me a way to remove data by a program or a software ?
    and also I need some data set which make non convex data sets, I need these for my experiment on EM and other algorithms.
    Thanks

  5. Bianca says

    Hey Karen,
    do you have a reliable reference for that 5% limit of missings for using EM? You would help me a lot! 🙂

  6. Lorelai says

    Hello, I don’t think you’re quite right about the EM underestimating parameters. In this process, the variance and covariance of that variable is also corrected … as explained in “The SAGE Handbook of Social Science Methodology”, by William Outhwaite and Stephen Turner.

    • Karen Grace-Martin says

      Hi Lorelai,

      It’s not that it underestimates the parameter values themselves, but the standard errors of actual model effects.

      So yes, if you’re just estimating means and correlations, you’re fine. But if you want to impute data points and use those in a model, your software doesn’t know that those are estimates and not real data points. That’s where any parameters estimated from those imputed data have too-small standard errors. There is more uncertainty than the model is accounting for. That’s why we need multiple imputation if we’re going to use p-values.

  7. pei ting says

    Hi Karen,

    Is it correct to say that once i clicked on “impute missing data” for a specific variable, that variable will have no missing data in the imputation dataset?

    I would like to check what went wrong with my procedure.

    I clicked on the Multiple Imputation –> Impute Missing data value in SPSS. All the tabs were left it as default.

    After I clicked “OK” on impute missing data, I noted random missing data is still available in the dataset of imputation_1, imputation_2, imputation_3, imputation_4 and imputation_5.

    Greatly appreciate if you could guide me through.

    Thanks very much!

    • Karen says

      Hi Pei,

      Hmm, that is indeed what should happen. I would have to troubleshoot it to figure out what is going wrong. It could be some default in your version of SPSS. I can’t think of one off the top of my head, though that’s often the cause.

  8. Sebastian says

    Hi Karen
    Would you tend to use “as many variables as possible” as predictors in EM imputation or only construct relevant ones? Why?
    E.g. use socioeconmic status, IQ asf. as predictors for reading proficiency?

    Thanks!

    • Karen says

      Hi Sebastian,

      As a general rule, you want to use as many predictors that are helpful for prediction. So there may be a predictor that isn’t theoretically important, but is helpful with prediction (for whatever unknown reason). But you don’t want to throw in everything you have. Adding unhelpful predictors just raises standard errors.

  9. Kirstine says

    Hi Karen,

    Just wondering what you would recommend to do with imputed EM values for ordinal scales. Do you leave the imputed values (with decimal places) or do you recode so that values lie within the original values (from 1.001 to 1.499 = 1 for example). The imputed values are needed for a CFA and multiple regression.

    Thanks for your time,

    Kirstine.

    • Marsha says

      Hi Karen,
      Not sure if you responded to Kirstine but I had the same question on imputed EM values for the ordinal scale..
      Thanks in advance.
      Marsha

      • Karen says

        Hi Marsha,

        As a general rule, you don’t want to round off any imputations. Even if the imputed values look weird, you need to have variation in there, so don’t round them off.

        • Mirjana says

          What to do when imputed EM vales are zero or negative, or exceed the maximum number (e.g., -4, 0,and 8 and Likert scale is from 1 to 7)?

          • Mirjana says

            Continue …. In SPSS is impossible to make constaints regarding maximum and minimum values for EM so how it should be solved. And also – Since EM does not impute values for categorical values, such as gender, what to do with them?

      • Henrique says

        Hello!
        I have the same doubt as Kristine and Marsha.
        How to input missing values for ordinal variables?
        EM function in SPSS is only available for continuous variables!


Leave a Reply

Your email address will not be published. Required fields are marked *

Please note that, due to the large number of comments submitted, any questions on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.