I’m sure I don’t need to explain to you all the problems that occur as a result of missing data. Anyone who has dealt with missing data—that means everyone who has ever worked with real data—knows about the loss of power and sample size, and the potential bias in your data that comes with listwise deletion.
Listwise deletion is the default method for dealing with missing data in most statistical software packages. It simply means excluding from the analysis any cases with data missing on any variables involved in the analysis.
A very simple, and in many ways appealing, method devised to overcome these problems is mean imputation. Once again, I’m sure you’ve heard of it–just plug in the mean for that variable for all the missing values. The nice part is the mean isn’t affected, and you don’t lose that case from the analysis.
And it’s so easy! SPSS even has a little button to click to just impute all those means.
But there are new problems.
While it’s true the mean doesn’t change, the relationships with other variables do. And that’s usually what you’re interested in, right? Well, now they’re biased.
And while the sample size remains at its full value, the standard error of that variable will be vastly underestimated–and this underestimation gets bigger the more missing data there are. Too-small standard errors lead to too-small p-values, so now you’re reporting results that should not be there.
There are other options. Multiple Imputation and Maximum Likelihood both solve these problems. But while Multiple Imputation is not available in all the major stats packages, it is very labor-intensive to do well. And Maximum Likelihood isn’t hard or labor intensive, but usually requires using structural equation modeling software, such as AMOS or MPlus.
The good news is there are other imputation techniques that are still quite simple, and don’t cause bias in some situations. And sometimes (although rarely) it really is okay to use mean imputation. When?
If your rate of missing data is very, very small, it honestly doesn’t matter what technique you use. I’m talking very, very, very small (2-3%).
There is another, better method for imputing single values, however, that is only slightly more difficult than mean imputation. It uses the E-M Algorithm, which stands for Expectation-Maximization. It is an iterative procedure in which it uses other variables to impute a value (Expectation), then checks whether that is the value most likely (Maximization). If not, it re-imputes a more likely value. This goes on until it reaches the most likely value.
EM imputations are better than mean imputations because they preserve the relationship with other variables, which is vital if you go on to use something like Factor Analysis or Linear Regression.
EM Imputations still underestimate standard error, however. Once again, this approach is only reasonable if the standard error of individual items is not vital, like in Factor Analysis, which doesn’t have p-values.
The heavy hitters like Multiple Imputation and Maximum Likelihood are still superior methods of dealing with missing data and are in most situations the only viable approach. But you need to fit the right tool to the size of the problem.
It may be true that backhoes are better at digging holes than trowels, but trowels are just right for digging small holes. It’s better to use a small tool like EM when it fits than to ignore the problem altogether.
EM Imputation is available in SAS, Stata, R, and SPSS Missing Values Analysis module.
{ 29 comments… read them below or add one }
Thank you for the post.
In a dataset with both continuos and categorical missings I can’t use EM.
Is a good strategy:
1)a logistic imputation of the categoricals;
2)Expect. Maxim. for the continuos.
3) Analysis (i.e. linear regression)
Thanks a lot!
Hi Gianfranco,
I can’t advise a strategy without digging in to all the details, but I can say that you cannot mix multiple imputation and EM. And generally speaking all imputation should be multiple.
Hi Karen,
Is it appropriate to use multiple imputation for entire outcomes (i.e. entire questionnaires). I’m only trying to produce descriptive stats for a feasibility trial so I have produced completer descriptive analyses (listwise deletion), however there is a large portion of participants with missing follow-up questionnaires. Therefore I wanted to use MI to impute these missing outcomes and compare the descriptive stats (and effect sizes) produced from an imputed data set to a unimputed data set. What do you think?
HI Charlotte,
Yes, it is. Read the John Graham article linked below–he talks about that exact situation.
But you might actually be better off with a mixed model since there is repeated measures. It will save a lot of time and be just as good. See: https://www.theanalysisfactor.com/missing-data-two-recommended-solutions/
Hi karen
Do you have R code for ” EM algorithm” ? I just wanna Impute missing data with EM .
Hi Mehdi,
Not easily accessible. I am pretty sure the Amelia package has it, if I’m remembering correctly.
Hi Karen, thanks for the valuable information about missing data. You mentioned
“If your rate of missing data is very, very small, it honestly doesn’t matter what technique you use. I’m talking very, very, very small (2-3%)”
Do you have a reference for that? It will really be usefull.
Regards
Hassan
Ooh, I did once. Perhaps it’s in John Graham’s very good article: http://www.stats.ox.ac.uk/~snijders/Graham2009.pdf
Thanks for providing the reference link!
Hi Karen,
unfortunately it is not in Graham (2009). Do you have any other suggestion in regard to a reference of”very, very, very small (2-3%)”.
that would be really handy. Thanks in advance.
Barbara
Hi Barbara,
Oh my, I just saw the reference last week, but I’m afraid I don’t remember.
Hi,
I want some datasets with missing data (I just can’t remove data by myself it has to be random) can you suggest some ? or can you suggest me a way to remove data by a program or a software ?
and also I need some data set which make non convex data sets, I need these for my experiment on EM and other algorithms.
Thanks
Hey Karen,
do you have a reliable reference for that 5% limit of missings for using EM? You would help me a lot! 🙂
Try: Annual Review of Psychology (Graham, 2009)
Hello, I don’t think you’re quite right about the EM underestimating parameters. In this process, the variance and covariance of that variable is also corrected … as explained in “The SAGE Handbook of Social Science Methodology”, by William Outhwaite and Stephen Turner.
Hi Lorelai,
It’s not that it underestimates the parameter values themselves, but the standard errors of actual model effects.
So yes, if you’re just estimating means and correlations, you’re fine. But if you want to impute data points and use those in a model, your software doesn’t know that those are estimates and not real data points. That’s where any parameters estimated from those imputed data have too-small standard errors. There is more uncertainty than the model is accounting for. That’s why we need multiple imputation if we’re going to use p-values.
Hi Karen,
Is it correct to say that once i clicked on “impute missing data” for a specific variable, that variable will have no missing data in the imputation dataset?
I would like to check what went wrong with my procedure.
I clicked on the Multiple Imputation –> Impute Missing data value in SPSS. All the tabs were left it as default.
After I clicked “OK” on impute missing data, I noted random missing data is still available in the dataset of imputation_1, imputation_2, imputation_3, imputation_4 and imputation_5.
Greatly appreciate if you could guide me through.
Thanks very much!
Hi Pei,
Hmm, that is indeed what should happen. I would have to troubleshoot it to figure out what is going wrong. It could be some default in your version of SPSS. I can’t think of one off the top of my head, though that’s often the cause.
Hi Karen
Would you tend to use “as many variables as possible” as predictors in EM imputation or only construct relevant ones? Why?
E.g. use socioeconmic status, IQ asf. as predictors for reading proficiency?
Thanks!
Hi Sebastian,
As a general rule, you want to use as many predictors that are helpful for prediction. So there may be a predictor that isn’t theoretically important, but is helpful with prediction (for whatever unknown reason). But you don’t want to throw in everything you have. Adding unhelpful predictors just raises standard errors.
What is the R function for the EM imputation?
Thanks
Kenny, I don’t use R (maybe an R user can jump in here), but I believe MICE can do it. I am pretty sure it does multiple imputation, and EM is generally one way of doing MI.
Just explore the “MI” package on R’s website
Or try the function “impute.mdr” from imputeMDR package
Hi Karen,
Just wondering what you would recommend to do with imputed EM values for ordinal scales. Do you leave the imputed values (with decimal places) or do you recode so that values lie within the original values (from 1.001 to 1.499 = 1 for example). The imputed values are needed for a CFA and multiple regression.
Thanks for your time,
Kirstine.
Hi Karen,
Not sure if you responded to Kirstine but I had the same question on imputed EM values for the ordinal scale..
Thanks in advance.
Marsha
Hi Marsha,
As a general rule, you don’t want to round off any imputations. Even if the imputed values look weird, you need to have variation in there, so don’t round them off.
What to do when imputed EM vales are zero or negative, or exceed the maximum number (e.g., -4, 0,and 8 and Likert scale is from 1 to 7)?
Continue …. In SPSS is impossible to make constaints regarding maximum and minimum values for EM so how it should be solved. And also – Since EM does not impute values for categorical values, such as gender, what to do with them?