I often hear concern about the non-normal distributions of independent variables in regression models, and I am here to ease your mind.
There are NO assumptions in any linear model about the distribution of the independent variables. Yes, you only get meaningful parameter estimates from nominal (unordered categories) or numerical (continuous or discrete) independent variables. But no, the model makes no assumptions about them. They do not need to be normally distributed or continuous.
It is useful, however, to understand the distribution of predictor variables to find influential outliers or concentrated values. A highly skewed independent variable may be made more symmetric with a transformation.
Multicollinearity occurs when two or more predictor variables in a regression model are redundant. It is a real problem, and it can do terrible things to your results. However, the dangers of multicollinearity seem to have been so drummed into students’ minds that it created a panic.
True multicolllinearity (the kind that messes things up) is pretty uncommon. High correlations among predictor variables may indicate multicollinearity, but it is NOT a reliable indicator that it exists. It does not necessarily indicate a problem. How high is too high depends on (more…)
A polynomial term–a quadratic (squared) or cubic (cubed) term turns a linear regression model into a curve. But because it is X that is squared or cubed, not the Beta coefficient, it still qualifies as a linear model. This makes it a nice, straightforward way to model curves without having to model complicated non-linear models.
But how do you know if you need one–when a linear model isn’t the best model? (more…)
I just heard recently about PSPP, which is a free, open source version of SPSS.
I have not tried it yet, but it does look promising. This is the description from its website:
It is a Free replacement for the proprietary program SPSS, and appears very similar to it with a few exceptions.
The most important of these exceptions are, that there are no “time bombs”; your copy of PSPP will not “expire” or deliberately stop working in the future. Neither are there any artificial limits on the number of cases or variables which you can use. There are no additional packages to purchase in order to get “advanced” functions; all functionality that PSPP currently supports is in the core package.
PSPP can perform descriptive statistics, T-tests, linear regression and non-parametric tests. Its backend is designed to perform its analyses as fast as possible, regardless of the size of the input data. You can use PSPP with its graphical interface or the more traditional syntax commands.
Sounds pretty good, huh?
The only downside I can see, though, is with the statement “no additional packages to purchase in order to get ‘advanced’ functions.” That appears to be because there aren’t any advanced functions. PSPP seems to correspond only to SPSS base. No Advanced Models, no Missing Values Analysis, no Complex Surveys. That means you can do one-way ANOVA and regression, but not GLM, logisitic regression, factor analysis.
So if you are only using SPSS for basic statistics, or for teaching an intro class, this may be just what you need. And perhaps if it takes off, as R has, we’ll see more advanced features soon.
If you’ve had any experience using PSPP, please tell me about it in a comment. I’d love to hear how well it works.
In addition to the five listed in this title, there are quite a few other options, so how do you choose which statistical software to use?
The default is to use whatever software they used in your statistics class–at least you know the basics.
And this might turn out pretty well, but chances are it will fail you at some point. Many times the stat package used in a class is chosen for its shallow learning curve, (more…)
If your graduate statistical training was anything like mine, you learned ANOVA in one class and Linear Regression in another. My professors would often say things like “ANOVA is just a special case of Regression,” but give vague answers when pressed.
It was not until I started consulting that I realized how closely related ANOVA and regression are. They’re not only related, they’re the same thing. Not a quarter and a nickel–different sides of the same coin.
So here is a very simple example that shows why. When someone showed me this, a light bulb went on, even though I already knew both ANOVA and multiple linear (more…)
