# Generalized Linear Models (GLMs) in R, Part 4: Options, Link Functions, and Interpretation

Last year I wrote several articles (GLM in R 1, GLM in R 2, GLM in R 3) that provided an introduction to Generalized Linear Models (GLMs) in R.

As a reminder, Generalized Linear Models are an extension of linear regression models that allow the dependent variable to be non-normal.

In our example for this week we fit a GLM to a set of education-related data.

Let’s read in a data set from an experiment consisting of numeracy test scores (numeracy), scores on an anxiety test (anxiety), and a binary outcome variable (success) that records whether or not the students eventually succeeded in gaining admission to a prestigious university through an admissions test.

We will use the glm() command to run a logistic regression, regressing success on the numeracy and anxiety scores.

```
A <- structure(list(numeracy = c(6.6, 7.1, 7.3, 7.5, 7.9, 7.9, 8,
8.2, 8.3, 8.3, 8.4, 8.4, 8.6, 8.7, 8.8, 8.8, 9.1, 9.1, 9.1, 9.3,
9.5, 9.8, 10.1, 10.5, 10.6, 10.6, 10.6, 10.7, 10.8, 11, 11.1,
11.2, 11.3, 12, 12.3, 12.4, 12.8, 12.8, 12.9, 13.4, 13.5, 13.6,
13.8, 14.2, 14.3, 14.5, 14.6, 15, 15.1, 15.7), anxiety = c(13.8,
14.6, 17.4, 14.9, 13.4, 13.5, 13.8, 16.6, 13.5, 15.7, 13.6, 14,
16.1, 10.5, 16.9, 17.4, 13.9, 15.8, 16.4, 14.7, 15, 13.3, 10.9,
12.4, 12.9, 16.6, 16.9, 15.4, 13.1, 17.3, 13.1, 14, 17.7, 10.6,
14.7, 10.1, 11.6, 14.2, 12.1, 13.9, 11.4, 15.1, 13, 11.3, 11.4,
10.4, 14.4, 11, 14, 13.4), success = c(0L, 0L, 0L, 1L, 0L, 1L,
0L, 0L, 1L, 0L, 1L, 1L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L,
1L, 1L, 1L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L)), .Names = c("numeracy",
"anxiety", "success"), row.names = c(NA, -50L), class = "data.frame")```
`attach(A)`
```names(A)
[1] "numeracy" "anxiety"  "success"```
```head(A)
numeracy anxiety  success
1      6.6    13.8       0
2      7.1    14.6       0
3      7.3    17.4       0
4      7.5    14.9       1
5      7.9    13.4       0
6      7.9    13.5       1```

The variable ‘success’ is a binary variable that takes the value 1 for individuals who succeeded in gaining admission, and the value 0 for those who did not. Let’s look at the mean values of numeracy and anxiety.

```mean(numeracy)
[1] 10.722```
```mean(anxiety)
[1] 13.954```

We begin by fitting a model that includes interactions through the asterisk formula operator. The most commonly used link for binary outcome variables is the logit link, though other links can be used.

`model1 <- glm(success ~ numeracy * anxiety, binomial)`

glm() is the function that tells R to run a generalized linear model.

Inside the parentheses we give R important information about the model. To the left of the ~ is the dependent variable: success. It must be coded 0 & 1 for glm to read it as binary.

After the ~, we list the two predictor variables. The * indicates that not only do we want each main effect, but we also want an interaction term between numeracy and anxiety.

And finally, after the comma, we specify that the distribution is binomial. The default link function in glm for a binomial outcome variable is the logit. More on that below.

We can access the model output using summary().

```summary(model1)
Call:
glm(formula = success ~ numeracy * anxiety, family = binomial)
Deviance Residuals:
Min        1Q    Median        3Q       Max
-1.85712  -0.33055   0.02531   0.34931   2.01048
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)       0.87883   46.45256   0.019    0.985
numeracy          1.94556    4.78250   0.407    0.684
anxiety          -0.44580    3.25151  -0.137    0.891
numeracy:anxiety -0.09581    0.33322  -0.288    0.774
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 68.029  on 49  degrees of freedom
Residual deviance: 28.201  on 46  degrees of freedom
AIC: 36.201

Number of Fisher Scoring iterations: 7```

The estimates (coefficients of the predictors – numeracy and anxiety) are now in logits. The coefficient of numeracy is: 1.94556, so that a one unit change in numeracy produces approximately a 1.95 unit change in the log odds (i.e. a 1.95 unit change in the logit).

From the signs of the two predictors, we see that numeracy influences admission positively, but anxiety influences admission negatively.

We can’t tell much more than that as most of us can’t think in terms of logits. Instead we can convert these logits to odds ratios.

We do this by exponentiating each coefficient. (This means raise the value e –approximately 2.72–to the power of the coefficient. e^b).

So, the odds ratio for numeracy is:

`OR = exp(1.94556) = 6.997549`

However, in this version of the model the estimates are non-significant, and we have a non-significant interaction. Model1 produces the following relationship between the logit (log odds) and the two predictors:

`logit(p) = 0.88 + 1.95* numeracy - 0.45 * anxiety - .10* interaction term`

The output produced by glm() includes several additional quantities that require discussion.

We see a z value for each estimate. The z value is the Wald statistic that tests the hypothesis that the estimate is zero. The null hypothesis is that the estimate has a normal distribution with mean zero and standard deviation of 1. The quoted p-value, P(>|z|), gives the tail area in a two-tailed test.

For our example, we have a Null Deviance of about 68.03 on 49 degrees of freedom. This value indicates poor fit (a significant difference between fitted values and observed values). Including the independent variables (numeracy and anxiety) decreased the deviance by nearly 40 points on 3 degrees of freedom. The Residual Deviance is 28.2 on 46 degrees of freedom (i.e. a loss of three degrees of freedom).

About the Author: David Lillis has taught R to many researchers and statisticians. His company, Sigma Statistics and Research Limited, provides both on-line instruction and face-to-face workshops on R, and coding services in R. David holds a doctorate in applied statistics.

Binary, Ordinal, and Multinomial Logistic Regression for Categorical Outcomes
Get beyond the frustration of learning odds ratios, logit link functions, and proportional odds assumptions on your own. See the incredible usefulness of logistic regression and categorical data analysis in this one-hour training.

1. Muhammed Bello Abdulkadir says

With the same data set(continuous response and continuous+categorical explanatory) , can I fit a GLM to normal, Poisson, and binomial distribution the choose the best model? If yes please guide me.

• Yes, a generalized linear model can be used for normal, Poisson, or binomial data. The best approach is to fit the model that best fits the variable you’re working with. But all these distributions are related. For example, both binomial and Poisson could be counts, but for binomial, the count is out of a specific number of trials, but for Poisson, there is an infinite number of trials. So a binomial starts to look more and more like a Poisson as the number of trials goes to infinity. So if you’re in one of those situations, yes, go ahead and try both. But if you’ve got say, 10 trials, just use the binomial.

2. Mika says

Hi,

Thank you VERY much for that series. It helps a lot.

Just a question, shouldn’t it be -0.1 instead of -1.0 here:

logit(p) = 0.88 + 1.95* numeracy – 0.45 * anxiety – 0.1* interaction term

Thanks!

3. Eddie says

Hi,
OR for numeracy is 6.99… and that of anxiety is 0.64…
What does that say about the probability of success?

4. Clara says

Hello,

Thank you for this entry! Do you happen to know if GLM’s default is to use Type I or Type III Sum of Squares? And, if I want to make sure it’s Type III, how do I do that?

Thank you,
Clara

5. Leandro Casiraghi says

Hi, I am trying to use the GLM function on my binary data, and I need some help with getting reports for factors when these are not continuous but categorical, and have over two levels. In this case, I get separated z-values for comparisons between one reference level and the others. It gets only worse when you include an interaction!
Is there a way to get z-values for the effect of an overall factor in such case?
Thanks!

6. Erik Gribbin says

In your introduction above, you state “As a reminder, Generalized Linear Models are an extension of linear regression models that allow the dependent variable to be non-normal”. Would it be better to say that Generalized Linear Models are an extension of linear regression models that allow the residuals to be non-normal?

As Karen points out in her article: “Assumptions of Linear Models are about Residuals, not the Response Variable”, linear regression does not make assumptions about the distribution of the dependent variable – only the residual’s distribution.

Please note that, due to the large number of comments submitted, any questions on problems related to a personal study/project will not be answered. We suggest joining Statistically Speaking, where you have access to a private forum and more resources 24/7.