I had a collaborative paper reviewed this week that had in it odds ratios (OR) from logistic regressions (used to examine whether one way of measuring norms was a stronger predictor of a dichotomous behavior than another measure of the norm). A reviewer said “The comparison of odds ratios across logistics regression models has been criticized as this approach might be subject to an unobserved heterogeneity bias (C Mood 2010, European Sociological Review). In order to solve this issue, the authors may want to consider the use of Average Marginal Effects as it is suggested in the study by Mood 2010.” The review is right, and I should have caught this before signing-off on the analyses in the paper. That pesky non-linear logistic function (the s-shape curve for the probabilities) creates problems of interpretation of coefficients from logistic regression, especially when we want to compare them to one another (e.g. when looking for interactions, when looking at sequential models like in mediation, or when simply comparing strengths of association between predictors as in the collaborative paper mentioned above). The interpretational problem emphasized by the Mood paper is the following: that in logistic regression the OR for one variable (X1) can change when we control for another variable (X2) even when X1 and X2 are not correlated. The same is not true in linear regression. In linear regression if X1 and X2 are uncorrelated then the beta for Y regressed on X1 will be the same whether we control for X2 or not. This phenomena in logistic regression (due to so-called ‘unobserved heterogeneity bias’) can create problems when you are trying to interpret differences in coefficients (i.e differences in ORs).
One straightforward way to avoid (or at least limit) these problems with using OR from logistic regression is to switch to using the Average Marginal Effects which transform back to the probability (rather than the log odds) scale and aggregate (i.e. marginalize) the effect estimates on the probability scale. You still fit a logistic (or log link) model but you don’t use the coefficients directly as effect estimates (i.e. you don’t use exp(beta) = OR), instead you use expected contrasts formed back on the probability scale. This can be done by the MARGINS command in commonly used software (in Stata, or SAS has a macro, or R package).
Another way to avoid these problem with using OR from logistic regression is to just use linear regression with the 0-1 outcome. I know, I know, if you have any experience doing data analysis (especially if you have a quantitative degree and took statistics classes) you may be saying, “enough is enough, a 0-1 variable *can’t* be fit with a linear regression model, it is clearly not continuous, it is clearly not normally distributed, the predictions will be outside the parameter space (0,1), and the relationship can’t be a straight line – all the assumptions of linear regression are wrong”. By using linear regression you are modeling the probabilities of Y=1 directly. The advantage of the linear regression is that it avoids the problem of the unobserved heterogeneity bias that creeps into logistic regression and we can interpret the effect estimates as the average risk difference in the outcome for a one unit change in the predictor. Recall that linear regression is modeling the E(Y) (i.e. expected or mean value of Y). When Y is a 0-1 variable then E(Y) is just the probability of Y=1. Indeed, the beta from the linear regression of a 0-1 outcome is a consistent (i.e. asymptotically unbiased) estimator of the Average Marginal Effect. Standard errors need a little care since the typical ones from linear regression rely on the normality assumption, but you can use robust standard errors or else use a linear binomial model. Economists have been using linear regression for 0-1 outcomes for a long time, see https://statisticalhorizons.com/linear-vs-logistic.
Finally, if you are a regular user of logistic regression, I want to give a strong recommendation to read the paper by Carina Mood “Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do About It” European Sociological Review , FEBRUARY 2010, Vol. 26, No. 1 , pp. 67-82. She wrote a terrifically clear close look into the problem and describes the solutions that I mention above. At the time she wrote the paper in 2010, there were not yet software options in STATA, SAS, and R for calculating the Average Marginal Effects (i.e. MARGINS commands), but now that there is, it seems to me that we should really start using them more than the OR in order to produce results with better interpretability. I know I will.