Seconder of the Vote of thanks to Vansteelandt and Dukes and Contribution to the Discussion of ‘Assumption-Lean Inference for Generalised Linear Model Parameters’
Vanessa Didelez · Journal of the Royal Statistical Society Series B (Statistical Methodology) · 2022
In my view, one of the most important contributions of the field of causal inference has been to place the target of inference, the desired estimand, at the centre of the analysis. The estimand is chosen in view of the research question, and typically reflects what decision problem we need to solve or what our ideal (target) trial would be. Crucially, the (causal) estimand is not automatically a parameter that happens to parametrise a chosen model. The role of models is mostly as a mere tool. For instance, with a survival outcome we may employ hazard regressions but still obtain effects in terms of survival probabilities. This is in some contrast to traditional ‘statistical modelling’, where the model often appears to be the starting point, somehow suggested by the data (e.g. a logistic regression for a binary outcome), and the natural coefficients are reported (e.g. log-odds-ratios). However, it has become abundantly clear that, as far as prediction is concerned, default regression models are regularly outperformed by machine learning methods such as random forests and similar flexible methods in competitions. The former are then typically defended with the argument of being more interpretable and their parameters being useful summaries of (conditional) dependence. The thoughtful proposal of Vansteelandt and Dukes (V&D) addresses many of these issues with an important lesson right at the start: models, even when only used as tools, may implicitly affect the meaning of our estimands and the desired summaries may be invalidated under misspecification. V&D therefore place a (particular) estimand at the core, aiming at a simple summary of a (high-dimensional) conditional dependence such that the estimand remains sensible regardless of whether a specific model holds. The influence function-based estimation method then uses flexible machine learning for the nuisance functions in a way that ensures valid inference. While I agree with many of V&D’s points, I am (very slightly) concerned that they might distract from asking scientifically relevant questions, which would conflict with the authors’ intention. Their proposal restricts our choice of estimand. For example, when the exposure A is continuous, V&D make the entirely valid point that the potential outcome Ya (and thus an estimand as E(Ya-Ya′)) typically represents an unrealistic intervention of setting A to exactly a even for people whose ‘natural’ value of A would be very different from a. But, such an estimand can be scientifically meaningful, for example when A is the dosage of a drug; in contrast, it is less meaningful when A is BMI or income, for example. However, the proposed alternative estimand does not solve this problem—only when we actually formulate a scientifically relevant question will we (possibly) find scientifically relevant answers. Giving less weight to problematic covariate regions does not achieve this—and it does not absolve us from trying harder to elicit what a scientifically relevant estimand might be instead, for instance for the effect of BMI on an outcome. Moreover, it is a useful feature of approaches like IPW or propensity score matching that they alert us to problems, for example when we carry out diagnostics and find that there is a lack of overlap and then go on to characterise regions of sufficient overlap; or even when we just have extreme weights and confidence intervals get very wide, this rightly indicates that no useful statement can be made about our estimand because there is too little information in the data. I worry that such aspects are perhaps lost with V&D’s approach, or it would be interesting to know if it could be supplemented by something analogous. In motivating the proposed estimands, V&D further refer to our ‘familiarity’ with main effects and interactions in GLMs. However, familiarity is not per-se a relevant criterion: it does, again, not ensure scientific relevance. There are plenty of examples, for instance the problematic causal interpretation of hazard ratios despite many medical statisticians being extremely familiar with them. Besides, what exactly is it that we think we are familiar with? When regression models are used to describe conditional dependencies, the correct interpretation of regression coefficients as just a measure of conditional dependence, and not as an effect, is rare. A clear distinction between covariates included to adjust for confounding and those included as potential ‘effect’ modifiers is also rare, typically no rationale for the inclusion of particular covariates is given at all—what then motivates the choice of L in the basic quantity E(Y|A, L)? In summary, I very much welcome V&D’s proposal as a huge improvement on traditional statistical regression modelling; but with regard to causal analyses there is still room for further research into combining the truly impressive results on assumption-lean valid inference with the quest for more scientifically meaningful estimands. No doubt, V&D will lead the way. It is with greatest pleasure that I second the vote of thanks for this most stimulating and important paper. The vote of thanks was passed by acclamation.