Most studies on intensive care unit (ICU) triage have focused on either patients admitted to or rejected from the ICU 1 2 3 . Few studies have documented improved survival by comparing similar patients admitted to ICUs and regular departments. The limitation of such observational research is the nonrandom assignment of the treatments, which may lead to selection bias 4 . Concerning ICU care, confounding by severity could plausibly occur in either direction; patients' severity intrinsically influences both the triage decision and the outcome. Some physicians may have been concerned that individuals with more severe organ failures would not benefit from ICU care while other physicians might recommend ICU care as a 'last resort' for their sickest patients, for fear of unintended negative effects.

To provide causal evidence from observational data, notably in critical care 5 , appropriate statistical tools have been proposed 6 7 . The propensity score (PS) was one of the first techniques that specifically addressed this question 6 . However, this method relies on the strong underlying assumption of exchangeability, that is, the absence of an unmeasured confounder, which cannot be tested. An attractive alternative approach is the instrumental variable (IV) method because it may consistently estimate the average treatment effect of exposure in marginal patients, even in the presence of unmeasured confounding. This method supposes that there is an instrument that is correlated with treatment but uncorrelated with unobserved patient severity. However, while PSs are mostly used in medical settings 8 , IV has been the standard method in econometrics 9 . Although proposed in the Health Sciences setting 10 , aside from introductory papers to IV for epidemiology 11 12 , IV has been poorly and only recently applied in medical research 13 14 15 .

In this paper, we sought to illustrate the use of the IV approach on an observational cohort study that aimed to evaluate the beneficial effect of ICU admission on hospital mortality (the ELDICUS study 16 ). Our objective was to assess such a benefit by introducing the concept of IV and by reviewing and comparing different IV approaches with a special focus on the selection of a valid instrument and on the best regression method in the case of a dichotomous outcome. In addition, some comparison between PS and IV with regard to estimating the causal benefit of ICU care from a large observational database was also provided.

A total of 8,201 patients were enrolled in the study: 6,752 (82.3%) patients were accepted, and 1,449 (17.7%) were rejected. Table 1 shows that major characteristics significantly differed between admitted and nonadmitted patients. The crude analysis revealed a reduction in hospital mortality associated with ICU admission (OR = 0.74, 95CI: 0.65-0.84, p < 0.0001; RD = -0.06, 95CI: -0.08;-0.03, p < 0.0001) (Tables 2 and 3). However, after adjusting for 35 baseline covariates considered associated with the outcome, ICU admission was associated with increased hospital mortality (OR = 1.25, 95CI: 1.07-1.46; p = 0.005; RD = 0.03, 95CI: 0.01; 0.05, p = 0.01).

ELDICUS is an observational study that intended to assess the benefit of ICU admission on mortality. Most previous studies have been based on cohort data analysed by standard statistical methods 4 . However, because ICU admission is likely determined jointly with an individual's likelihood of death, conventional estimates might be biased 47 48 . The instrumental variable method, which was initially developed for use with econometrics, has been proposed to handle such sources of bias, but it is still seldom applied to medical data 26 13 15 . To our knowledge, this is the first study to use IV analysis to examine the effect of first ICU admission on in-hospital mortality on critically ill patients. We explored the results by IV methods, using different instruments and different methods adapted to dichotomous exposures and outcomes as sensitivity analyses 38 . These results were compared with those obtained by standard regressions and propensity based analyses, using the in-hospital mortality as the primary end point.

We first used PS matched analysis 49 . Both the adjusted and the propensity based analyses found ICU admission to be associated with increased hospital mortality. However, PS methods might have some limitations. First, given the large imbalance in sample sizes between admitted and nonadmitted patients (82.3% of patients admitted to the ICU), the matching-without-replacement approach resulted in a dramatic reduction in the sample size. Indeed, only 20.4% of admitted patients could be matched to nonadmitted patients. Second, the PS does not handle the situation of unmeasured confounding. In the context of critically ill patients, it is likely that all the prognostic factors for hospital mortality would not be measurable at the time of ICU triage. Therefore, we sought to compare the results obtained with the PS with those obtained with specific methods that would handle the potential unmeasured confounding.

Instrumental variable methods are becoming increasingly popular because they seem to overcome the problem of unobserved confounding in observational studies 25 . The principle of IV analysis is to evaluate how much the variation in the treatment variable that is induced by the instrument affects the outcome. Although appealing, IV methods rely on strong assumptions that might limit their use in practice: first, the absence of any direct effect of the instrument on the outcome (usually described as the main assumption); second, that the variation in the IV causes substantial variations in the treatment variable (usually described as the IV strength); and third, that the relationship between the IV and the treatment is unconfounded. The main issue is finding a good instrument. However, because these assumptions are not empirically verifiable 12 25 the choice of a good instrument first relies on carefully evaluating the key assumptions of IV when identifying a potential IV. In our example, three variables served as potential instruments. The first IV was countries of enrolment, which shared close populations in terms of health status and medical resources 31 . This IV found no effect on the outcome but did find variations in the treatment exposure due to the countries' own policies regarding ICU admission. The second IV, the physician's age, has been suggested to influence the triage decision 30 but not to modify the outcome, given that ICU care is not provided solely by the physician who admitted the patient. Finally, the third IV was the physician specialization, which was chosen because in most European countries ICU physicians may be anaesthesiologists or intensivists 29 , and this characteristic may influence the admission policy while not affecting the outcome.

We then selected the best instrument from among these three potential IVs by examining the strength of the association between the IV and the treatment, as evaluated by the partial F-statistic and the partial r² from the first-stage regression 27 32 . All three selected instruments had partial F-statistics greater than 10, a threshold that supposedly indicates that the instrument is not weak 23 . However, the partial r² values were smaller than those usually reported in the medical or the economic literature 14 27 . Because the treatment variable was naturally binomial in our database, we sought to propose a more appropriate solution to evaluate the strength of the association between the IV and treatment. Using an OR as a measure of the association between treatment exposure and the IV, we found results similar to those obtained using the F-statistic or the partial r². The quality of the instrument was also evaluated by its ability to reduce the imbalance in the major covariates 27 . However, the IV-based analysis yielded estimates far different from those obtained with the propensity-matched sample. Indeed, the propensity-based estimates were similar to those obtained with conventional multivariate regression models, supporting a negative effect of ICU admission on in-hospital mortality, while all IV analyses resulted in a lack of impact of ICU admission on in-hospital mortality. Of course, because we do not know the true association between ICU utilization and hospital death, we cannot formally conclude that the one method is better than the other. A simulation study to explore differences between these analytical methods with respect to controlling for confounding would be of interest. Nevertheless, in the context of ICU patients, because hospital mortality is usually considered highly multifactorial the presence of unmeasured confounders appears likely. The absence of concordance between PS- and IV-based estimates may support the existence of unmeasured confounding. However, as previously emphasized by several authors 32 23 , the use of weak instruments may lead to large standard errors in the IV estimates or even bias in the IV estimates if the weakness is associated with a small sample size or a violation of the main assumption. In our case, IV methods undoubtedly yielded estimates with larger confidence intervals; thus, the limited partial r² can be considered a threat to the validity of the IV method. However, Martens et al. showed that when bias occurs in the IV estimates, it is in the direction of the ordinary least squares estimation 23 . In contrast, our results of the 2SLS estimator were far different from the results obtained using ordinary least square regression. This finding supports the idea that, despite the limited partial r² that may explain the large standard errors, the large sample size and the validity of the main assumption limited the bias in the IV estimates. Nevertheless, this finding could illustrate the low precision of the estimates and thus the low statistical power of treatment comparison.

The second limitation of IV techniques is that they rely on multiple stage linear models, which might be nonadaptive in the context of dichotomous outcome measures 37 . We compared the results obtained by the different methods previously proposed in the context of dichotomous outcomes 37 and found relatively large differences between the various IV approaches. Indeed, if all IV estimations led to a nonsignificant effect of ICU admission, then the 2SLS estimator was the only one that was far different from the crude analysis, which is expected to be the most biased method. As previously described in the case of weak instruments 23 , all other IV estimators seemed biased in the direction of the unadjusted ordinary least squares estimation. Hence, our results strongly support the use of standard 2SLS methods, even when dealing with dichotomous outcome measures.

Our results could be compared with those based on a previously published propensity-based analysis of the ELDICUS database 16 . Our IV estimate did not conflict with previous PS estimates, though larger confidence intervals modified the conclusions. However, our PS results were different from those previously published 16 . This difference can be explained by major differences in the analytic procedure: first, we considered hospital mortality not 28- and 90-days mortalities; second, we used a PS matching method 21 whereas Iapichino provided estimates adjusted on PS quintiles. Thus, conditional estimates provided by Iapichino can substantially differ from marginal estimates reached by the former, especially when using the OR as the association measure, because of its noncollapsibility 43 . Moreover, we only assessed the benefit of the ICU first triage decision whereas Iapichino considered all the triages independently. Finally, differences in the patient selection should be stressed because we analysed a total of 8,201 patients including 6,752 (82%) first admissions. Conversely, Iapichino 16 included in the analysis of 28-day mortality 7,308 first admissions, a lower number because of the exclusion of patients with a lack of information on time of triage, triage decision, or outcome and the exclusion of those referred to a coronary unit. These results suggested an ICU benefit among severe patients and were confirmed with 6,500 patients triaged only once. It is likely that Iapichino's cohort included somewhat more severe patients, suggesting an ICU benefit among severe patients.

Finally, like randomized clinical trials, external validity depends on the studied population, and it should be emphasized that IV- and PS-matching attempt to estimate different effects of treatment. Indeed, IV approaches yield estimates of a local average treatment effect (LATE) 50 51 52 while propensity-based approaches yield estimates of the average treatment effect on the treated (ATT) 45 . Informally, the effect of ICU admission, as estimated via PS matching, can be defined as the effect observed in the patients admitted as compared with the effect observed in patients with a similar propensity for ICU admission but who were not admitted. PS matching does not capture the effect of ICU admission in nonadmitted patients who had a very low probability of being admitted. The IV approach yields estimates of the treatment effect not only in the treated but also in a restricted subgroup of patients for whom the instrument was informative about treatment assignment; these are the so-called "marginal" or compliers. Noncompliers, as opposed to compliers, are patients who, whatever the value of the instrument, would always have been treated or untreated. Hence, in our situation, the effect of ICU admission on hospital mortality is not captured by the IV approach for the patients who, whatever the value of the physician's specialization, i.e., the chosen instrument, would have always been accepted or rejected from the ICU. Thus, it is important for researchers to state the treatment-effect concept that they are trying to identify before beginning estimation 53 .

Instrumental variable methods offer an appealing alternative to handle the selection bias related to nonrandomized designs, especially when the presence of significant unmeasured confounding is suspected. Applied to the ELDICUS database, this analysis failed to show any significant beneficial effect of ICU admission on hospital mortality. When the clinical question underlying the creation of the database is to assess a local average treatment effect, effort should be made to incorporate in the dataset covariates that behave as appropriate instruments, allowing IV analysis if the presence of unmeasured confounding is suspected.

IV: instrumental variables; PS: propensity score; ICU: intensive care unit; GMM: Generalized methods of moments; OR: odds ratio; RD: risk difference; 95CI: 95% Confidence interval; LATE: local average treatment effect; ATT: average treatment effect on the treated

The authors declare that they have no competing interests.

RP performed the analysis and wrote the manuscript, CS was the principal investigator of ELDICUS, DP was the French principal investigator of ELDICUS, and SC supervised the analysis and the elaboration of the manuscript. All authors read and approved the final manuscript.