Fig. 3
From: Performance metrics for models designed to predict treatment effect
Calibration plot of pairwise treatment effect of DPP data from patients receiving lifestyle intervention. This Figure depicts observed versus predicted pairwise treatment effect by smoothed calibration curves (blue line with 95% confidence interval displayed by grey shaded area) and quantiles of predicted pairwise treatment effect (black dots) of lifestyle intervention versus placebo treatment. Observed pairwise treatment effect was obtained by matching patients based on patient characteristics. Smoothed calibration curves were obtained by local regression of the observed pairwise treatment effect of matched patient pairs on predicted pairwise treatment effect of matched patient pairs. For prediction of individualized treatment effect, we used: a risk modelling approach (panel A), a treatment effect modelling approach (panel B), and a causal forest (panel C). Confidence intervals around metric values were obtained using 100 bootstrap samples. Abbreviations: CitL, calibration-in-the-large; Eavg-B, Eavg-for-benefit; E50-B, E50-for-benefit; E90-B, E90-for-benefit; CE-B, cross-entropy-for-benefit; Brier-B, Brier-for-benefit; C-B, C-for-benefit