Fig. 2
From: Performance metrics for models designed to predict treatment effect
Calibration plots of pairwise treatment effect of simulated data from patients receiving lifestyle intervention. This Figure depicts observed versus predicted pairwise treatment effect by smoothed calibration curves (blue line) and quantiles of predicted pairwise treatment effect (black dots) of simulated data from the lifestyle intervention versus placebo treatment. Observed pairwise treatment effect was obtained by matching patients based on patient characteristics. Smoothed calibration curves were obtained by local regression of the observed pairwise treatment effect of matched patient pairs on predicted pairwise treatment effect of matched patient pairs. For prediction of individualized treatment effect, we used a treatment effect modelling approach for the “optimal model” (panel A) and three “perturbed models” that overestimate average treatment effect (panel B), risk heterogeneity (panel C), and treatment effect heterogeneity (panel D). The average treatment effect is 13.0, 20.9, 13.0 (after a correction of \({\beta }_{W}\) with -0.195), and 13.0 (after a correction of \({\beta }_{W}\) with -0.19), respectively. Abbreviations: RMSE, root mean squared error; CitL, calibration-in-the-large; Eavg-B, Eavg-for-benefit; E50-B, E50-for-benefit; E90-B, E90-for-benefit; CE-B, cross-entropy-for-benefit; Brier-B, Brier-for-benefit; C-B, C-for-benefit