Purpose: Differences in the benefits of treatment on 5-year overall survival have been observed in 12 randomized phase III colon cancer adjuvant clinical trials from the ACCENT group. We investigated the reasons for these differences by incorporating the distribution of the observed covariates from each trial.
Materials and methods: We applied state-of-the-art transportability methods on the basis of causal inference, and compared them with a conventional meta-analysis approach to predict the treatment effect for the target population. Prediction errors were defined to evaluate whether the identifiability conditions necessary for causal inference were satisfied among the 12 trials, and to measure the performance of each method.
Results: In the one-trial-at-a-time transportability analysis, the ranks of prediction errors for the target population were mostly consistent with the discrepancy in treatment effects among the 12 trials across the three models. The overall prediction errors between the leave-one-trial-out transportability method and the conventional individual participant data meta-analysis approach were very similar, and more than 40% lower than the overall prediction errors from the one-trial-at-a-time transportability method.
Conclusion: The discrepancy in treatment effects among the 12 trials is unlikely to arise from the choice of model specification or distribution of observed covariates but from the distribution of unobserved covariates or study-level features. The ability to quantify heterogeneity among the 12 trials was greatly reduced in both the leave-one-trial-out transportability method and the conventional meta-analysis approach compared with the one-trial-at-a-time transportability method.