Background: In the process of research and development of a new treatment, clinical trials are conducted to evaluate its safety and efficacy. Key to streamlining the process is to utilize appropriate historical information on an outcome of a control treatment when designing and analyzing a clinical trial.
Methods: For the use of such historical control information, there exist a meta-analytic approach and power prior approach. In this article, we evaluate their performance with regard to the type I error (TIE) rate and power through a simulation study where we analyze the data on a binary outcome of an experimental treatment and a control treatment from a new small-scale trial, along with the corresponding data of the control treatment from multiple historical trials. The reason is that the difference in the performance between the 2 approaches has not been clear.
Results: When historical trials were homogeneous, the power was higher in the power prior approach and the meta-analytic approach using a beta-binomial model with a less noninformative prior than the other approaches. However, when heterogeneous historical trials were mixed, the power was lower, or the TIE rates got inflated.
Conclusions: To make use of historical control data, if importance is attached to control of the TIE rate, the meta-analytic approach using a normal-normal hierarchical model may be preferable to the power prior approach, whereas if attached to improvement of the power, this preference be reversed. Anyway, the best approach should be chosen by comparing the operational characteristics of the approaches.
Keywords: Bayesian clinical trial design; borrow strength; historical control data; operational characteristics; small-scale trial.