From single-arm studies to externally controlled studies. Methodological considerations and guidelines

Abstract

Single-arm studies are sometimes used as pivotal studies but they have methodological limitations which prevent them from obtaining the high level of reliability as for a randomised controlled study which remains the gold standard in the evaluation of new treatments. The objective of this roundtable was to discuss the limitations of these single-arm studies, to analyse available and acceptable solutions in order to propose guidelines for their conduct and assessment. Single-arm studies themselves are intrinsically inappropriate for demonstrating the benefit of a new treatment because it is impossible to infer the benefit from a value obtained under treatment without knowing what it would have been in the absence of the new treatment. The implication is that comparison with other data is necessary. However this comparison has limitations due to (1) the post hoc choice of the reference used for comparison, (2) the confusion bias for which an adjustment approach is imperative and, (3) the other biases, measure and attrition among others. When these limitations are taken into account this should, first and foremost, lead to the conduct of externally controlled trials instead of single-arm trials as is proposed by the latest version of ICH E10. Moreover, the external control must be formalised in the study protocol with a priori selection of both the reference control and the formal method of comparison: test in relation to a standard, adjustment on individual data, a synthetic control group or matching-adjusted indirect comparisons (MAIC). Lastly, externally controlled studies must be restricted to situations where randomisation is infeasible. To be acceptable, these studies must be able to guarantee freedom from residual confusion bias, which is only truly acceptable if the observed effect is dramatic and the usual course of the disease is highly predicable.

Keywords: Control group; External control study; Indirect comparison; Matching-adjusted indirect comparisons; Non-comparative; Single-arm study.