A Two-Step Framework for Validating Causal Effect Estimates

Abstract

Background: Comparing causal effect estimates obtained using observational data to those obtained from the gold standard (i.e., randomized controlled trials [RCTs]) helps assess the validity of these estimates. However, comparisons are challenging due to differences between observational data and RCT generated data. The unknown treatment assignment mechanism in the observational data and varying sampling mechanisms between the RCT and the observational data can lead to confounding and sampling bias, respectively.

Aims: The objective of this study is to propose a two-step framework to validate causal effect estimates obtained from observational data by adjusting for both mechanisms.

Materials and methods: An estimator of causal effects related to the two mechanisms is constructed. A two-step framework for comparing causal effect estimates is derived from the estimator. An R package RCTrep is developed to implement the framework in practice.

Results: A simulation study is conducted to show that using our framework observational data can produce causal effect estimates similar to those of an RCT. A real-world application of the framework to validate treatment effects of adjuvant chemotherapy obtained from registry data is demonstrated.

Conclusion: This study constructs a framework for comparing causal effect estimates between observational data and RCT data, facilitating the assessment of the validity of causal effect estimates obtained from observational data.

Keywords: causal estimates; sampling mechanism; treatment assignment mechanism; validation.