Performance of methods for analyzing continuous data from stratified cluster randomized trials - A simulation study

Sayem Borhan; Jinhui Ma; Alexandra Papaioannou; Jonathan Adachi; Lehana Thabane

doi:10.1016/j.conctc.2023.101115

Performance of methods for analyzing continuous data from stratified cluster randomized trials - A simulation study

Contemp Clin Trials Commun. 2023 Mar 14:33:101115. doi: 10.1016/j.conctc.2023.101115. eCollection 2023 Jun.

Authors

Sayem Borhan^{1

2}, Jinhui Ma¹, Alexandra Papaioannou^{3

4}, Jonathan Adachi^{3

4}, Lehana Thabane^{1

2}

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
² Biostatistics Unit, Research Institute of St Joseph's Healthcare, Hamilton, ON, Canada.
³ GERAS Centre, Hamilton Health Sciences, Hamilton, ON, Canada.
⁴ Department of Medicine, McMaster University, Hamilton, ON, Canada.

Abstract

Background: The adoption of cluster randomized trials (CRTs) with the stratified design is currently gaining widespread interest. In the stratified design, clusters are first grouped into two or more strata and then randomized into treatment groups within each stratum. In this study, we evaluated the performance of several commonly used methods for analyzing continuous data from stratified CRTs.

Methods: This is a simulation study where we compared four methods: mixed-effects, generalized estimating equation (GEE), cluster-level (CL) linear regression and meta-regression methods to analyze the continuous data from stratified CRTs using a simulation study with varying numbers of clusters, cluster sizes, intra-cluster correlation coefficients (ICCs) and effect sizes. This study was based on a stratified CRT with one stratification variable with two strata. The performance of the methods was evaluated in terms of the type I error rate, empirical power, root mean square error (RMSE), and width and coverage of the 95% confidence interval (CI).

Results: GEE and meta-regression methods had high type I error rates, higher than 10%, for the small number of clusters. All methods had similar accuracy, measured through RMSE, except meta-regression. Similarly, all methods but meta-regression had similar widths of 95% CIs for the small number of clusters. For the same sample size, the empirical power for all methods decreased as the value of the ICC increased.

Conclusion: In this study, we evaluated the performance of several methods for analyzing continuous data from stratified CRTs. Meta-regression was the least efficient method compared to other methods.

Keywords: Cluster randomized trials; Continuous; Simulation; Stratified design.