When is a meta-analysis conclusive? A guide to Trial Sequential Analysis with an example of remote ischemic preconditioning for renoprotection in patients undergoing cardiac surgery

Abstract

Regardless of whether a randomized trial finds a statistically significant effect for an intervention or not, readers often wonder if the trial was large enough to be conclusive. To answer this question, we can estimate the required sample size for a trial by considering how commonly the outcome occurs, the smallest effect of clinical importance and the acceptable risk of falsely detecting or rejecting that effect. But when is a meta-analysis conclusive? We explain and illustrate the interpretation of Trial Sequential Analysis (TSA), a method increasingly used to answer this question. We conducted a conventional meta-analysis which suggested that, in adults undergoing cardiac surgery, remote ischemic preconditioning does not provide a statistically significant reduction in acute kidney injury (AKI) [12 trials, 4230 patients; relative risk 0.87 (95% confidence interval 0.74-1.02); P = 0.08; I2= 35%] or the risk of receiving acute dialysis [5 trials, 2111 patients; relative risk 1.15 (95% confidence interval 0.42-3.19); P = 0.78; I2 = 59%]. TSA demonstrates that as little as a 20% relative risk reduction in AKI is unlikely. Reliably finding effects on acute dialysis and smaller effects on AKI would require much more evidence. Notably, conventional meta-analyses conducted at one of the two earlier time points may have prematurely declared a statistically significant reduction in AKI, even though at no point in the TSA was there sufficient evidence to support such an effect. With this and other examples, we demonstrate that the TSA can prevent premature conclusions from meta-analyses.

Keywords: information size; meta-analysis; monitoring boundaries; remote ischemic preconditioning; sample size; trial sequential analysis.