Unrestricted weighted least squares represent medical research better than random effects in 67,308 Cochrane meta-analyses

T D Stanley; John P A Ioannidis; Maximilian Maier; Hristos Doucouliagos; Willem M Otte; František Bartoš

doi:10.1016/j.jclinepi.2023.03.004

Unrestricted weighted least squares represent medical research better than random effects in 67,308 Cochrane meta-analyses

J Clin Epidemiol. 2023 May:157:53-58. doi: 10.1016/j.jclinepi.2023.03.004. Epub 2023 Mar 6.

Authors

T D Stanley¹, John P A Ioannidis², Maximilian Maier³, Hristos Doucouliagos⁴, Willem M Otte⁵, František Bartoš⁶

Affiliations

¹ Department of Economics, Deakin University, Melbourne, Australia; Deakin Laboratory for the Meta-Analysis of Research (DeLMAR), Deakin University, Melbourne, Australia. Electronic address: tom.stanley1@deakin.edu.au.
² Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA; Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA; Department of Epidemiology and Biostatistics, Stanford University School of Medicine, Stanford, CA, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, USA.
³ Department of Experimental Psychology, University College London, London, United Kingdom.
⁴ Department of Economics, Deakin University, Melbourne, Australia; Deakin Laboratory for the Meta-Analysis of Research (DeLMAR), Deakin University, Melbourne, Australia.
⁵ Department of Pediatric Neurology, UMC Utrecht Brain Center, University Medical Center Utrecht, and Utrecht University, Utrecht, Netherlands.
⁶ Department of Psychological Methods, University of Amsterdam, Amsterdam, Netherlands.

PMID: 36889450
DOI: 10.1016/j.jclinepi.2023.03.004

Abstract

Objectives: To evaluate how well meta-analysis mean estimators represent reported medical research and establish which meta-analysis method is better using widely accepted model selection measures: Akaike information criterion (AIC) and Bayesian information criterion (BIC).

Study design and setting: We compiled 67,308 meta-analyses from the Cochrane Database of Systematic Reviews (CDSR) published between 1997 and 2020, collectively encompassing nearly 600,000 medical findings. We compared unrestricted weighted least squares (UWLS) vs. random effects (RE); fixed effect was also secondarily considered.

Results: The probability that a randomly selected systematic review from the CDSR would favor UWLS over RE is 79.4% (95% confidence interval [CI_95%]: 79.1; 79.7). The odds ratio that a Cochrane systematic review would substantially favor UWLS over RE is 9.33 (CI_95%: 8.94; 9.73) using the conventional criterion that a difference in AIC (or BIC) of two or larger represents a 'substantial' improvement. UWLS's advantage over RE is most prominent in the presence of low heterogeneity. However, UWLS also has a notable advantage in high heterogeneity research, across different sizes of meta-analyses and types of outcomes.

Conclusion: UWLS frequently dominates RE in medical research, often substantially. Thus, the UWLS should be reported routinely in the meta-analysis of clinical trials.

Keywords: Cochrane systematic reviews; Meta-analysis; Random effects; Unrestricted weighted least squares.

Publication types

Meta-Analysis
Research Support, Non-U.S. Gov't

MeSH terms

Bayes Theorem
Biomedical Research*
Humans
Least-Squares Analysis
Systematic Reviews as Topic