An efficient strategy allowed English-speaking reviewers to identify foreign-language articles eligible for a systematic review

Abstract

Objective: To assess English-speaking reviewers' accuracy in determining the eligibility of foreign-language articles for a systematic review.

Study design and settings: Systematic review of randomized controlled trials of therapy for fibromyalgia. Guided by 10 questions, English-speaking reviewers screened non-English-language articles for eligibility. Teams of two native-language speakers provided reference standard judgments of eligibility.

Results: Of 15,466 potentially eligible articles, we retrieved 763 in full text, of which 133 were published in 19 non-English languages; 53 trials published in 11 languages other than English proved eligible. Of the 53 eligible articles, English-language reviewers guided by the 10 questions mistakenly judged 6 as ineligible; of the 80 ineligible articles, 8 were incorrectly judged eligible by English-language reviewers (sensitivity=0.89; specificity=0.90). Use of a simple three-step rule (excluding languages with less than three articles, reviewing titles and abstracts for clear indications of eligibility, and noting the lack of a clearly reported statistical analysis unless the word "random" appears) led to accurate classification of 51 of 53 articles (sensitivity=0.96; specificity=0.70).

Conclusion: Our findings show promise for limiting the need for non-English-language review teams in systematic reviews with large numbers of potentially eligible non-English-language articles.

Keywords: Bias; English-language bias; Fibromyalgia; Language of publication; Methodology; Systematic review.