Internal validation of Automated Visual Evaluation (AVE) on smartphone images for cervical cancer screening in a prospective study in Zambia

Abstract

Objectives: Visual inspection with acetic acid (VIA) is a low-cost approach for cervical cancer screening used in most low- and middle-income countries (LMICs) but, similar to other visual tests, is subjective and requires sustained training and quality assurance. We developed, trained, and validated an artificial-intelligence-based "Automated Visual Evaluation" (AVE) tool that can be adapted to run on smartphones to assess smartphone-captured images of the cervix and identify precancerous lesions, helping augment VIA performance.

Design: Prospective study.

Setting: Eight public health facilities in Zambia.

Participants: A total of 8204 women aged 25-55.

Interventions: Cervical images captured on commonly used low-cost smartphone models were matched with key clinical information including human immunodeficiency virus (HIV) and human papillomavirus (HPV) status, plus histopathology analysis (where applicable), to develop and train an AVE algorithm and evaluate its performance for use as a primary screen and triage test for women who are HPV positive.

Main outcome measures: Area under the receiver operating curve (AUC); sensitivity; specificity.

Results: As a general population screening tool for cervical precancerous lesions, AVE identified cases of cervical precancerous and cancerous (CIN2+) lesions with high performance (AUC = 0.91, 95% confidence interval [CI] = 0.89-0.93), which translates to a sensitivity of 85% (95% CI = 81%-90%) and specificity of 86% (95% CI = 84%-88%) based on maximizing the Youden's index. This represents a considerable improvement over naked eye VIA, which as per a meta-analysis by the World Health Organization (WHO) has a sensitivity of 66% and specificity of 87%. For women living with HIV, the AUC of AVE was 0.91 (95% CI = 0.88-0.93), and among those testing positive for high-risk HPV types, the AUC was 0.87 (95% CI = 0.83-0.91).

Conclusions: These results demonstrate the feasibility of utilizing AVE on images captured using a commonly available smartphone by nurses in a screening program, and support our ongoing efforts for moving to more broadly evaluate AVE for its clinical sensitivity, specificity, feasibility, and acceptability across a wider range of settings. Limitations of this study include potential inflation of performance estimates due to verification bias (as biopsies were only obtained from participants with visible aceto-white cervical lesions) and due to this being an internal validation (the test data, while independent from that used to develop the algorithm was drawn from the same study).

Keywords: cancer prevention; cancer risk factors; deep learning; epidemiology and prevention; gynecological oncology; machine learning; statistical methods; women's cancer.