A quantitative assessment of the consistency of projections from five mathematical models of the HIV epidemic in South Africa: a model comparison study

Abstract

Background: Mathematical models are increasingly used to inform HIV policy and planning. Comparing estimates obtained using different mathematical models can test the robustness of estimates and highlight research gaps. As part of a larger project aiming to determine the optimal allocation of funding for HIV services, in this study we compare projections from five mathematical models of the HIV epidemic in South Africa: EMOD-HIV, Goals, HIV-Synthesis, Optima, and Thembisa.

Methods: The five modelling groups produced estimates of the total population, HIV incidence, HIV prevalence, proportion of people living with HIV who are diagnosed, ART coverage, proportion of those on ART who are virally suppressed, AIDS-related deaths, total deaths, and the proportion of adult males who are circumcised. Estimates were made under a "status quo" scenario for the period 1990 to 2040. For each output variable we assessed the consistency of model estimates by calculating the coefficient of variation and examining the trend over time.

Results: For most outputs there was significant inter-model variability between 1990 and 2005, when limited data was available for calibration, good consistency from 2005 to 2025, and increasing variability towards the end of the projection period. Estimates of HIV incidence, deaths in people living with HIV, and total deaths displayed the largest long-term variability, with standard deviations between 35 and 65% of the cross-model means. Despite this variability, all models predicted a gradual decline in HIV incidence in the long-term. Projections related to the UNAIDS 95-95-95 targets were more consistent, with the coefficients of variation below 0.1 for all groups except children.

Conclusions: While models produced consistent estimates for several outputs, there are areas of variability that should be investigated. This is important if projections are to be used in subsequent cost-effectiveness studies.

Keywords: Coefficient of variation; Epidemic projections; HIV; Mathematical modelling; Model comparison.