Using Active Learning for Speeding up Calibration in Simulation Models

Mucahit Cevik; Mehmet Ali Ergun; Natasha K Stout; Amy Trentham-Dietz; Mark Craven; Oguzhan Alagoz

doi:10.1177/0272989X15611359

Using Active Learning for Speeding up Calibration in Simulation Models

Med Decis Making. 2016 Jul;36(5):581-93. doi: 10.1177/0272989X15611359. Epub 2015 Oct 15.

Authors

Mucahit Cevik¹, Mehmet Ali Ergun¹, Natasha K Stout², Amy Trentham-Dietz³, Mark Craven⁴, Oguzhan Alagoz^{1

3}

Affiliations

¹ Department of Industrial and Systems Engineering, University of Wisconsin, Madison, WI, USA (MC, MAE, OA)
² Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA (NKS)
³ Department of Population Health Sciences and Carbone Cancer Center, University of Wisconsin, Madison, WI, USA (AT-D, OA)
⁴ Department of Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA (MC)

Abstract

Background: Most cancer simulation models include unobservable parameters that determine disease onset and tumor growth. These parameters play an important role in matching key outcomes such as cancer incidence and mortality, and their values are typically estimated via a lengthy calibration procedure, which involves evaluating a large number of combinations of parameter values via simulation. The objective of this study is to demonstrate how machine learning approaches can be used to accelerate the calibration process by reducing the number of parameter combinations that are actually evaluated.

Methods: Active learning is a popular machine learning method that enables a learning algorithm such as artificial neural networks to interactively choose which parameter combinations to evaluate. We developed an active learning algorithm to expedite the calibration process. Our algorithm determines the parameter combinations that are more likely to produce desired outputs and therefore reduces the number of simulation runs performed during calibration. We demonstrate our method using the previously developed University of Wisconsin breast cancer simulation model (UWBCS).

Results: In a recent study, calibration of the UWBCS required the evaluation of 378 000 input parameter combinations to build a race-specific model, and only 69 of these combinations produced results that closely matched observed data. By using the active learning algorithm in conjunction with standard calibration methods, we identify all 69 parameter combinations by evaluating only 5620 of the 378 000 combinations.

Conclusion: Machine learning methods hold potential in guiding model developers in the selection of more promising parameter combinations and hence speeding up the calibration process. Applying our machine learning algorithm to one model shows that evaluating only 1.49% of all parameter combinations would be sufficient for the calibration.

Keywords: active learning; artificial neural networks; calibration; cancer simulation; machine learning.

MeSH terms

Algorithms
Calibration
Humans
Learning*
Models, Theoretical*
Neural Networks, Computer

Abstract

MeSH terms

Grants and funding