Objective: The purpose of this study was to evaluate the performance of different supervised machine learning algorithms to predict achievement of minimum clinically important difference (MCID) in neck pain after surgery in patients with cervical spondylotic myelopathy (CSM).
Methods: This was a retrospective analysis of the prospective Quality Outcomes Database CSM cohort. The data set was divided into an 80% training and a 20% test set. Various supervised learning algorithms (including logistic regression, support vector machine, decision tree, random forest, extra trees, gaussian naïve Bayes, k-nearest neighbors, multilayer perceptron, and extreme gradient boosted trees) were evaluated on their performance to predict achievement of MCID in neck pain at 3 and 24 months after surgery, given a set of predicting baseline features. Model performance was assessed with accuracy, F1 score, area under the receiver operating characteristic curve, precision, recall/sensitivity, and specificity.
Results: In total, 535 patients (46.9%) achieved MCID for neck pain at 3 months and 569 patients (49.9%) achieved it at 24 months. In each follow-up cohort, 501 patients (93.6%) were satisfied at 3 months after surgery and 569 patients (100%) were satisfied at 24 months after surgery. Of the supervised machine learning algorithms tested, logistic regression demonstrated the best accuracy (3 months: 0.76 ± 0.031, 24 months: 0.773 ± 0.044), followed by F1 score (3 months: 0.759 ± 0.019, 24 months: 0.777 ± 0.039) and area under the receiver operating characteristic curve (3 months: 0.762 ± 0.027, 24 months: 0.773 ± 0.043) at predicting achievement of MCID for neck pain at both follow-up time points, with fair performance. The best precision was also demonstrated by logistic regression at 3 (0.724 ± 0.058) and 24 (0.780 ± 0.097) months. The best recall/sensitivity was demonstrated by multilayer perceptron at 3 months (0.841 ± 0.094) and by extra trees at 24 months (0.817 ± 0.115). Highest specificity was shown by support vector machine at 3 months (0.952 ± 0.013) and by logistic regression at 24 months (0.747 ± 0.18).
Conclusions: Appropriate selection of models for studies should be based on the strengths of each model and the aims of the studies. For maximally predicting true achievement of MCID in neck pain, of all the predictions in this balanced data set the appropriate metric for the authors' study was precision. For both short- and long-term follow-ups, logistic regression demonstrated the highest precision of all models tested. Logistic regression performed consistently the best of all models tested and remains a powerful model for clinical classification tasks.
Keywords: Quality Outcomes Database; cervical spondylotic myelopathy; machine learning; neck pain; patient satisfaction; patient-reported outcomes.