Data Quality Estimation Via Model Performance: Machine Learning as a Validation Tool

Stud Health Technol Inform. 2023 Jun 29:305:369-372. doi: 10.3233/SHTI230508.

Abstract

In our recent study, the attempt to classify neurosurgical operative reports into routinely used expert-derived classes exhibited an F-score not exceeding 0.74. This study aimed to test how improving the classifier (target variable) affected the short text classification with deep learning on real-world data. We redesigned the target variable based on three strict principles when applicable: pathology, localization, and manipulation type. The deep learning significantly improved with the best result of operative report classification into 13 classes (accuracy = 0.995, F1 = 0.990). Reasonable text classification with machine learning should be a two-way process: the model performance must be ensured by the unambiguous textual representation reflected in corresponding target variables. At the same time, the validity of human-generated codification can be inspected via machine learning.

Keywords: Neurosurgery; artificial intelligence; classification; deep learning; machine learning; neurosurgical procedures.

MeSH terms

  • Data Accuracy*
  • Humans
  • Machine Learning*