The Generic Error Rating Tool: A Novel Approach to Assessment of Performance and Surgical Education in Gynecologic Laparoscopy

Abstract

Objective: Global rating scales are commonly used to rate surgeons' skill level. However, these tools lack granularity required for specific skill feedback. Recently, an alternative framework has been developed that is designed to measure technical errors during laparoscopy. The purpose of the present study was to gather validity evidence for the Generic Error Rating Tool (GERT) in gynecologic laparoscopy.

Design: Video recordings of total laparoscopic hysterectomies were analyzed by 2 blinded reviewers using the GERT and the Objective Structured Assessment of Technical Skills (OSATS) scale. Several sources of validity were examined according to the unitary framework of validity. Main outcomes were interrater and intrarater reliability regarding total number of errors and events. Further, surgeons were grouped according to OSATS scores (OSATS ≥ 28 = high performers and OSATS < 28 = low performers), and the number of errors and events was compared between groups. Correlation analysis between GERT and OSATS scores was performed. Lastly, error distribution within procedure steps was explored and compared between high- and low-performing surgeons.

Setting: University teaching hospital.

Participants: A total of 20 anonymized video recordings of total laparoscopic hysterectomies.

Results: Interrater and intrarater reliability was high (intraclass correlation coefficient >0.95) for total number of errors and events. Low performers made significantly more errors than high performers did (median = 49.5 [interquartile range: 34.5-66] vs median = 31 [interquartile range: 16.75-35.25], p = 0.002). There was a significant negative correlation between individual OSATS scores and total number of errors (Spearman ρ = -0.76, p < 0.001, and ρ = -0.88, p < 0.001, for raters 1 and 2, respectively). Error distribution varied between operative steps, and low performers made more errors in some steps, but not in others.

Conclusion: GERT allows for objective and reproducible assessment of technical errors during gynecologic laparoscopy and could be used for performance analysis and personalized surgical education and training.

Keywords: Practice-Based Learning and Improvement; Professionalism; Systems-Based Practice; error analysis; gynecology; laparoscopy; surgical education; technical error.