Introduction: The Patient Outcomes in Simulation Education network has developed tools for the assessment of competency to perform the infant lumbar puncture (ILP) procedure. The objective of this study was to evaluate the validity and reliability of these tools in a simulated setting.
Methods: We developed a 4-point anchored global rating scale (GRS) and 15-item dichotomous checklist instrument to assess ILP performance in a simulated environment. Video recordings of 60 subjects performing an unsupervised lumbar puncture on an infant bench top simulator were collected prospectively; 20 performed by subjects in each of 3 categories (beginner, intermediate experienced, or expert). Three blinded, expert raters independently scored each subject's video recording using the GRS and checklist instruments.
Results: The final version of the scoring instruments is presented. Across all subject groups, higher GRS scores were found with advancing level of experience (P < 0.01). Total checklist scores were similar between the expert and intermediate experienced groups (P = 0.54). Both groups scored higher than the beginner group on the checklist instrument (P < 0.01). For each rater, a significant positive correlation was found between GRS scores and total checklist scores (median ρ = 0.75, P < 0.01). Cronbach α coefficient for the checklist was 0.77. The intraclass correlation coefficients between raters for the GRS and total checklist scores were 0.71 and 0.52, respectively.
Conclusions: This study provides some initial evidence to support the validity and reliability of the ILP-anchored GRS. Acceptable internal consistency was found for the checklist instrument. The GRS instrument outperformed the checklist in its discriminant ability and interrater agreement.