Subjective and objective quantification of physician's workload and performance during radiation therapy planning tasks

Lukasz M Mazur; Prithima R Mosaly; Lesley M Hoyle; Ellen L Jones; Lawrence B Marks

doi:10.1016/j.prro.2013.01.001

Subjective and objective quantification of physician's workload and performance during radiation therapy planning tasks

Pract Radiat Oncol. 2013 Oct-Dec;3(4):e171-7. doi: 10.1016/j.prro.2013.01.001. Epub 2013 Feb 21.

Authors

Lukasz M Mazur¹, Prithima R Mosaly², Lesley M Hoyle², Ellen L Jones², Lawrence B Marks²

Affiliations

¹ Department of Radiation Oncology, University of North Carolina, Chapel Hill, North Carolina. Electronic address: lmazur@med.unc.edu.
² Department of Radiation Oncology, University of North Carolina, Chapel Hill, North Carolina.

PMID: 24674415
DOI: 10.1016/j.prro.2013.01.001

Abstract

Purpose: To quantify, and compare, workload for several common physician-based treatment planning tasks using objective and subjective measures of workload. To assess the relationship between workload and performance to define workload levels where performance could be expected to decline.

Methods and materials: Nine physicians performed the same 3 tasks on each of 2 cases ("easy" vs "hard"). Workload was assessed objectively throughout the tasks (via monitoring of pupil size and blink rate), and subjectively at the end of each case (via National Aeronautics and Space Administration Task Load Index; NASA-TLX). NASA-TLX assesses the 6 dimensions (mental, physical, and temporal demands, frustration, effort, and performance); scores > or ≈ 50 are associated with reduced performance in other industries. Performance was measured using participants' stated willingness to approve the treatment plan. Differences in subjective and objective workload between cases, tasks, and experience were assessed using analysis of variance (ANOVA). The correlation between subjective and objective workload measures were assessed via the Pearson correlation test. The relationships between workload and performance measures were assessed using the t test.

Results: Eighteen case-wise and 54 task-wise assessments were obtained. Subjective NASA-TLX scores (P < .001), but not time-weighted averages of objective scores (P > .1), were significantly lower for the easy vs hard case. Most correlations between the subjective and objective measures were not significant, except between average blink rate and NASA-TLX scores (r = -0.34, P = .02), for task-wise assessments. Performance appeared to decline at NASA-TLX scores of ≥55.

Conclusions: The NASA-TLX may provide a reasonable method to quantify subjective workload for broad activities, and objective physiologic eye-based measures may be useful to monitor workload for more granular tasks within activities. The subjective and objective measures, as herein quantified, do not necessarily track each other, and more work is needed to assess their utilities. From a series of controlled experiments, we found that performance appears to decline at subjective workload levels ≥55 (as measured via NASA-TLX), which is consistent with findings from other industries.