Objective. To develop and establish validity for a grading rubric to evaluate diabetes subjective, objective, assessment, plan (SOAP) note writing on primary care (PC) advanced pharmacy practice experiences (APPEs), and to assess reliability and student perceptions of the rubric. Methods. Ten PC APPE faculty members collaborated to develop a rubric to provide formative and summative feedback on three written SOAP notes per APPE student over a 10-month period. Correlation analyses were conducted between rubric scores and three criterion variables to assess criterion-related validity: APPE grades, Pharmaceutical Care Ability Profile Scores, and Global Impression Scores. Inter-rater and intra-rater reliability testing were completed using Cohen's kappa and Intraclass Correlation Coefficients (ICC). Student perceptions were assessed through an anonymous student survey. Results. Fifty-one students and 167 SOAP notes were evaluated using the final rubric. The mean score significantly increased from the first to second SOAP note and from the first to third SOAP note. Statistically significant positive correlations were found between final rubric scores and criterion variables. The ICC for inter-rater reliability was fair (.59) for final rubric scores and excellent for intra-rater reliability (.98 to1.00). Students responded that the rubric improved their ability (84.9%) and confidence (92.4%) to write SOAP notes. Conclusion. The rubric may be used to make valid decisions about students' SOAP note writing ability and may increase their confidence in this area. The use of the rubric allows for greater reliability among multiple graders, supporting grading consistency.
Keywords: SOAP note; experiential education; reliability; rubric; validity.