Process evaluation and challenges in collecting data from an after-school sports and literacy program in a diverse, low-income community

Abstract

Background: To understand how programs succeed we must evaluate them. The purpose of this paper was to examine the intervention and evaluation processes and identify areas of success and challenges in delivering and evaluating the Reflective Educational Approach to Character and Health (REACH) after-school program in a low SES community.

Methods: Sixty-six fifth and sixth graders from two schools participated. Data were collected at the intervention and control schools at four time points over the year. Data collected at baseline (Aug/Sep 2016), and Time 4 (May, 2017) were used for the analysis. Reach and fidelity to the intervention were assessed along with information on the process of data collection for variables of interest (e.g., accelerometer-measured physical activity level, FITNESSGRAM tests, vertical jump, and literacy). Missing data were imputed and a 2(School) by 2(Time) mixed factorial ANOVA was performed to examine the efficacy of the intervention.

Results: There were no significant time by school interactions for any of the outcomes assessed. There were several challenges in evaluating the program such as low compliance to wearing the accelerometer, lack of effort during literacy tests, peers affecting effort on physical tests, and missing data.

Conclusions: We posit that research implementation be re-examined in community-based research that is conducted with students at the elementary level in low-income communities. Future research should engage with youth in the data collection process to make it a meaningful experience for them and to possibly reduce attrition from the program.

Keywords: Fitness; Latino/a; Low-income; Physical activity; Title I; Youth development.