Development of a Generic Workshop Appraisal Scale (WASC) for Organizational Health Interventions and Evaluation

Annemarie Fridrich; Georg F Bauer; Gregor J Jenny

doi:10.3389/fpsyg.2020.02115

Development of a Generic Workshop Appraisal Scale (WASC) for Organizational Health Interventions and Evaluation

Front Psychol. 2020 Aug 18:11:2115. doi: 10.3389/fpsyg.2020.02115. eCollection 2020.

Authors

Annemarie Fridrich¹, Georg F Bauer², Gregor J Jenny²

Affiliations

¹ Stiftung Patientensicherheit Schweiz, Zurich, Switzerland.
² Center of Salutogenesis, Division of Public and Organizational Health, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, Switzerland.

Abstract

This study presents the development of a generic workshop appraisal scale (WASC) for the evaluation of organizational health interventions. Based on the session evaluation questionnaire (SEQ) by Stiles (1980), we developed a short, generic 10-item scale with pairs of adjectives, covering five facets: comprehensibility, relevance, novelty, activation, and valence. Our study is based on N = 499 employees from four organizations who participated in 41 workshops and filled out an evaluation questionnaire on-site. The questionnaire contained the newly developed WASC, as well as items capturing satisfaction with the developed output and outcome expectancies. Results from confirmative factor analysis confirmed the hypothesized five-factor structure of the WASC. The factor structure was found to be nearly invariant across the four organizations, a result that needs to be replicated in larger samples. Analysis of intra-class correlations indicated that 25% of the variance in workshop appraisal can be explained at workshop level. Hereby, perceived relevance and novelty exhibited lower amounts of shared variance, indicating that corresponding workshop appraisals are influenced more by individual factors and less by group dynamics. Furthermore, results from mediation analysis revealed that participants' workshop appraisals were significantly related to their outcome expectancies, and that this relationship was mediated by output satisfaction. Again, the facets showed differential effects: Relevance and comprehensibility seem to contribute most to the total effect on outcome expectancy, followed by activation, whereas valence and especially novelty play a minor role. Taken together, participants' workshop appraisals - together with output satisfaction and outcome expectancy - may be helpful for monitoring the implementation process and allow for corrective action if necessary.

Keywords: evaluation research; intervention research; organizational health intervention; outcome expectancy; process appraisal; process evaluation; scale development.