Development of qualified items for nursing education assessment: The progress testing experience

Bruna Moreno Dias; Lúcia Marta Giunta da Silva; Pedro Tadao Hamamoto Filho; Valdes Roberto Bollela; Carmen Silvia Gabriel

doi:10.1016/j.nepr.2024.104199

Development of qualified items for nursing education assessment: The progress testing experience

Nurse Educ Pract. 2024 Nov 7:81:104199. doi: 10.1016/j.nepr.2024.104199. Online ahead of print.

Authors

Bruna Moreno Dias¹, Lúcia Marta Giunta da Silva², Pedro Tadao Hamamoto Filho³, Valdes Roberto Bollela⁴, Carmen Silvia Gabriel⁵

Affiliations

¹ University of São Paulo, Ribeirão Preto College of Nursing, São Paulo, Brazil. Electronic address: brunamorenodias@gmail.com.
² Federal University of São Paulo, Paulista School of Nursing, São Paulo, Brazil. Electronic address: lsilva@unifesp.br.
³ Universidade Estadual Paulista, Botucatu Medical School, Botucatu, Brazil. Electronic address: pedro.hamamoto@unesp.br.
⁴ University of São Paulo, Ribeirão Preto Medical School, São Paulo, Brazil. Electronic address: vbollela@fmrp.usp.br.
⁵ University of São Paulo, Ribeirão Preto College of Nursing, São Paulo, Brazil. Electronic address: cgabriel@eerp.usp.br.

PMID: 39536428
DOI: 10.1016/j.nepr.2024.104199

Abstract

Aim: To analyze the psychometric characteristics of items in the nursing inter-institutional progress testing for the years 2019, 2021, 2022 and 2023.

Background: Progress testing is a validated method for evaluating professional undergraduate education, aimed at identifying knowledge gain in a continuous and progressive manner, with potential benefits for nursing education. However, for its results to be useful, the evaluation items used in the test must have good psychometric performance.

Design: A cross-sectional study.

Methods: A sample of 377 items (multiple-choice questions) was applied to 4678 students in four years of progress testing. The difficulty and discrimination indexes were analyzed using descriptive statistics, ANOVA and simple linear regression.

Results: The average difficulty index of the test items ranged between 0.39 and 0.46. The areas of child and adolescent health, women's health and adult health had the most difficult items, while the areas of management, mental health and public health had the least difficult items. Discrimination index ranged from 0.35 to 0.43. There was a difference between discrimination index between the years of application (p < 0.001), with a significant increase in the discrimination index (p < 0.001) in the trend analysis. Students in the final years showed lower levels of difficulty and discrimination when compared with students in the initial years, demonstrating that the test is easier and there is less dispersion of performance among students in the final years.

Conclusions: The items are not difficult and have good discrimination. A gradual annual increase in the discrimination index of the items was observed. This study provides useful information for the psychometric analysis and quality assurance of knowledge assessment items, both for the implementation of similar PT experiences and in the use of multiple-choice questions for other knowledge assessment purposes.

Keywords: Academic performance; Baccalaureate; Education; Formative feedback; Nursing; Nursing faculty; Nursing students; Universities.