Measuring Research Capacity: Development of the PACER Tool

Stephen K Stacey; Melanie Steiner-Sherwood; Paul Crawford; Joseph W LeMaster; Catherine McCarty; Tanvir Turin Chowdhury; Amanda Weidner; Peter H Seidenberg

doi:10.3122/jabfm.2024.240085R1

Measuring Research Capacity: Development of the PACER Tool

J Am Board Fam Med. 2024 Nov 15:jabfm.2024.240085R1. doi: 10.3122/jabfm.2024.240085R1. Online ahead of print.

Authors

Stephen K Stacey¹, Melanie Steiner-Sherwood², Paul Crawford², Joseph W LeMaster², Catherine McCarty², Tanvir Turin Chowdhury², Amanda Weidner², Peter H Seidenberg²

Affiliations

¹ From the Department of Family Medicine, Indiana University School of Medicine, Bloomington, IN (SKS); Department of Family Medicine, Mayo Clinic Health System - Southwest Wisconsin region, La Crosse, WI (MS-S); Military Primary Care Research Network, Indiana University School of Medicine, Indianapolis, IN (PC); Department of Family Medicine and Community Health, Uniformed Services University, Bethesda, MD (JWL); Department of Family Medicine and Biobehavioral Health, University of Kansas School of Medicine, Kansas City, KS (CM); Department of Family Medicine, University of Minnesota Medical School, Duluth, MN (TTC); Department of Family Medicine, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada (AW); Department of Family Medicine, University of Washington, Seattle, WA, and LSU Health School of Medicine, Shreveport, LA (PHS). stacey.stephen@mayo.edu.
² From the Department of Family Medicine, Indiana University School of Medicine, Bloomington, IN (SKS); Department of Family Medicine, Mayo Clinic Health System - Southwest Wisconsin region, La Crosse, WI (MS-S); Military Primary Care Research Network, Indiana University School of Medicine, Indianapolis, IN (PC); Department of Family Medicine and Community Health, Uniformed Services University, Bethesda, MD (JWL); Department of Family Medicine and Biobehavioral Health, University of Kansas School of Medicine, Kansas City, KS (CM); Department of Family Medicine, University of Minnesota Medical School, Duluth, MN (TTC); Department of Family Medicine, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada (AW); Department of Family Medicine, University of Washington, Seattle, WA, and LSU Health School of Medicine, Shreveport, LA (PHS).

PMID: 39547812
DOI: 10.3122/jabfm.2024.240085R1

Abstract

Evaluating research activity in research departments and education programs is conventionally accomplished through measurement of research funding or bibliometrics. This limited perspective of research activity restricts a more comprehensive evaluation of a program's actual research capacity, ultimately hindering efforts to enhance and expand it. The objective of this study was to conduct a scoping review of the existing literature pertaining to the measurement of research productivity in research institutions. Using these findings, the study aimed to create a standardized research measurement tool, the Productivity And Capacity Evaluation in Research (PACER) Tool. The evidence review identified 726 relevant articles in a literature search of PubMed, Web of Science, Embase, ERIC, CINAHL, and Google Scholar with the keywords "research capacity" and "research productivity." Thirty-nine English-language studies applicable to research measurement were assessed in full and 20 were included in the data extraction. Capacity/productivity metrics were identified, and the relevance of each metric was data-charted according to 3 criteria: the metric was objective, organizational in scale, and applicable to varied research domains. This produced 42 research capacity/productivity metrics that fell into 7 relevant categories: bibliometrics, impact, ongoing research, collaboration activities, funding, personnel, and education/academics. With the expertise of a Delphi panel of researchers, research leaders, and organizational leadership, 31 of these 42 metrics were included in the final PACER Tool. This multifaceted tool enables research departments to benchmark research capacity and research productivity against other programs, monitor capacity development over time, and provide valuable strategic insights for decisions such as resource allocation.

Keywords: ADFM/NAPCRG Research Summitt 2023; Benchmarking; Bibliometrics; Capacity Building; Efficiency; Health Care Quality Indicators; Health Personnel; Leadership; Research Personnel; Resource Allocation; Systematic Review.