Bridging the Divide Between Evaluation Training and Practice for Public Health Professionals

By Bree Hemingway

With limited resources in public health, program evaluation is an important tool to ensure that a program is not only effective and efficient— but that it is equitable. It is imperative that public health professionals are trained properly on the foundations of evaluation practice as inadequate training could lead to ineffective practice. In 2020, we conducted a mixed-methods study of public health professionals in the United States to understand how they utilize evaluation in their work. The study, consisting of an online survey with 89 public health practitioners and follow-up interviews with 17 of the survey respondents, aimed to understand the extent to which Master of Public Health (MPH) graduates engage in evaluation on the job, to learn how MPH graduates implement evaluation, and to hear from MPH graduates about how their academic training prepared them for the evaluation work they perform. 

My own experience with program evaluation in various public health settings has piqued my research interest in understanding how public health students are prepared to use program evaluation in practice. Current research on evaluation practice showed that although program evaluation is considered an essential practice in the field of public health, it is not implemented consistently in the field. We hoped that this mixed-methods study would help provide a better understanding of how other MPH graduates are using program evaluation in their careers.

Survey participants were recruited through public health organizations including the Society of Public Health Educators, state and regional public health associations, public health schools, and social media networks. We used purposive sampling to select interview participants so that we would have a diverse sample of individuals representing a broad range of evaluation training, experience with evaluation, and employment types.  

Framework for program evaluation in public health. Source: https://www.cdc.gov/evaluation/framework/index.htm

We designed both the survey and the interview around the Centers for Disease Control and Prevention Framework for Evaluation in Public Health (CDC Framework) because it is commonly used in public health. In the survey, we asked participants to approximate time spent on evaluation activities in their current position, their role in the activity, how well their MPH program prepared them for evaluation activities within each step, and their confidence with each step. We also asked their familiarity with the professional association for evaluators in the United States, exposure to evaluation in their coursework, graduation year, concentration, current employer type, and the total amount of time spent conducting evaluation.

In the interviews, participants were asked to discuss specific examples of their evaluation experience and reflect on their academic preparation. Participants identified which evaluation skills they felt they did or did not have when they entered the workforce and shared what resources they used to strengthen their capacity.

After analyzing the survey data and the interviews, we found that MPH graduates participate in evaluation activities related to all six Centers for Disease Control and Prevention framework steps— 1. engage stakeholders, 2. describe the program, 3. focus the evaluation design, 4. gather credible evidence, 5. justify conclusions, 6. ensure use and share lessons learned. In addition to these activities, MPH graduates engage in evaluation capacity building, evaluating for health equity and social justice, and funding activities. During the interviews we were able to get a better understanding of how participants were engaged in evaluation beyond the six steps of the CDC evaluation framework. Interview participants described being formally or informally responsible for building evaluation capacity within their organization— including serving as an internal consultant to advise other on evaluation projects.  Many also described the importance of health equity and social justice as an outcome of their evaluation efforts. Not surprisingly, nearly all interviewee participants discussed grant seeking, grant writing, and grant reporting activities and identified grant funding as a specific motivating factor for program evaluation.  

Participants noted a disconnect between academic preparation and community practice, were least confident in focusing the evaluation design, and most often used surveys to collect data. Despite the prominence of evaluation in their work, half of the survey respondents had never heard of the primary professional association for evaluators in the United States—the American Evaluation Association (AEA).

Evaluation is a central component of public health practice.  It is critical that public health professionals commissioning, participating in, or leading evaluations are adequately equipped to perform well in these roles. Our study suggests that there is room for growth. Although the CEPH accreditation criteria does include competencies regarding conducting program evaluation, connecting the public health community with the broader professional practice of evaluation is important to advance public health evaluation practice.

Article Details
Aligning Public Health Training and Practice in Evaluation: Implications and Recommendations for Educators
Bree L. Hemingway, MPH, CHES, Sarah Douville, MBA, and Leslie A. Fierro, PhD, MPH
First published online September 22, 2021
DOI: 10.1177/23733799211033621
Pedagogy in Health Promotion

About the Author