Virginia Tech® home

Identifying Trends in Training Needs of Food Safety Professionals in Virginia

ID

FST-448NP

Authors as Published

Molli Coleman, Undergraduate Student, Department of Food Science and Technology, Virginia Tech; Joell Eifert, Extension Specialist, Food Science and Technology, Virginia Tech; Alexis M. Hamilton, Assistant Professor and Extension Specialist, Department of Food Science and Technology, Virginia Tech

Introduction

The Food Safety Modernization Act (FSMA) was passed in 2011 and included the Preventive Controls for Human Food (PCHF) Rule (21 CFR Part 117). Under this rule, covered food businesses must employ at least one individual who has successfully completed a food safety training that the Food and Drug Administration (FDA) has deemed adequate or otherwise have sufficient job experience to develop and apply a food safety system (21 CFR §117.3).

Currently, the only recognized training course to meet this regulatory requirement is the Food Safety Preventive Controls Alliance (FSPCA) PCHF course. Businesses continue to struggle to apply learned concepts to ensure regulatory compliance. To efficiently target extension and outreach efforts that support the continued regulatory compliance needs of Virginia food businesses, it is necessary to evaluate the short-term outcomes of the PCHF trainings to better understand gaps in food safety knowledge.

To evaluate the short-term impact of this training, the FSMA Southern Center for Food Safety Training, Outreach and Technical Assistance developed a voluntary assessment to be administered before and after the course to assess participant change in knowledge as a result of instruction. It covered food safety concepts addressed in the course (Table 1) and, in 2017, was refined from a 23- to a 15-question version.

Table 1: FSPCA PCHF course structure and connection to regulation.

Module Title (Number)

Relevant Section of Code of Federal Regulation

Introduction (1)

Part 117

Food Safety Plan (2)

117.126

Good Manufacturing Practices and Other

Prerequisite Programs (3)

117.10-110

Biological Food Safety

Hazards (4)

117.130(b)(1)(i)

Chemical, Physical, and

Economically Motivated Food Safety Hazards (5)

117.130(b)(1)(ii-iii)

Preliminary Steps in

Developing a Food Safety Plan (6)

N/A

Resources for Preparing

Food Safety Plans (7)

N/A

Hazard Analysis and

Preventive Controls Determination (8)

117.130-136

Process Preventive

Controls (9)

117.135(a)(1)

Food Allergen Preventive

Controls (10)

117.135(a)(2)

Sanitation Preventive

Controls (11)

117.135(a)(3)

Supply-chain Preventive

Controls (12)

117.405-475

Verification and Validation

Procedures (13)

117.135-165

Record-keeping

Procedures (14)

117.301-335

Recall Plan (15)

117.139

Regulation Overview (16)

Part 117

Methods

For every PCHF training hosted by Virginia Cooperative Extension in Virginia, pre-and post-tests were administered before and after trainings were delivered, respectively. Questions were categorized by module and topic area (Table 2). The surveys (n=480) were examined to assess the change in knowledge between pre- (244) and post- (236) test scores.

Metadata about the year the training took place (2017- 2022), the test version used (23- vs. 15-questions), and the format of the training (in-person, virtual, hybrid) were also collected. Modules and topics were sorted according to change in knowledge (low: <15%; high: >15%) and post-test score (low: <50%; high: >50%) before ranking. Rankings were assigned according to prioritized need, with highest priority given to modules and topics with low knowledge change and post-test scores and lowest priority given to those with high changes in knowledge and post-test scores. The data were analyzed using Crosstabs iQ in Qualtrics.

Table 2: Pre-/Post-test scores and change in knowledge by topic and test version

Module Topic (Number)

Pre-test

Post-test

Δ k1

Prerequisite Program (3)

44.2

60.0

15.8

Food Safety

Hazards (4)

19.0

27.1

8.1

Food Safety Hazards (4)

27.0

44.5

17.5

Food Safety Hazards (4)

44.8

83.2

38.4

Hazard

Analysis (4)

48.5

60.0

11.5

Food Safety Hazards (5)

36.2

57.4

10.8

Hazard

Analysis (8)

46.6

57.4

10.8

Preventive Controls (8)

11.7

37.4

25.7

Corrective Actions (9)

21.5

35.5

14.0

Hazard

Analysis (11)

17.2

43.9

26.7

Food Safety Plan (12)

49.1

69.0

19.1

Validation (13)

21.5

42.0

20.5

Recordkeeping (14)

16.0

33.5

17.5

Recalls (15)

85.9

91.0

5.1

Δk1 = change in knowledge.

Results

Between 2017-2022, 244 pre-tests and 236 post-tests were taking in Virginia. In 2017, 81 pre- and 81 post- tests were taken on the 23-question test version and 18 pre- and 16 post-tests were taken on the 15-question version. After 2017, only the 15-question test was used. From 2018-2022, 145 pre- and 139 post-tests were collected.

The test did not include questions from each module (Table 2), and questions were not distributed evenly among the modules represented (Figure 1).

This is a pie chart describing the distribution of food safety topics by test question.
Figure 1: Distribution of questions on 15-question test.

On average, 3 additional questions were answered correctly on the 23-question post-test (57.96%), and 3 additional questions were answered correctly on the 15-question post-test (54.87%) when compared to the pre-test. Given that (i) the number of questions answered correctly were similar across both test versions, (ii) the identical question content (except for the 8 questions that were removed), and (iii) the greater quantity of 15-question tests, the following analysis and recommendations are based on the 15-question test version only. For the 8 questions that were not carried over to the 15-question test, a short summary of those findings is included at the end of this section.

Overall, modules 3 (Good Manufacturing Practices and Other Prerequisite Programs), 12 (Supply-Chain Preventive Controls) and 15 (Recall Plan), resulted in consistently high scores ranging from 60.0-100.0%. Modules 5 (Chemical, Physical, and Economically Motivated Food Safety Hazards), 11 (Sanitation Preventive Controls), and 14 (Record-keeping Procedures all had consistently low scores ranging from 33.5-48.4% on both test versions. Modules 9 (Process Preventive Controls), 11, and 14 resulted in low changes in knowledge ranging from 14.0-19.9% and low post-test scores ranging from 33.5-43.9%. Knowledge change also varied within and between modules. For example, questions addressing module 4 content resulted in both low (8.1%) and high (11.5- 38.4%) changes in knowledge, depending on the question (Table 2).

Questions and their associated topics were then ranked based on priority, with 3 questions identified as high, 9 questions identified as medium, and 3 questions identified as low (Figure 1). High priority topics included corrective actions, while prerequisite programs, and food safety plans were low priorities. Other topics like hazard analysis, preventive controls, recalls, recordkeeping, and validation, while important, were identified as medium priority topics. Interestingly, food safety hazards were simultaneously identified as high, medium, and low priorities. Given that understanding of this topic was not consistently satisfactory (high post-test score), it is recommended that additional training, resources, or other guidance is provided to scaffold current professionals’ knowledge to enhance food safety outcomes.

This is an image detailing the topics for which additional resources, training efforts, etc. are need from high to low priority.
Figure 2: Prioritization of food safety educational topics.

Of the 8 questions that were not addressed on the 15- question test, food safety plans were identified as a potential high priority need based on post-test and knowledge change alone.

Discussion and Conclusions

Since the PCHF Rule, there have been improvements in Virginia professionals’ food safety knowledge. Despite these improvements, it is apparent that there are still gaps in understanding of food safety concepts. Additional educational resources (e.g., fact sheets, trainings, and other professional development opportunities) are needed in order to help businesses better understand and identify food safety hazards and implement appropriate food safety practices for their businesses. It is recommended that Extension educators prioritize educational initiatives that address content related to food safety hazards and corrective actions, in addition to emphasizing these content areas more in PCHF trainings. It will also be important to continue to monitor both change in participant knowledge and post-test scores to adequately assess if proposed educational initiatives are truly fostering better food safety outcomes across Virginia.

References

21 CFR Part 117


Virginia Cooperative Extension materials are available for public use, reprint, or citation without further permission, provided the use includes credit to the author and to Virginia Cooperative Extension, Virginia Tech, and Virginia State University.

Virginia Cooperative Extension is a partnership of Virginia Tech, Virginia State University, the U.S. Department of Agriculture, and local governments. Its programs and employment are open to all, regardless of age, color, disability, sex (including pregnancy), gender, gender identity, gender expression, genetic information, ethnicity or national origin, political affiliation, race, religion, sexual orientation, or military status, or any other basis protected by law.

Publication Date

May 26, 2023