In 2014, the estimated rate of injuries for all private-sector workers due to workplace violence that resulted in days away from work was 1.7 per 10,000 workers, whereas the estimated rate for private-sector hospital workers was 8.9 per 10,000 workers, and 18.7 per 10,000 workers in nursing and residential care workers (BLS, 2015b). Even with these statistics highlighting the frequency and severity of workplace violence, the incidence is likely an underrepresentation of reality given that nurses often do not report workplace violence due to the belief that dealing with patient violence is a part of their job (Arnetz et al., 2015; Gillespie, Leming-Lee, Crutcher, & Mattei, in press; Sato, Wakabayashi, Kiyoshi-Teo, & Fukahori, 2013).

Nurses may experience both physical and non-physical consequences as a result of workplace violence. Physical outcomes related to nonfatal workplace violence typically include bruises/contusions, temporary discoloration/slap marks, cuts/lacerations/scratches/abrasions, and bites/punctures (Gerberich et al., 2004; Speroni et al., 2014). Physical outcomes are often self-treated but may require treatment from another health care provider, and in some cases, hospitalization (Gerberich et al., 2004).

Even if health care workers are not physically affected by workplace violence, there can be many psychological and emotional effects. Nurses have reported feeling incompetent, helpless, overwhelmed, anxious, frustrated, angry, irritable, and guilty after experiencing workplace violence (Gerberich et al., 2004; Lanza, 2006; Ryan et al., 2008). Furthermore, nurses who have experienced workplace violence also have indicated experiencing post-traumatic stress, lower job satisfaction, reluctance to care for patients, and deterioration of personal relationships (Eric, 2011; Laschinger & Nosko, 2015; Laschinger, Wong, & Grau, 2012). These negative emotions may also have led to lower quality patient care (Arnetz & Arnetz, 2001; Laschinger, 2014), more absenteeism (Gerberich et al., 2004; Fujishiro, Gee, & de Castro, 2011), and increased costs for health care systems (Chapman & Styles, 2006; Speroni et al., 2014).

Workplace violence against health care workers continues to be an issue with limited attention. The majority of research, to date, has focused on the patient – not the nurse (Lanza, 2006). Although workplace violence training has been touted for more than a decade as essential to preventing workplace violence, many health care workers have not been receiving this type of education (Beech & Leather, 2006; Wassell, 2009), which is one reason why NIOSH developed an online course.

Although other workplace violence prevention trainings do exist, there are limitations associated with most. For example, some training is available only on-site in a presentational style, which is typically inconvenient for many nurses. Additionally, of those that are available online, they may be restricted in other ways, such as limited availability (e.g., membership in an organization), no continuing education units, expense, and general content not specific to nurses.

NIOSH created an online course to raise awareness of the problem and to provide instruction on workplace violence prevention techniques (Hartley, Ridenour, Craine, & Costa, 2012). Through interactive video real-life cases and textual content, participants receive information about personal safety measures as well as violence recognition cues.

The online course is segmented into 13 units with 20-minute modules that address the following areas:

Upon completion of the free course, nursing professionals should have acquired an awareness of the problem of workplace violence as well as increased knowledge of how to prevent and de-escalate violence in their own workplaces.

Results from a focus group discussion of nine nurses in Boston, Massachusetts, suggested that the course was useful, and the nurses provided feedback for minor course revisions. One particularly salient recommendation was to collaborate with nursing schools to include the course as part of their curriculum (Hartley et al., 2014). NIOSH researchers were already considering this very issue, so a pilot study was launched to determine the course’s viability to serve in such a capacity. The primary objective of this study was to evaluate the overall effectiveness of the course at raising awareness and increasing knowledge about workplace violence with a nursing student population. The secondary objective of the study was to obtain input regarding suggestions for improving the course to encourage participation and enhance message retention among nursing students.

Nursing students enrolled in a Bachelor of Science in Nursing or Master of Science in Nursing program at a mid-Atlantic university at the beginning of the Spring 2014 semester were recruited for participation in this study. The principal investigator attended the nursing student orientation class the first week of the semester to distribute flyers and announce the study. Additionally, nursing faculty made announcements in their courses and posted study information on a nursing student listserv. Students were offered community service hours for participating in the study. Although 61 nursing students expressed interest in participating in the study, only 55 students completed the workplace violence prevention online course. The final survey sample included 48 students, resulting in a 79% participation rate (see Table 1 for course participation rates). Of those that left the study for various reasons, 92% were female, the majority were seniors, slightly below 70% were 21 years of age or younger, and 85% did not have any medical license (e.g. LPN, RN). The remainder of the study sample is representative of the study population. Forty-five students expressed interest in participating in the focus group discussions, and 40 students (33 females and 7 males) actually participated, resulting in an 89% participation rate. Four women and one man signed-up for a focus group discussion but did not attend.

A previously developed survey based on the principles of the Theory of Planned Behavior (Ajzen, 1985, 1991) measuring awareness and knowledge of workplace violence was presented to three nursing students currently enrolled at the study site (these students did not participate in the study) to assess readability and comprehension. Based on their feedback, minor modifications were made to three questions to enhance clarity. Then, after making revisions to the questionnaire and receiving approval from the Institutional Review Board, students who agreed to participate in the survey were given a link to an online survey that served as a pre-test to assess baseline awareness and knowledge. The cover letter stated that by clicking on the survey button link, they were providing informed consent for becoming a study participant. Participants were asked a series of demographic questions (e.g., age, sex, year in school, work experience) and were required to enter the last four digits of their student identification number so that data from all three collection points could be linked. Then, participants proceeded to the three awareness items and 10 knowledge questions. ¹ The first awareness statement was “Workplace violence is a problem for nurses” to which participants responded: “True,” “False,” or “I don’t know.” All of the remaining items included a question with a five-option multiple choice answer format. For example, one of the knowledge questions asked, “What is workplace violence?” Participants selected from four definitions and the common fifth alternative to all questions was “I don’t know.” This option was offered so that participants were not forced to guess at an answer if they did not know the correct response. The goal was to assess if the online course was a useful tool to raise awareness and increase knowledge about workplace violence with nursing students and how this course could be effectively utilized not only with nursing students but also with practicing nurses.

Correct answers to the three awareness questions were summed; therefore, scores on the awareness portion of the survey could range from 0 (completely unaware) to 3 (fully aware) for each participant. Likewise, correct answers to the 10 knowledge questions were summed, and therefore, scores on the knowledge portion of the survey could range from 0 (no knowledge) to 10 (very knowledgeable) for each participant.

Once students completed the pre-test survey, they sent an email to the principal investigator stating their completion of the pre-test survey. The principal investigator then sent students a link to the online course, which they were able to access at their leisure, preferably during a four-week time period. The course had start-stop functionality, which allowed participants to return to the course where they exited so that it was not necessary to complete the course in one sitting. NIOSH estimated that the course would take three hours to complete. After students completed the online course, they once again contacted the principal investigator to access the first post-test survey (mean time frame was 11 days from pre-test date). In addition to the awareness and knowledge questions, students also were asked to respond to three statements measuring usefulness that were based on a 5-point scale with anchors ranging from “None” or “Not at all” to “Most” or “Very” (e.g., “No new information,” “Not at all useful,” “Most information was new,” “Very likely”), one yes/no question regarding recommending the course to others, and three open-ended questions pertaining to most useful aspects, least useful aspects, and why useful/not useful. Finally, four weeks after completion of the immediate post-test survey, the principal investigator emailed participants a link to the second post-test survey, which asked respondents to answer the same post-test survey questions plus their application of course content, sharing of course material with others, and what portion of the course had the most impact.

Additionally, after the completion of the online course and first post-test survey, students were invited to participate in a focus group discussion to share ideas for course design, message retention, and participation enhancement.² Students discussed what they liked and disliked about the course, format and content changes they would make to the course, tips for presenting course material so it is easier to retain, and suggestions for encouraging and motivating health care providers and students to participate in the course. Five focus group discussions with a total of 40 participants were conducted. Two focus groups met on-campus in a private classroom (n = 9, n = 11, respectively). The conversations were audio-recorded and transcribed verbatim within two days after the conclusion of the focus groups. Three focus groups were conducted online using Chatzy (a private online chat room) (n = 10, n = 4, n = 6, respectively). Transcripts of discussions were printed. In total, 40 pages of single-spaced transcripts were used in data analysis.

At the conclusion of data collection, quantitative data from surveys were downloaded, cleaned, and entered into SPSS for statistical analysis. Descriptive statistics were used to characterize participants based on collected demographic information. Changes in awareness and knowledge at the three time points were assessed using one-way repeated measures ANOVAs.

Transcripts were analyzed qualitatively using a grounded theory and constant comparative methodology (Glaser & Strauss, 1967). First, the principal investigator open coded the data (Corbin & Strauss, 2015) to assess emergent themes in the data in relation to course design as well as strategies for message retention and participation enhancement. Then, the principal investigator engaged in axial coding to collapse open codes into larger thematic categories (Corbin & Strauss, 2015). Themes included two categories for enhancing participation: incentive and promotion. Incentive subthemes included continuing education units (CEUs), job/class advancement, and monetary/tangible reward. Promotion subcategories included specific types of print, electronic, and oral communication. Themes for message retention included content and format suggestions such as knowledge checks and interactive presentation. This coding process was used to create a codebook with detailed themes and subthemes, from which all the data were coded. Twenty percent of the data, twice the recommended minimum, was then coded by an independent coder to assess intercoder reliability (Lombard, Snyder-Dutch, & Bracken, in press; Tracy, 2013). Scott’s pi (Scott, 1955) was an acceptable 0.91. Because the intercoder reliability was higher than the recommended minimum of 0.70, the standard practice of one researcher coding all of the data was utilized (Lombard et al., in press; Tracy, 2013)

The first aim of the study was to assess what changes in awareness occurred among nursing students after completing NIOSH’s online workplace violence prevention course. The mean awareness scores differed significantly between pre-course and both post-course time points (Wilk’s λ = 0.319, F(2, 46) = 49.01, p < .001). Post hoc tests using the Bonferroni correction revealed that course participation increased awareness of workplace violence from pre-course scores (M = .75, SD = .438) to immediate post-course (M = 2.13, SD = .789) and four-week post-course (M = 1.96, SD = .771) scores on a 3-item measure (see Table 3). Although mean awareness scores decreased slightly comparing the immediate post-course score to the four-week post-course score, the increase from pre-test to four-week post-course was still significant. Participants were fairly unaware of workplace violence in health care settings prior to course participation.

The second aim of the study was to assess what changes in knowledge occurred among nursing students after completing NIOSH’s online workplace violence prevention course. Mauchly’s test indicated that the assumption of sphericity had been violated, X²(2) = 17.08, p < .001; therefore, degrees of freedom were adjusted using the Huynh-Feldt correction (ε = 0.784). The mean knowledge scores differed significantly between pre-course and both post-course time points (Wilk’s λ = 0.495, F(1.57, 73.66) = 37.26, p < .001). Post hoc tests using the Bonferroni correction revealed that course participation increased knowledge of workplace violence from pre-course scores (M = 6.65, SD = 1.45) to immediate post-course (M = 8.56, SD = 1.32) and four-week post-course (M = 8.19, SD = 1.42) scores on a 10-item measure (see Table 3). Similar to the awareness scores, although mean knowledge scores decreased slightly comparing the immediate post-course score to the four-week post-course score, the increase from pre-test to four-week post-course was still significant.

The third aim of the study was to assess what changes could be made to NIOSH’s online workplace violence prevention course to enhance participation and message retention. Results revealed that the course was well-received and participants liked the course’s content (e.g., Madison³ stated, “I thought the course was very informative”), format (e.g., Debbie shared, “I liked how it was online and you could start and stop at your own pace and that made it a lot easier to complete”), and accessibility (e.g., Jeff highlighted, “Easy access. You could walk away from it and come back to complete”). In fact, participants overwhelmingly recommended that the course be offered to incoming nursing students to increase exposure to the information before working in a health care setting. Still, participants provided several suggestions for increasing participation not only among fellow nursing students but also among practicing nurses.

Because students completed the course at their own pace, there is no way of knowing if other information may have influenced the participants (e.g., other training, communicating with peers who were participating). Similarly, there is no way of assessing how participating in the focus group discussions may have affected awareness and retention of the course content. Meeting to discuss the course before the four-week follow-up may have enhanced retention of information. Because the survey data were collected anonymously, comparison of data between those who participated in the focus group discussions with those who did not is not possible. Finally, the small sample size is a limitation for generalizability.

This study can serve as a starting point for future research in this area. Because workplace violence among nurses is a growing problem, it is imperative to find ways to address this issue for the health and safety of all involved. Continued assessment is likely beneficial. For instance, a next step could be to evaluate the questions that course participants can answer to receive CEUs if they are working professionals. Additionally, it is important to assess this course in other nursing student populations. For example, a few of the participants had already completed a “lower” level nursing program (e.g., CNA, LPN). It may be beneficial then to begin even earlier and at different levels because all levels and types of health care providers are at risk. It may also be useful to assess the workplace violence course with other affiliated health care professionals to assess transferability of the content.

Finally, besides assessing this course with additional and different health care providers, it is very important to conduct an even longer longitudinal study to assess behavior. Although it is valuable to recognize knowledge increases, it is more beneficial to assess if this increased knowledge actually leads to behavior change. Then, we will be able to determine if the course is effective in reducing workplace violence.