Studies have documented the use of EHMRs during N95 FFR shortages as far back as the H1N1 Influenza in 2009 (e.g., BREATHE, Department of Veterans Affairs 2009; Hines et al. 2017; Lawrence et al. 2017), revealing consistent advantages and disadvantages. Cost-efficiency in combination with environmental sustainability and supply chain resilience are consistent advantages referenced. Specifically, although EHMRs have a higher upfront cost their procurement has been found to be more cost-effective long-term (Chalikonda et al. 2020; Ramsdell et al. 2023). Additionally, stockpiled EHMRs can decrease organizational reliance on N95 FFRs, which are susceptible to shortages during surges in demand – as revealed among healthcare and first responder settings and other nationwide analyses (Alper et al. 2023; NASEM et al. 2019). Further, EHMRs can contribute to environmental sustainability by generating less waste, which has been raised as a global waste management system issue (Calma 2020). One study found that healthcare personnel (HCP) valued being able to minimize environmental impacts so long as their respirator is protective and functional (Venesoja et al. 2021).

Additionally, research has shown that HCP and first responders often perceive a higher sense of protection while using EHMRs than with N95 FFRs and, despite being less comfortable, they preferred EHMRs during higher-risk scenarios (Hines et al. 2019a; Ramsdell et al. 2023). However, EHMR use has been formidable to sustain, in part because employees prefer the convenience of disposable FFRs. Consequently, this preferred use has been found to wane once higher-risk scenarios subside, minimizing the perceived need for “extra” protection (Hines et al. 2017, 2019a; Brosseau et al. 2021).

Similarly, issues related to worker accountability around cleaning, disinfection, and maintenance of EHMRs have been identified as an ongoing barrier to sustainability regardless of organization size and type (Brown et al. 2018; Hines et al. 2019b, 2020, 2023; Brosseau et al. 2021; Thurman et al. 2023). In summary, individual-level factors such as employee accountability around cleaning and decontamination, storage, and perceptions of risk continue to dominate as barriers to sustainable EHMR practices (Hines et al. 2017).

As already noted, there are isolated studies of EHMR use in healthcare (Hines et al. 2021) and first responder (Tracy 2020) settings demonstrating individual- and organizational-level barriers to EHMR use. However, research to date has incompletely moved past the initial identification of barriers to present lessons learned and possible solutions that can be operationalized for widespread implementation through an EHMR-based RPP that is supported by management. Specifically, longitudinal data collected at the organizational level has been a gap in the research-to-practice literature. The longitudinal nature of research documenting EHMR experiences at the organizational level is imperative to track the variables that may impact not only adoption barriers but also maintenance of EHMRs at unique organizations of varying sizes and locations (NASEM et al. 2019).

To support a shift in the types of respiratory protection procured and implemented, it is necessary to assess organizational- and individual-level experiences over time. To the authors’ knowledge, few studies have aimed to understand the evolution of EHMR-based RPPs in healthcare environments (i.e., Hines et al. 2017; Brown et al. 2018). Although useful, these case examples did not identify and discuss specific RPP elements and practices as leading indicators to improve program effectiveness and sustainability. Leading indicators reflect workplace activities that are aimed at improving safety and health via a proactive management system (Reiman and Pietikäinen 2012). A lack of proactive, leading indicators (such as those entailed in an RPP) has been deemed a critical problem in occupational safety and health management (Laitinen et al. 2013).

The current study empirically identified challenges around establishing, implementing, maintaining, and ultimately sustaining an EHMR-based RPP in a variety of health delivery settings. Research questions were based on data collected from 42 organizations that participated in EHMR demonstration projects for one year:

What are the most important perceived elements and practices of an EHMR-based RPP to support a long-term program?

In 2021, the Strategic National Stockpile (SNS) sought to stockpile EHMRs to alleviate N95 FFR shortages. To inform a request for proposals from manufacturers, the National Institute for Occupational Safety and Health (NIOSH) and the SNS collaborated to draft and post a federal register notice (85 FR 56618, Vol. 85, No. 178) to ascertain the willingness of organizations to use EHMRs, experiences using EHMRs, and perceived concerns and advantages to their use (see Haas et al. 2021). However, a majority of the 22 public comments affirmed the need for EHMR designs without an exhalation valve (EV) or with an exhalation valve filter (EVF). This design feature was a relevant concern during the COVID-19 pandemic because of general source control in all environments especially in sterile work settings (Fernando et al. 2021).

Source control refers to the filtering of respiratory secretions to prevent the transmission of disease agents from the source person to others. Based on industry needs, some EHMR manufacturers redesigned respirators to protect the wearer and provide adequate source control to protect others (Greenawald et al. 2021). From the time this federal register notice was posted to when it closed, several NIOSH-approved EHMRs without an exhalation valve or with a filtered exhalation valve had come to market (Greenawald et al. 2021).

The SNS used the public comments to inform and solicit open proposals, eventually purchasing 375,000 EHMRs without exhalation valves or with exhalation valve filters from three manufacturers. Following public requests from U.S. health delivery settings, the SNS shipped approximately 96,511 EHMRs to 49 organizations. After these organizations received EHMRs from the SNS, points of contact who were overseeing distribution were invited to provide feedback about their organization’s experiences. Participation was voluntary and could consist of making an online survey available to employees who were fit tested for an EHMR, or for organizational leaders (hereafter referred to as points of contact [POC]) who oversaw aspects of fit testing, training, dissemination, or oversight, to participate in virtual interviews and an impact evaluation survey.

Forty-three organizations volunteered to participate. Cumulatively, these organizations received 52,291 EHMRs from the SNS. Of the 43 organizations, approximately 48% represented healthcare settings and 52% represented first responder settings. Further, approximately 23% of these organizations (n = 10) had previous experience using EHMRs in their setting and almost 77% (n = 33) had no organizational experience using EHMRs. Other data collected during the year-long project are published elsewhere (Haas et al. 2024a, 2024b, in press).

Using qualitative data collected (Haas et al. 2024a), researchers identified four programmatic elements and several practices within these elements that were consistent challenges for organizations throughout the year. The elements and practices were used to develop an impact evaluation survey in which POCs were asked to rank the challenges that were most important to address and overcome in supporting a long-term EHMR-based RPP. Specifically, respondents were asked to “Rank the overarching elements in terms of importance when it comes to sustaining an EHMR program in healthcare/first responder organizations.” Then, within each of the four elements, specific practices were listed that were again, requested to be ranked in order of importance based on experiences overseeing their organization’s EHMR program. The four elements are listed below with the number of practices that were assessed within the element in parentheses. Refer to Table 1 for a list of the practices within each element:

Access to EHMR models/designs (six practices)

After ranking the elements and respective practices, several open-ended questions were asked to better understand the challenges and suggestions to sustain EHMR-based RPPs. Respondents were also asked to share their job title and role in the EHMR demonstration effort, and more information about their organization, including the number of employees at their specific site and approximately, across their entire company. Specifically, during the qualitative data collected throughout the longitudinal study, references to organization size were referenced as a mechanism that influenced easier access to resources such as respiratory protection and fit test training, as well as communication that was developed and disseminated as a part of the EHMR rollout across participating sites (Haas et al. 2024a).

Therefore, the difference in site-specific and company-wide size may influence the structural gaps or proficiencies in roles, work processes, accountabilities, etc. just as the size of the employer may impact the level of resources necessary to develop, implement, and support a program long term. Consequently, it was important to understand the nuances of size at different levels of a company to ascertain if different barriers exist across different levels of organizational preparedness and response.

POCs were emailed a public survey link via CDC’s REDCap (Harris et al. 2009, 2019) between September 15 and November 15, 2022. The email also explained the purpose of the survey, that participation was voluntary, and that participants could exit the survey at any time without penalty. The survey took approximately 30 min to complete.

Researchers used the convenience sample of 43 hospitals, dental clinics, long-term care facilities, ambulatory units, and fire/police departments that received EHMRs from the SNS for this study. Five settings were unable to deploy the EHMRs but were still invited to participate in pre- and post-data collection, including this survey. There were 42 organizational responses (97.7% response) that represented positions ranging from frontline and managerial roles such as fire chief, health nurse, paramedic supervisor, and industrial hygienist to more executive and director roles. Specifically, 13 respondents reported frontline roles in the fire services including assistant/deputy chief or chief, captain, sergeant, or fire marshal; 11 reported director or vice president-level positions such as director of employee health, clinical operations, system infection prevention, environmental, health, and safety, supply chain, or ambulatory care; seven indicated they were a manager in some capacity ranging from managing industrial hygiene to environmental safety and risk; six reported being frontline roles including paramedics and registered nurses; and five reported serving in a coordinator or liaison role for emergency response, risk, safety, or environmental engineering.

As reported earlier, approximately 23% of the participating organizations had previous experience using EHMRs in their settings. Of the respondents who participated, representing their organization, however, 31% (n=13) indicated they had previous experience implementing and overseeing the use of EHMRs among employees and 69% (n=29) did not have this experience. This means that some respondents may have had experience using EHMRs at a previous job.

Before completing analyses, researchers checked the data with attention to any differences between the participating healthcare and first responder settings. Our t-tests showed that the elements and practices were not statistically significantly different from each other (adjusted p<0.0125) in relation to the mean perceived, ranked importance. Consequently, one aggregate group was retained for this study. In addition, a univariate analysis of variance (ANOVA) showed no statistically significant difference between the EHMR model(s) that organizations reported receiving vs. the rankings they provided. So, future analyses examining the EHMR model as a possible covariate were not performed.

For the first question, which was to understand the most important elements and respective practices within those elements, researchers calculated the average (i.e., mean [M]) score for each element and practice. For the second question, which was to determine if organizational size—as measured by (1) employees at their specific site and (2) employees at the entire company across locations—had an impact on the perceived importance of the elements and practices, ANOVA tests were completed. Variances were checked for homogeneity using Levene’s test. Games-Howell post hoc comparisons were performed and used for non-significant Levene’s test results. Post hoc analyses using Games-Howell indicated where the different sizes were statistically significantly different from each other, if applicable. Data analysis was performed using IBM SPSS 26.

Respondents were also asked to provide what role(s) they had in their organization’s EHMR demonstration project throughout the year. Examples provided were:

RPP administrator including education and updates made to the program.

Respondents were asked how many individuals were fit tested to an EHMR throughout the year. Researchers compared the number reported with the initial amount that was requested. Approximately 31% of the EHMRs that were shipped from the SNS were assigned to employees after they were fit-tested.

Respondents also reported what type of respirator was currently being used as their primary form of respiratory protection. At baseline, there were no organizations primarily using EHMRs. Of the 42 responses in the one-year survey, 66.7% (n=28) were primarily using N95 FFRs; 21.4% (n=9) primarily using EHMRs; 9.5% (n=4) primarily using something else; and 2.4% (n=1) were using no respiratory protection. Respondents were also asked to report employee perceptions of EHMRs, based on their own observations. Of the 42 responses, 42.9% (n=18) observed that the EHMRs were more comfortable than employees expected; 16.7% (n=7) observed the EHMRs were less comfortable than they expected; and 40.5% (n=17) observed that the comfort level was what employees expected.

Organization size was ascertained by asking 1) At just your specific location where EHMRs were distributed, what is the total number of persons who work there? Responses were categorized into three sizes. First, 50% of the organizations (n=21) reported having 1–99 employees, which was designated as a small organization; 21.4% (n=9) employed 100–999 employees, designated as medium size; and large organizations were categorized as employing more than 1,000 employees, which was 28.6% (n=12) of the sample.

First, looking at the overall ranks of the four elements, sustaining an EHMR program was considered the most important area to focus future efforts (M=2.94; SD = 1.12 on a 4-point scale), followed by daily maintenance of the program (M=2.72; SD = 0.974), development and implementation of the program (M=2.42; SD = 1.05), and access to EHMRs (M=1.91; SD = 1.11), respectively. Table 2 shows the averages for each element based on organizational size (i.e., small, medium, and large).

A one-way ANOVA identified differences in rankings among the four elements based on organizational size for both the specific site location where deployment occurred and based on the total number of employees across the company’s locations. Table 3 shows that ranking for “Access to EHMR models/designs” was statistically significantly different among the organization categories based on size.

Post hoc analyses using Games-Howell showed that large organizations were statistically significantly different when compared to small- and medium-sized organizations (small: p=0.001 and medium: p=0.015). In other words, large organizations did not perceive their ability to obtain and make EHMRs available as an important challenge that needed to be addressed when compared to smaller- and medium-sized organizations.

Respondents rank ordered practices within each element based on perceived importance to sustain an EHMR program. The ranks for each set of practices within the four elements are shown in Table 1. The practices are listed by ranking—rather than averages—with “1” indicating the most important. The ranking is presented rather than averages to aid interpretation. Also, significant ANOVA results followed by post hoc analyses indicated very few statistically significant differences among small, medium, and large organizations when comparing the rankings, minimizing the need to report averages and standard deviations for all practices. See the supplementary information for the practice averages that informed the ranking list in Table 1.

First, regarding access to EHMR models and designs, having access to EHMRs that are comfortable was ranked the most important factor. However, EHMR durability and the ability to secure replacement parts for EHMRs also ranked high. Second, when considering practices to support the development of an EHMR program, respondents felt that time to adequately fit test employees was most important; however, the second most important practice was the “commitment of the organization to ensure safety and wellbeing.” Next, respondents ranked the ability of the workforce to overcome design-specific EHMR barriers to use such as communication and moisture build-up as the most important aspect for organizations trying to maintain a program. Ensuring individual efficacy around using EHMRs, as well as the ability of organizations to overcome cultural barriers to EHMR use, were the second- and third-ranked areas of importance. Last, individual user accountability of EHMRs including proper cleaning and disinfection was ranked most important to support program sustainability. Organizational support for EHMRs as routine respiratory protection was the second most important practice for program sustainability.

User accountability was perceived as the most important challenge among respondents in sustaining their organization’s EHMR program. Specifically, respondents recorded sentiments about the difficulties in “holding staff accountable for disinfection and maintenance” (Organization 40). These results support previous research highlighting user accountability as a consistent challenge (Brown et al. 2018; Hines et al. 2019a; Brosseau et al. 2021; Hines et al. 2023; Thurman et al. 2023). However, the open-ended feedback also noted the importance of moving beyond individual cleaning and disinfection toward a more systems level of maintenance – much like what other large healthcare systems have done in central sterile processing (i.e., Hines et al. 2021).

Other respondent feedback noted the need to better “Train key personnel who can assist with group buy-in and developing specific types of messaging to enhance user accountability” (Organization 39). Previous efforts showed that employee onboarding that opted to fit test using EHMRs rather than N95 FFRs was perceived to improve knowledge and expectations around EHMR use (Haas et al. 2023). Therefore, it is possible that not only improving the quality of training but strategically providing EHMRs to employees as a part of the culture, may help program sustainability and showcase the organization’s support for EHMRs – which was another high-ranking practice in this element. Specific to organizational buy-in and support, research (Haas et al. 2024a, 2024b) has shown that effective communication from leadership can encourage EHMR adoption and employee accountability. Therefore, if leadership support is an element that can be improved to support workers, a primary challenge remains how to identify and mitigate other organizational or institutional-level barriers.

As discussed earlier, during higher-risk scenarios, HCP and first responders have typically preferred EHMRs (Hines et al. 2019a; Ramsdell et al. 2023). Other research has shown that perceived risk toward exposure influences decision-making around respiratory protection as well as organizational reinforcement of PPE use (Chu et al. 2021; Haas et al. 2022). To help combat the ebb and flow of perceived risk and subsequent respirator use, organizations may need to develop messages that speak to overall employee health. Some respondents noted the value of reiterating the importance of respiratory protection to prevent exposure to other viruses that may result in a typical cold (Ramsdell et al. 2023a). In other words, EHMRs can be used anytime, not only during a pandemic or public health emergency. The availability of EHMRs may help send the message that employee health is paramount for frontline workers to remain protected if respirator shortages occur.

However, changes in perceived risk can also influence organizational decision-making and support for EHMRs—ranked as the second most important practice within the sustainability element. Respondents noted in their feedback that the execution of their RPP and EHMR program “became laxer over time”—which supports the need for organizational support as an important practice within program sustainability (Organization 28). Consequently, ensuring that procedures and processes are proactive and risk-based vs. more prescriptive may enhance an EHMR-based RPP. Examples provided in the survey included situating EHMRs in their RPP before supply chain issues occurred, practicing with these EHMRs in similar scenarios to accurately predict communication issues, and proactively getting leadership buy-in that could enhance daily oversight of the program.

The third-ranked practice within program sustainability was standardization and predictability of the respiratory supply chain. Several respondents noted the reactive nature of their organization based on the availability or lack of supplies. Many respondents felt that if EHMRs were stockpiled, the uptake by organizational leadership and employees would have been better – some indicating that EHMRs would be the go-to PPE among frontline workers during routine operations. As one respondent wrote in the survey feedback, “We will now always fit test and stockpile EHMRs” (Organization 21). Notably, recommendations to stockpile EHMRs have been made during previous pandemics and it has been shown that rapid EHMR training and fit testing is feasible during public health emergencies (Shaffer and Janssen 2015; Pompeii et al. 2020).

Other researchers have made similar arguments, noting that, even if the regular use of EHMRs does not occur during routine times, introducing and perhaps fit-testing reusable respirators into health settings now, along with stockpiling, can help increase resiliency (Hicks et al. 2021). Fortunately, several large hospitals have completed widespread distributions of EHMRs and developed implementation guidelines for organizations to reference (Chalikonda et al. 2020; Hines et al. 2021). Smaller organizations may not have the capacity to overhaul work processes and update all aspects of an RPP but can scale their efforts over time. Based on the barriers that organizations encountered throughout this study from procurement to implementation, these results provide the opportunity for those overseeing RPPs or their organization’s supply chain to take a proactive approach to ensure not only worker health but future pandemic preparedness. A plethora of research discussed throughout this paper has provided resources and support for integrating EHMRs but has also shown the value of adopting a stockpile of EHMRs for pandemic preparedness.

Large organizations ranked the availability or ease of obtaining EHMRs as being statistically significantly less important than small- and medium-sized organizations. Even though comparing rankings based on organization size did not reveal many statistically significant differences, the differences in element averages (refer to Table 2) indicates that this factor may impact the ability to effectively procure, develop, implement, and sustain an EHMR-based RPP. Several respondents indicated in open-ended feedback that limited personnel and staff stifled their internal capacity to provide fit testing as required in the RPP. This finding aligns with other research showing that smaller organizations lack resources to devote to non-production-related activities (Champoux and Brun 2003; Hasle and Limborg 2006).

Additionally, five organizations were unable to deploy EHMRs received from the SNS – these organizations were comprised of rural hospitals or fire departments that did not have “access to proper medical clearance evaluations on-site or the ability to procure additional fit testing supplies” (Organization 86). Specifically, the feedback from these organizations unable to deploy EHMRs showed that resources beyond merely procuring EHMRs and undergoing fit testing with employees are needed. Rather, compliance with all aspects of OSHA’s respiratory protection standard (29 CFR 1910.134) such as employee medical clearance to use tight-fitting respirators; procedures for cleaning, disinfecting, storing, inspecting, and maintaining respirators; to ongoing training and annual fit testing are all aspects of an RPP that help ensure effective respirator use. These findings demonstrate the need to ensure equitable availability of PPE, supplies, and even resources such as training programs, across public health systems.

Alternatively, in the current study respondents who represented large organizations ranked aspects of program maintenance and sustainability as more important, although there was no statistically significant difference among the sizes. Larger organizations often have higher rates of employee participation in safety and health committees (Liu et al. 2010); however, employee participation often entails more routine policies and procedures, which can make it harder to sustain employee engagement over time (Wachter and Yorio 2014; Haas et al. 2022). An example in the current study would be the lack of ongoing participation in user disinfection and maintenance of EHMRs – requiring the need for Central Sterile Processing at the organizational level.

Similarly, safety and health management in large companies is often more complex than in small ones (Brooks 2008). These differences may be why smaller organizations are often able to communicate more effectively or develop proactive approaches disseminated through personal contact (Wachter and Yorio 2014). Consequently, large organizations may need to take a more conscious, proactive approach to encouraging workers’ participation in RPPs, including identifying and reminding employees of tasks and procedures that require respiratory protection.