The literature review covered NIOSH TWH materials and published research on the TWH framework, including research about TWH intervention strategies and training materials such as the Hierarchy of Controls (Anger et al., 2015; NIOSH, 2015; Sawhney et al., 2018; Schwatka et al., 2020; Sorensen et al., 2018). Certain features integral to the TWH framework already align with the SC literature (Flin et al., 2000; Zohar, 1980). These include open communication about the associated domain (be it safety, physical health, or mental well-being), the provision of resources and training per domain (such as personal protective equipment [PPE] and training regarding its proper usage for long-term health, or employee assistance programs and communicating their usefulness for well-being), and the removal of obstacles related to the domain regardless of productivity (such as competing demands between work and the domain). These consistencies across frameworks aided the initial item generation in which TWH experts and SMEs were recruited to assess theoretical saturation and propose dimensions at bi-weekly meetings for three months (6 meetings total).

During these meetings, we collected feedback regarding content validity of the two proposed sub-climate scales from 15 SMEs with expertise in relevant fields, including occupational health and safety experts, industrial engineers, TWH practitioners, and educators in academia, government, and the private sector. They were given the list of newly generated items alongside the definition of each climate domain and asked for their feedback regarding construct correspondence using a 10-point Likert scale, and to provide their thoughts on each item’s uniqueness, clarity, and construct distinctiveness in a written comment. This modified Hinkin and Tracey (1999) approach allowed us to gather their thoughts on our proposed multilevel climate while also giving SMEs an opportunity to assess potential deficiencies, which the traditional Hinkin and Tracey approach (Colquitt et al., 2019) does not require to evaluate content validity (e.g., the clarity and relevance of each item). Suggestions about the proposed dimensionality of the scale, specific definitions of safety, physical health, and mental well-being, and options for discrete items were solicited as well. Items were then altered or removed based on SMEs’ suggestions through an iterative discussion process with SMEs, utilizing their ratings and comments. This feedback was used to refine the quality of the initial sets of 27 and 29 items for PHC and MWC, respectively.

These items were then presented to 10 frontline employees through a cognitive interview process to assess readability, clarity, and appropriateness. Participants read each item aloud, provided their rating, explained their reasoning for said rating, and shared their opinion on how easily the item could be understood by their peers. The interview results were used to improve the items before our original panel of SMEs provided a final review. At the end of their review, quantitative ratings of content validity were given in order to assess the assumed utility and contribution of each item to the validity of the entire scale. This process, drawn from the Delphi Method (Gordon, 1994), culminated in a final approval rating per item. This led to the removal of more items in each sub-climate, resulting in a revised scale of 20 items for PHC and 21 items for MWC.

Consensus from our field and academic experts determined the scope of physical health and mental well-being under the TWH-c framework. The insights gathered through these interviews were instrumental in refining the definitions, providing clarity, and facilitating the development of a robust measurement tool for organizational culture and climate related to TWH. The implementation of the safety sub-climate, with a short introduction to interpret the associated prompt, was not altered due to precedent (Zohar & Luria, 2005; Zohar & Polachek, 2014). SMEs suggested that physical health should be considered as “how well your body is working” with examples including freedom from illness or injury. The PHC scale should inquire about how organizations support physical health through personal improvement and maintenance of physical health via exercise, diet, and sleep. Specific examples include providing healthy diet plans, discounted gym memberships, and information on proper sleep. Respondents were directed to focus on their company’s initiatives in supporting physical health with the prompt provided in Table 4. SMEs defined mental well-being as the representation of a person’s mental health and quality of life, with examples being the presence of positive emotions, high morale, and reduction of burnout and stress. Organizations can support this through various means, such as providing stress prevention programs, employee assistance, and designing work to avoid burnout. This feedback from SMEs helped guide our definition and survey prompt for MWC to focus on what organizations are doing to support employee mental well-being. The MWC prompt is also presented in Table 4. Our SMEs helped provide the scope for physical health and mental well-being under the TWH approach and advised how to clearly communicate this through prompts provided to survey respondents.

Sample 1 was recruited through a survey panel provider (Qualtrics) to obtain data from a population of workers from various industries. Participants from this panel were vetted in order to match the specifications of the project specifically as a general sample for EFA. Completion times below five minutes (one standard deviation below the mean response time), suspicious responses to open-ended items, failure on attention checks, and suspicious invariance (participants with rating variance < 0.20 were flagged for review; Lopez-Fernandez et al., 2018) were grounds for removal from the dataset. This level of fidelity was communicated with the panel provider during data collection to communicate the level of data quality necessary. The attention check followed the panel provider’s recommendation to be present in the first half of the survey (Appendix A).

Sample 1 included 788 anonymous participants for our general sample who met the criteria quoted above. The panel provider ensured that participants were from a variety of industries, at least 18 years of age, employed full-time, and compensated appropriately based on survey length. Of these participants, more than half identified as women (56%) and as white (69.4%), with ages ranging from 18–65 years (M = 44.80, SD = 15.06).

The other two samples were from a manufacturing company (Sample 2) and a utility company (Sample 3) located in the U.S. Pacific Northwest; all participants completed an IRB-approved informed consent process. Sample 2 included 376 participants with 66.2% identifying as men and 32.2% with ages ranging from 25–34 years. Most of this sample identified as white (58.5%) and had tenure with the company ranging from 1–5 years (45.7%). Sample 3 included 165 participants with 49.7% identifying as men and 27.9% in their 50s. Most of this sample also identified as white (73.3%), with 24% of this sample’s tenure ranging from 1–5 years with their company.

Datasets were cleaned by removing multivariate outliers and incomplete cases listwise. Mahalanobis distance cut-offs were 45.31 for PHC and 46.79 for MWC. Regarding assumption checking, Bartlett’s test of sphericity was significant for both PHC and MWC, and there was adequate sampling accuracy via the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (0.967 and 0.956, respectively). There was passable linearity, normality and homoscedasticity, and nothing to suggest additivity (no overlap in variables implied by redundant agreement or being too highly correlated) for both PHC and MWC datasets, as evidenced by the absence of significant departures from normal distribution and the presence of consistent variances across levels of the predictor variables within each climate domain.

EFA were performed on Sample 1 to examine the factor structures of both the PHC and MWC sub-climate scales. Oblique rotation was used, retaining factors with eigenvalues above 1.0. Items with substantial cross-loadings across multiple factors (> 0.40) or low factor loadings (< 0.40) were removed. Confirmatory factor analyses were then conducted using Samples 2 and 3 on both the PHC and MWC sub-climate scales to confirm the factor structures obtained from the general sample. Model fit was assessed with the χ² statistic, comparative fit index (CFI), Tucker-Lewis index (TLI), standardized root mean square residual (SRMR), and root mean square error of approximation (RMSEA), utilizing cut-off recommendations by Hu and Bentler (1999) (i.e., CFI and TLI > .90, SRMR and RMSEA < .08). Models were examined using the larger company sample first (Sample 2), before reaffirming with the smaller sample (Sample 3). After the models were confirmed for each sub-climate, second-order CFA was conducted on a combined dataset of both companies to see if a meaningful TWH-c emerged from combining SC, PHC, and MWC. A grouping variable for the company was included to maximize power. The model specified that Total Worker Health Climate would explain the relationships among Zohar’s full 16-item SC scale (Zohar & Luria, 2005) and the newly generated 14-item PHC and 10-item MWC sub-climate scales from the prior factor analyses.

For PHC, EFA analysis using Sample 1 showed a three-factor structure retaining 16 of the 20 original items that fit the data best. The first factor (Communication and Resources; six items) encompassed educating workers and offering programs to improve health. The second (Hygiene and Recovery; three items) addressed policies, resources, and enforcement of hygiene and injury-related recovery. The last factor (Commitment and Action; seven items) focused on the promotion of worker health and taking action to ensure health. For MWC, EFA procedures on Sample 1 displayed a single-factor structure retaining all 21 items as the best explanation for the data, as they all loaded onto one general factor encompassing mental well-being in the workplace. Table 1 and Table 2 present the eigenvalues, variance explained, and final EFA loadings for both PHC and MWC, respectively.

When the viable 16-item PHC model was assessed with CFA using Sample 2’s data, the model fit was reasonable (CFI = 0.94; TLI = 0.93; RMSEA = 0.09; SRMR = 0.05). However, improvements were suggested by modification indices and supported by theoretical considerations, including item redundancy and SME feedback. Using this information, items 6 and 16 were trimmed to enhance the model’s overall fit (CFI = 0.96; TLI = 0.95; RMSEA = 0.07; SRMR = 0.05). All fit indices for tested CFA models are presented in Table 3. This new 14-item model (see Table 4) also fits Sample 3’s data well (CFI = 0.97; TLI = 0.97; RMSEA = 0.07; SRMR = 0.03).

For MWC, all items were placed into a CFA model using Sample 2’s data, which yielded inadequate fit (CFI = 0.90 TLI = 0.88; RMSEA = 0.12; SRMR = 0.04). Items were trimmed using a combination of initial EFA loadings, CFA loadings, and modification indices, as well as theoretical considerations (interview and SME feedback, TWH literature, etc.) This helped remove additional redundancy and interrelation between items, yielding a 10-item scale with the following fit in Sample 2 (CFI = 0.99; TLI = 0.98; RMSEA = 0.07; SRMR = 0.02). All fit indices for tested CFA models are presented in Table 3. This 10-item scale (see Table 4) also fits well using the data from Sample 3 (CFI = 0.98; TLI = 0.98; RMSEA = 0.08; SRMR = 0.02).

Second-order CFA was conducted on a combined dataset of both companies (Samples 2 and 3) to see if a meaningful TWH-c would emerge. The model specified that TWH-c would explain the relationships among Zohar’s full 16-item SC scale (Zohar & Luria, 2005) and the newly generated 14-item PHC and 10-item MWC scales from the prior factor analyses. This yielded adequate fit (CFI = 0.92; TLI = 0.91; RMSEA = 0.07; SRMR = 0.05), justifying the combination of sub-climate scores into one equally weighted TWH-c aggregate, formed by an average of each sub-climate. These findings support our hypotheses of establishing a reliable measure of a meaningful combined TWH-c comprising safety, physical health, and mental well-being sub-climates. Final scales and their reliabilities can be found in Table 4.

To assess convergent validity, AVEs were calculated. PHC showed AVE values of 0.56 in Sample 2 and 0.67 in Sample 3, while MWC had AVE values of 0.74 in Sample 2 and 0.76 in Sample 3. These values exceed the threshold of 0.50 suggested by Fornell and Larcker (1981), indicating that the constructs explain a significant proportion of variance in their indicators. For comparison, the AVE of the established SC scale was 0.60 in Sample 2 and 0.66 in Sample 3, further supporting acceptable convergent validity.

Discriminant validity was examined by comparing the AVE for each construct to the square of its correlation (r²) with Diversity Climate (DC; McKay et al., 2007), a distinct construct focused on inclusivity and equity that was specifically collected to test its discrimination from our novel climate scales. For PHC (AVE = 0.56 in Sample 2 and 0.67 in Sample 3), r² values were 0.34 (r = 0.58) in Sample 2 and 0.41 (r = 0.64) in Sample 3, with AVE values exceeding r², supporting discriminant validity. Discriminant validity was also supported for MWC (AVE = 0.74 in Sample 2 and 0.76 in Sample 3), with r² values of 0.48 (r = 0.69) in Sample 2 and 0.59 (r = 0.77) in Sample 3. Similarly, SC showed consistent results (AVE = 0.60 in Sample 2 and 0.66 in Sample 3), with r² values of 0.21 (r = 0.46) in Sample 2 and 0.29 (r = 0.54) in Sample 3. A second-order CFA model confirmed these findings, with fit indices (CFI = 0.92; TLI = 0.91; RMSEA = 0.07; SRMR = 0.05) and a significant chi-square difference test (χ²(Δdf = 1390) = 2860.2, p < .01), further supporting the distinctiveness of these constructs.

Homogeneity tests and multilevel CFA using company data across units tested whether PHC and MWC as sub-climate scores could be examined at the team level as shared climate perceptions (Bliese, 2000). Individual-level climate scores capture each employee’s individual perception of the work environment. The unit-level climate, then, emerges through the aggregation of these individual scores. This aggregation reflects a more collective and shared understanding of the climate within the unit. To assess the unit level, two additional samples were collected from a working population, with data from two timepoints (Times 1 and 2, with a 4-month lag) to examine the time-lag prediction of TWH-c on outcome variables at the collective-level. These additional samples were collected to both provide time after climate assessments and to increase the number of collective units per sample, which enhances our ability to assess collective outcomes of climate and provides a more robust assessment of collective-level perceptions. Sample 4 comprises 339 participants from 69 smaller organizations in China at Time 1, where we assessed organization-level climates. These organizations are from a variety of industries, such as manufacturing, construction, healthcare, agriculture, mining, transportation, and finance. At Time 2, data were collected from 50 of these organizations, focusing on organization-level outcomes and obtaining objective outcomes reported by top management teams. Across both time points, 56% of participants identified as men, and were 33.90 years old on average (SD = 6.03) with 7.15 years of industry tenure (SD = 1.92). The Cronbach’s α for SC, PHC, and MWC are 0.90, 0.95, and 0.96, respectively.

Sample 5 consisted of 104 work units from two organizations in the healthcare and logistic industries located in Taiwan. In this study, we operationalized units based on shared leadership or supervision, common location or environment, and regular social/collaborative interaction, rather than the traditional workgroup definition tied to formal organizational structures. This approach better captured functional relationships and communication patterns shaping daily work experiences in these organizations, regardless of formal hierarchy. At Time 1, we assessed both organization-level climates and outcomes, while Time 2 focused on collecting objective outcomes. However, only one of the two organizations provided objective outcome data, which was reported by their human resources management team. Across both time points, 51% of participants identified as men and 55% of participants were under 40 years old. Average employee tenure was 2.45 years (SD = 1.34). The Cronbach’s α for SC, PHC, and MWC are 0.99, 0.97, and 0.98, respectively.

To justify aggregation of individual responses into unit-level constructs, reliability and within-group agreement indices (rwg(j)) and intra-class correlations (ICC(1) and ICC(2)) were calculated. These indices assess the extent to which employee perceptions within a work unit are shared and distinguishable from those of other units. A multilevel CFA was then conducted to evaluate the fit of the TWH-c construct at the unit level. Analyses were performed on Samples 4 (organization level) and 5 (unit level). Our choice to include unit-level assessment despite the referent addressing overall organizational climate is based on the collective sensemaking mentioned prior, as this can also exist within local contexts (Beus et al., 2023). Even when item referents are framed at the organization level, meaningful and reliable unit-level climates can still emerge when shared interpretations are shaped by local leadership and social dynamics (Klein & Kozlowski, 2000). All climate variables were subjected to homogeneity tests, with SC serving as a comparative benchmark given its established role as a shared unit-level construct (Murphy et al., 2018).

In Sample 4, agreement and between-organization differentiation were strong across all variables. SC showed rwg(j) = 0.96, ICC(1) = 0.29, and ICC(2) = 0.67; PHC had rwg(j) = 0.97, ICC(1) = 0.34, and ICC(2) = 0.72; and MWC had rwg(j) = 0.90, ICC(1) = 0.17, and ICC(2) = 0.50. The ICC(1) values signify meaningful between-unit variance while sharing the same organization, which is a necessary condition for establishing unit-level constructs. The ICC(2) values indicate between-unit distinction, and the rwg(j) values indicate consensus within units regarding climate perceptions, which can be expected from groups within the same organization. The ICC(2) values further suggest acceptable reliability of the aggregated unit-level means, supporting their stability as indicators of shared unit climate. Overall, Sample 4 demonstrates a well-differentiated climate structure appropriate for multilevel analysis.

Sample 5 showed similar within-unit agreement and between-unit differentiation. SC yielded rwg(j) = 0.96, ICC(1) = 0.25, and ICC(2) = 0.72; PHC had rwg(j) = 0.94, ICC(1) = 0.15, and ICC(2) = 0.56; and MWC showed rwg(j) = 0.90, ICC(1) = 0.23, and ICC(2) = 0.70. Collectively, the agreement and reliability indices across both samples provide strong support for aggregation and confirm the appropriateness of Samples 4 (organization level) and 5 (unit level) for multilevel analysis of TWH-c and its outcomes.

Multilevel CFA involved specifying the measurement model of an overarching TWH-c construct based on the composite scores of the SC, three factors for the PHC, and the MWC of Sample 4 (Figure 1). Sample 4 was used specifically to maintain alignment between the structure of the data and the referent of the scale, which is held at the organization level. Satisfactory fit was detected with χ² = 16.80 (df = 10, p = 0.08), CFI = 0.99, TLI = 0.99, RMSEA = 0.04, SRMR(_{individual level}) = 0.02, and SRMR_{(organization level)} = 0.03. The findings offer further evidence of the robustness and validity of the TWH-c construct operationalized at the group level.

Criterion-related validity was assessed using separate regression analyses for each climate, predicting outcomes within that climate’s domain, in Samples 2, 4, and 5. Specifically, the data from Samples 2, 4, and 5 were used to assess criterion-related validity using individual-, organization-, and unit-level outcomes, respectively. This allows for testing the robustness of the sub-climate–outcome relationships across multiple levels of analysis. The aggregate TWH-c was also assessed, as it is presumed to be associated with outcomes across domains and serves as a single metric of overall TWH perceptions in a workplace. Correlations with outcomes at the individual level for Samples 2 and 3, and at the collective level for Samples 4 and 5, were conducted for each sub-climate as well as the TWH-c aggregate score. Reliability statistics and correlations among all climates and outcomes from each source of data are presented below. Criteria in the study include: 1. safety outcomes, 2. physical health outcomes, 3. mental well-being outcomes, and 4. job-related outcomes. For the aggregated collective outcome, ICC(1) and rwg(j) are presented to justify the aggregation of individual perceptions into collective-level variables. ICC(2) values are also presented, although Bliese et al. (2019) suggest that high ICC(2) values, while beneficial, are not essential for detecting emergent collective effects. See Tables 5–8 for correlations among all climates and outcomes for each sample. See Appendix B for all the following measures used at the individual and collective levels.