IntroductionA recent approach within the construction industry to increase safety on worksites has been evaluating contractors’ performance during the bidding process; however, measuring the safety performance of a company (such as a general contractor or a subcontractor) in the construction industry can be challenging. Traditional safety performance metrics rely on lagging indicators of safety, such as lost workdays; restricted work activity injuries; OSHA recordable injuries; and the Experience Modification Rate (EMR), which is a measure of a company’s past loss experience used by insurance companies to set premiums (Hinze and Godfrey 2003; Hoonakker et al. 2005; Siu et al. 2004). However, these traditional, injury-based metrics may present a skewed picture of safety performance, as they do not account for leading indicators (e.g., organizational programs and policies) that are important determinants of worksite safety (Beus et al. 2010; Christian et al. 2009; Flin et al. 2000).
With the goal of improving safety, a group of construction safety professionals developed a contractor safety assessment program (CSAP) called ConstructSecure, Inc. that integrates these traditional injury-based measures with leading indicators of safety. ConstructSecure, Inc., a commercial product, generates a CSAP score on a 100-point scale that allows for easy interpretation. The final score is based in part on the EMR, lost time and OSHA recordable injury rate, and OSHA experience (number of citations, the severity, the regulation, and the penalty assessed). Points are also added to the final score based on an assessment of the company’s safety management system through a series of questions on management commitment, employee involvement, hazard inspection and identification, worker training, and program evaluation, all of which are components of what OSHA defines as an Injury and Illness Prevention Program (I2P2) (OSHA 2012). Additionally, the quality and comprehensiveness of the company’s safety program is assessed and added to the score. The company’s written safety programs are uploaded to ConstructSecure’s website, and the text is read and assessed for certain elements related to workplace hazards and safety practices.
All CSAP data are entered by one individual, typically an environmental health and safety manager. Many general contractors and owners (e.g., Harvard University, Skanska) now require all companies bidding on projects to be registered within a CSAP, allowing project managers to evaluate subcontractors and general contractors before beginning work.
A CSAP metric is thought to be a balanced scorecard; it combines many different safety performance metrics and allows for an assessment of contractor safety. As proposed by Kaplan and Norton, a balanced scorecard approach to measuring performance (safety or otherwise) is the most efficient way to compare companies (Kaplan and Norton 1992). This measurement tool brings together disparate elements of a company in order to complement one another and provide a more accurate reflection of the safety performance. While a CSAP score may reflect certain aspects of a company’s safety performance, its ability to reflect the safety climate of a company is unknown. Furthermore, as a CSAP is based on written safety plans and incident history, it is unable to capture the dissemination or communication of the formal safety policies and procedures to workers.
Safety climate measures workers’ perception of the safety culture of their organization at one point in time, and has been found to predict safety-related outcomes (Huang et al. 2006; Wallace et al. 2006), such as injury frequency (Johnson 2007) and levels of under-reporting (Probst et al. 2008). Safety culture represents the set of attitudes, beliefs, values, and priorities held by managers and employees that directly influences the development, implementation, performance, oversight, and enforcement of health and safety in the work environment (Guldenmund 2000; NORA 2008).
Therefore, the objective of this exploratory study was to test if a CSAP safety performance score provided any reflection of safety climate on a worksite. The central hypothesis was that safety culture, as measured through the safety climate of an organization, was associated with the level of an organization’s health safety management programs and policies, as measured through a CSAP performance metric.
Materials and MethodsStudy Design and Participant EligibilityA cross-sectional survey in English was administered to construction workers throughout eastern Massachusetts on commercial construction sites through non-probability convenience sampling methods between January and July of 2012. All workers on the visited construction sites aged 18–65 were eligible to complete the survey, provided they had not previously taken part in the study at another site. Surveys collected from workers employed by companies not registered in ConstructSecure, Inc., the CSAP database, were excluded from analyses.
As perceptions of safety climate often differ between managers and workers (Gittleman et al. 2010), environmental health and safety managers from the companies with three or more employees surveyed were contacted separately and asked to complete a manager survey. The individual identified in the CSAP database as the person who completed the most recent ConstructSecure, Inc. application was approached.
Study MeasuresThe worker survey was developed based on a conceptual model (Figure 1) that described the framework of safety climate and its relation to other organizational factors. The survey contained Dedobbeleer and Beland’s (1991) nine-item safety climate scale for construction, as well as potential demographic covariates such as age, gender, race/ethnicity, education, trade, and job title. Each of the nine questions was assigned a point value from 0–10 based on the item response, and then summed together to determine the total construct score for each respondent. The safety climate scores had the potential range of 0–90.
Each worker was assigned a CSAP score that corresponded to the score of his/her self-identified company (either a general contractor or a subcontractor). Company CSAP scores were obtained from the ConstructSecure, Inc. online database on the day the survey was completed. The scores had the potential range of 0–100.
The manager survey was completed online through Qualtrics (https://harvard.qualtrics.com/). It contained the same Dedobbeleer and Beland (1991) nine-item safety climate scale as the worker survey, with an additional self-safety climate assessment scale developed by the investigators based on ten questions from the Laborers’ Health and Safety Fund of North America (Schneider 2011).
AnalysisThe workers who completed the survey were first categorized into either low or high CSAP groups based on a threshold of 86.1, the sample median CSAP score of the companies represented by the workers surveyed. The value selected as the high/low cutoff point in this study, while numerically high, closely matched the median value in the full CSAP database (87.4).
Differences in demographics, job-related factors, and worker safety climate scores between the high and low CSAP groups were then evaluated through two-sample t-tests and Fisher’s exact test.
Workers were assigned their company’s CSAP score and correlations between their company’s CSAP score and safety climate were assessed using Spearman’s correlation coefficient. The correlations were initially evaluated for all workers. In addition, since safety climate is a group-level construct, the correlations were also evaluated for all workers with at least four other co-workers from the same company surveyed (≥ 5 workers) and for all workers with at least nine other co-workers from the same company surveyed (≥ 10 workers).
Correlations were also evaluated at the company level, where each company was assigned a safety climate score – the average of all workers surveyed from the company. Separate correlation analyses were also performed for companies with five or more workers and ten or more workers.
Additionally, as a site’s general contractor is often responsible for managing the health and safety of a worksite, correlations were also evaluated at the general contractor level, where each general contractor was assigned a safety climate score calculated as the average of all workers surveyed on their sites.
In order to aggregate individual responses to the group level, within-group agreement indices were calculated. Values of intraclass correlation coefficients, specifically ICC(1) and ICC(2), were calculated for groupings of participants by company and by general contractor. Additionally, ICC(1) and ICC(2) were calculated for companies with 5 or more employees surveyed and for companies with 10 or more employees surveyed. While there is no standard guideline on an acceptable ICC(1) value, the most widely accepted criterion is greater than 0.10 to denote a medium effect size (Murphy and Myors 1998). There is also no definitive guideline on an acceptable ICC(2) value, but the most widely accepted criterion for ICC(2) is greater than 0.70 (LeBreton and Senter 2008; Ostroff and Schmitt 1993) The results for ICC(1) and ICC(2) for all of the company groupings were 0.11 and 0.44, and for the general contractor groupings were 0.046 and 0.59, respectively. ICC(1) and ICC(2) for companies with 5 or more employees were 0.13 and 0.70, and for companies with 10 or more employees were 0.13 and 0.77, respectively. As ICC(1) values are lower and ICC(2) values are higher, these results indicate that individual responses can be aggregated to these group levels. Confirmatory factor analysis was completed on the nine-item safety climate scale and resulted in two factors, worker involvement and management commitment, which was consistent with previous studies (Dedobbeleer and Béland 1991).
Finally, correlations between manager-assessed safety climate, worker-assessed safety climate, and the CSAP score were also evaluated using the Spearman coefficient.
All data analyses were completed in SAS version 9.2 (SAS Institute Inc., Cary, N.C.) and were considered significant at p < 0.05. Data collection methods used in this study were reviewed and approved by the Harvard School of Public Health’s Office of Regulatory Affairs and Research Compliance. The approved methods included a protocol in which construction workers were approached and invited to participate in the survey. Participants were informed verbally that by completing the survey, they indicated consent. As the survey was a one-time occurrence, in which individual identifiers were not required, written consent was waived in order to maintain anonymity.
ResultsStudy PopulationCompleted surveys were obtained from 401 workers across 26 sites under 14 different general contractors (Table I). These workers were employed by 68 different companies, of which 58 were registered with the CSAP database. The respondents were primarily male (97%), with a mean age of 42 years (Table II). The majority of the respondents were non-Hispanic (91%) white (80%). Most respondents were union members (92%) and had an average tenure with their current company of seven years (ranging from one day to 33 years). Individuals from 25 different trades were surveyed, with the carpentry and masonry trades being sampled more so than any other trades (23%). There were 180 workers who did not indicate a trade or left the question blank. Of all the respondents, 44% had a high school education or GED, and 40% had some college or trade school education. With the exception of trade, there were no significant differences in the demographic characteristics of the workers from the high and low CSAP categories.
Of the 27 companies contacted for the manager survey, 14 companies returned completed surveys. Only 11 of these surveys included a company name, causing three surveys to be excluded from the study. These eleven individuals were all safety managers and ranged in age from 26 to 54 years. Their tenure with their company ranged from one year to 44 years, and all but two individuals were male. Approximately 78% of the respondents had at least a bachelor’s degree, and all were white, non-Hispanic.
CorrelationsThere were no statistically significant differences in the safety climate scores between workers from high and low CSAP scoring companies (Table III). This result was true across all companies with CSAP scores (n=56) at the individual level (n=336).
Most Spearman correlations of worker safety climate and the sub-scale (worker involvement and management commitment) scores with company’s CSAP score were very weak and not significant (Table IV). There was a small positive correlation (r=0.11) between the CSAP score and the worker involvement score (p=0.038) at the individual level for all workers. There were also small positive correlations of r=0.16, r=0.12, and r=0.16 between the CSAP score and safety climate score (p=0.012), worker involvement score (p=0.047), and the management commitment score (p=0.012), respectively, at the individual level for companies who had five or more surveys. Additionally, there were small positive correlations between the CSAP score and the individual safety climate score (r=0.16, p=0.02) and the worker involvement score (r=0.15, p=0.033), when including only workers from companies with ten or more surveys. These correlations seem to be mainly driven by a large number of higher scoring companies and a small number of low scoring companies (Figures 2 & 3), and the association disappeared when at all other levels of grouping. Some correlations did increase when examined at the company level (Figure 4); however, the small number of companies reduced the power, and hence the correlations were not statistically significant.
Similarly, some correlations did increase when examined at the general contractor level. The correlation between the general contractor CSAP score and the general contractor average safety climate score was 0.11 (p=0.71). The correlations between the general contractor CSAP score and the general contractor average worker involvement score and management commitment score were −0.050 (p=0.87) and 0.048 (p=0.87), respectively. Again, the low number (n=14) reduced the power; none were statistically significant.
Spearman correlations conducted between the manager-assessed safety climate scale and the average climate score from their workers was moderate (r=−0.26), but not statistically significant (p=0.44) (Figure 5). The correlation between the manager-assessed safety climate score and the company’s CSAP score was weak and not statistically significant (r=−0.023, p=0.95). The modified LHSFNA scale had weak correlations with both worker safety climate and the CSAP score (r=−0.41, p=0.22 and r=−0.0046, p=0.99, respectively).
DiscussionThe goal of this exploratory study was to examine the association between workers’ safety climate scores and a score of their respective company’s (their direct employer, either a subcontractor or a general contractor) health and safety management systems, a Contractor Safety Assessment Program (CSAP) performance metric. Overall, the results presented here suggest that workers’ safety climate scores from a given company were largely independent of that company’s assessment of its health and safety management systems (as measured by a CSAP). There were, at best, weak, non-significant correlations between workers’ safety climate scores and the CSAP score for either their immediate employer (the subcontractor) or the general contractor for the worksite. The independence of the worker safety climate score and the CSAP performance metric can exist for many reasons, including some of the basic assumptions about safety climate in the construction industry and potential limitations of the data collection.
The lack of correlation may be due to a difference in what CSAP measures compared to what safety climate captures. CSAP scores are calculated through a computer algorithm that scans and scores formally written company policies and procedures and then combines that score with other leading and lagging safety performance indicators. A CSAP does not capture the dissemination or communication of these formal safety policies and procedures to workers. Safety climate, on the other hand, pertains to the communication of safety as a priority from top management and direct supervisors to workers (Zohar and Luria 2003). A company may have formal policies and procedures that present safety as a top priority, but just because those policies and procedures exist does not mean that they are implemented accordingly (Zohar 2008). For example, site supervisors and foremen who do not value safety themselves may not enact their company’s formal safety policies and procedures as they are written. This in turn can prevent employees from receiving the message that safety is a priority, thus negatively impacting employees’ perception of safety climate (Zohar and Luria 2003). Conversely, the formal policies and procedures of a company may not consider safety extensively, but the supervisors of that company may act and communicate in a way that shows employees that their safety is valued, which increases employees’ safety climate perception. These two scenarios highlight the potential disconnect between the written programs and policies of a company and what is enacted at the worksite and reflected in the safety climate measurements.
The lack of association may also be due to the complex nature of climate in the construction industry and the fact that most measures of climate are based on more stable workforces. Measuring safety climate in the construction industry differs from most climate research (Guldenmund 2000). Unlike a stable cohort of workers in a manufacturing plant, as one example, workers on construction projects vary day to day with different social interactions and networks. Most safety climate research has been conceptualized and conducted in industries that have relatively stable and traditional organizational structures (e.g., (Fleming et al. 1998; Zohar 1980; Zohar and Luria 2005). For example, within one organization in manufacturing, employees are trained to complete jobs in specific departments, and within those departments they report to assigned line supervisors. Line supervisors, who directly manage those front-line employees, are overseen by higher-level managers. Typically, in an organization in the manufacturing industry, employees work in the same teams or departments and report to the same supervisors. This allows for social interactions among coworkers and communication between supervisors and their employees that are mechanisms through which safety climate perceptions form. Social interactions help employees to gather and interpret information regarding the true priority of safety in their organization (Schneider and Reichers 1983). Communication with supervisors also demonstrates to employees the true priority of safety through the ways supervisors enact formal safety policies and procedures and handle competing demands between safety and productivity or profit.
A potential limitation in our study was the choice of our climate metric. In order to capture employees’ safety climate perceptions, the proper psychological measure is needed. The Dedobbeleer and Beland (1991) measure was chosen because it is the only measure, to the authors’ knowledge, of safety climate specific to the construction industry. It is important to have industry-specific measures, as the nature of safety climate in each industry may differ. Unfortunately, the Dedobbeleer and Beland (1991) measure was constructed in a way that may not accurately evaluate employees’ true safety climate perceptions. It is important that safety climate questions be specific to different reference groups in an organization. In traditional organizations, employees form safety climate perceptions using information about their direct supervisors and their organization separately (Zohar, 2010). For example, an employee may believe his supervisor cares about his safety while the organization, as a whole, does not. To determine the overall safety climate perception that an employee has for his company, all aspects of his organization have to be taken into consideration. This is more complicated in the construction industry, but in the Dedobbeleer and Beland (1991) measure, there are a limited number of questions and the referent category changes between the job itself, the worksite, and the company. Different referents are important, but it must be done in a systematic and comprehensive way and it must also be clear to the workers who they should be thinking of when answering questions. For example, as each referent is not defined in the Dedobbeleer and Beland (1991) measure, one employee may be thinking of the “company” as the general contractor while another employee is thinking about his direct employer.
An additional issue is the limited number of questions and factors used by Dedobbeleer and Beland (1991) to measure safety climate. While more research is needed to determine the overall factor structure of safety climate, which may differ in different industries, Zohar (1980) found eight factors and so examining only two may limit results.
Furthermore, selection bias of both the worksites visited and the workers surveyed could have impacted the findings. Companies with either very high or very low CSAP scores may have been more willing to allow surveying on-site. This could have occurred for two reasons. Companies with high CSAP scores may have felt confident in having researchers survey their employees about safety or companies with low CSAP scores may have wanted to prove their safety climate scores were higher than their CSAP scores. It is unlikely, however, that individual workers would know their company CSAP scores; thus, any resulting biases are assumed to be nondifferential.
There may also be some selection bias in the contractors included in the study, as they must be registered in the CSAP database. The contractors must have, at a minimum, some value of safety and safety management in their organizations to simply register for the CSAP. However, as seen in Table V, the distribution in this study sample mirrors the distribution of companies in the full dataset. As more owners and general contractors require that subcontractors register with CSAP, the scores are less skewed by companies with more robust safety programs and represent a less biased picture of commercial construction.
The contractors included in this study were limited to commercial contractors working in the greater Boston area. As a result, the findings may not be generalizable to industrial or residential construction, or to small commercial companies, outside of the northeast. However, the data obtained in this study can be used to shape future studies that expand the study radius and scope.
Finally, the power of this study to examine the association between managers’ perception and employees’ perception was limited due to the small sample size of managers surveyed (n=11).
The transitory nature of construction raises questions about how construction workers form their safety climate perceptions. Do they bring the safety perceptions they have formed from their company to each job? Do they form new perceptions for each worksite? Is it the union, subcontractor, site, or other subgroup that most influences workers’ perception of safety climate? Most of the available safety climate literature in the construction industry has included theoretical and organizational models that have been used to develop fundamental safety climate in classical work-organizational industries. Most studies have used abbreviated climate scales with origins in health care or manufacturing or with few validation studies conducted in the construction industry (Jorgensen et al. 2007; Kines et al. 2010).
Measuring safety climate in the construction industry is complex and has not received much conceptual attention in the safety climate literature. Up to this point, most studies that address safety climate have treated the organizational layers on the construction site as similar to any other industry. It is important to determine the ways in which construction workers would group themselves in terms of safety climate groups. For example, it may be a general contractor on a worksite or a union that is influencing construction workers’ safety perception more so than any other reference group. It is not for researchers to decide what makes the most sense; however, researchers can understand how safety climate works in the construction industry from the workers themselves. This study highlights the need for safety climate research in construction to recognize and address the numerous dimensions of the construction site.
Conclusions and ContributionsThis exploratory study is one of the first to evaluate whether a newly developed and widely used measure of contractor safety performance is associated with safety climate measures. CSAP programs are used with increasing frequency in contractor hiring decisions, yet the question of their relationship with safety climate remains. With 401 workers surveyed, from 26 different worksites of varying scope and size, this study provides the important first step in understanding the correlation between a CSAP measure and safety climate.
Workers’ safety climate scores, as measured in this study, were independent of an overall measure of their company’s health and safety management systems, a CSAP safety performance score. Safety climate in construction is a complex construct, which is reflected in the challenges encountered in its measurement.