Numerous methods have been developed for quantifying the risk of upper limb musculoskeletal injuries, including self-reports, observation, video-based single-frame analysis, and direct measurements (Joshi and Deshpande, 2019). Each technique has advantages and disadvantages (David, 2005). The hand activity level (HAL) rating is a 10-point visual analog scale based on hand speed and rest pauses, where HAL=0 when the hands are idle most of the time and there are no regular exertions, and HAL=10 when there is rapid steady motion, continuous exertion and difficulty keeping up. The observed HAL rating is recorded as an integer value. Originally introduced by Latko, et al. (1997), the HAL rating has been adopted by the American Conference of Governmental Industrial Hygienists (ACGIH) in 2001 Hand Activity Threshold Limit Value^® (TLV^®) (ACGIH, 2021).

Epidemiological investigations demonstrated significant relationships between the TLV^® action limit and elbow/forearm tendonitis, carpal tunnel syndrome (CTS) (Franzblau et al., 2005), and musculoskeletal pain (Werner et al., 2005). Longitudinal studies of manufacturing workers found that the TLV^® for HAL was predictive of increased risk for carpal tunnel syndrome (Garg, et al., 2012, Yung et al., 2019, Kapellusch et al., 2014a), predicted increased risk for CTS while controlling for obesity and job strain (Burt, et al., 2013), and was associated with increased risk for flexor tendon entrapment of the digits (Kapellusch et al., 2014b).

The observational method for ascertaining the HAL rating for a job requires a trained observer viewing workers performing the job on-site or observing a video of the job off-site. Takala, et al. (2010) recommended the observational HAL rating for general screening purposes. Without training, interobserver reliability is moderate (Kappa = 0.52) but with training, reliability increases (Kappa = 0.70) (Ebersole and Armstrong, 2002).

In addition to observer ratings, the HAL may also be determined from a lookup table (HAL_T), which is part of the TLV^® by measuring exertion frequency (F) and percent duty cycle (D) where: (1) F=no.exertionsexertiontime+resttime and (2) D=100exertiontimeexertiontime+resttime where exertion times and rest times are totaled within a cycle. Radwin, et al. (2014) developed an equation for estimating HAL based on measurements of F and D using data from Latko, et al. (1997) to continuously predict HAL values consistent with the TLV^® look-up table. The equation: (3) HAL=6.56lnDF1.311+3.18F1.31 can be utilized to directly measure HAL and is also part of the TLV^®. The direct measurement method is typically applied by using video single-frame analysis.

Video single-frame analysis has been used for objectively measuring F and D utilizing software such as multimedia video task analysis (MVTA) by marking the video frames when exertions start and end in a cycle (Yen and Radwin, 1995). This method has been frequently used for calculating HAL directly from videos (Bao, et al, 2006a; Burt et al., 2013; Harris, et al., 2011). Although the method is objective, it is time intensive, requiring the video to be viewed by an analyst in slow-motion or by single-frame.

Bao et al. (2006b) observed poor correlation between HAL observations, self-reports, and direct measurements and noted several reasons for different ratings, including using different definitions of an exertion. Wurzelbacher, et al. (2010) compared more than 700 tasks for 484 workers and found that while correlated (Spearman rank = 0.49), the agreement between HAL ratings (within ±1point) from observations and calculated from videos was 61%. An objective automated method for measuring HAL is therefore desirable, especially for routine practice in industry.

Chen et al. (2013) developed an automated method for measuring HAL using a cross correlation-based template matching algorithm to track the motion trajectory of a selected region of interest (ROI) over successive video frames. The authors demonstrated that HAL can be calculated by automatically measuring F and D from hand displacement, speed and acceleration signals.

To further pursue the above goal, Akkas et al. (2015) developed an equation to measure HAL directly from RMS hand speed (S) and D since the HAL scale is anchored against the speed of hand motion/exertions and rest pauses. Furthermore, measuring F of repetitive motion is more challenging when the motion becomes more complex, such as when exertions and motions coincide. The equation was: (4) HAL=10e-15.87+0.02D+2.25lnS1+e-15.87+0.02D+2.25lnS

Finally, to measure D from single video cameras, Akkas et al. (2016) developed machine learning algorithms for laboratory simulations and tested them for real factory repetitive motion tasks to determine how accurately the algorithms work for different tasks (Akkas et al., 2017). A Feature Vector Training (FVT) algorithm was trained using corresponding phases (i.e. exertion and rest) based on the first cycle of each video clip. The first cycle of each video that were manually annotated using MVTA were inputted for the first cycle exertion and rest elemental times were used to train a k-nearest neighbor (kNN) classifier as the state estimator. The algorithm used k=1 since only one cycle was used for training. Subsequent exertion and rest states for the remaining nine cycles were automatically classified by the algorithm.

In the current study we compare the HAL values calculated using the three different methods; HAL based on an observer ratings (HAL_O), HAL calculated from single-frame-video analysis for measuring F and D (HAL_F) (Radwin et al., 2015), and HAL calculated using computer vision marker-less tracking for measuring hand speed and the kinematic properties from the hand tracking signal (i.e. location, speed and acceleration) to identify patterns when an exertion was made to measure D (HAL_C) (Akkas et al., 2015; 2016). Moreover, we provide recommendations based on these analyses for obtaining more consistent measurements of HAL.

The HAL values were statistically different among the three methods (HAL_C, HAL_F and HAL_O) for calculating HAL (F(2, 1251)=15.71, p<.001), where the videos were treated as a random variable. The contrast between HAL_C − HAL_F was 0.35, and the 95% Tukey multiple family-wise confidence interval ranged from 0.05 to 0.74 (p=.02). The average absolute difference between HAL_C −HAL_F was 1.20 (SD=0.83, Max=4.89, 95% CI=0.11). The contrasts between HAL_O −HAL_F was 0.56, and the 95% Tukey multiple family-wise confidence interval was 0.27 to 0.86 (p<.001). The average absolute difference between HAL_O −HAL_F was 1.31 (SD=1.02, Max=5.75, 95% CI=0.12). There was no statistically significant difference in the contrasts between HAL_C −HAL_O with an overall difference of −0.220, and the 95% Tukey multiple family-wise confidence interval was −0.52 to 0.08 (p=.19). The average absolute difference between HAL_C −HAL_O was 1.38 (SD=1.14, Max=7.16, 95% CI=0.14).

Histograms for the differences between HAL_C −HAL_F, HAL_C −HAL_O, and HAL_F −HAL_O are plotted in Figure 2. Since the HAL scale requires an integer value between 0 and 10 (ACGIH, 2021), a difference of one HAL rounded to the nearest integer between −1.5 < HAL_C − HAL_F < 1.5 was observed for 68% (N=284 cases), and between −2.0 < HAL_C −HAL_F < 2.0 was observed for 89% (N=374 cases) of the data, as shown in Figure 2A. A difference between −1.5 < HAL_C −HAL_O < 1.5 was observed for 64% (N=269 cases), and between and −2.0 < HAL_C − HAL_O < 2.0 was observed for 77% (N=324 cases) of the data of the data, as shown in Figure 2B. A difference between −1.5 < HAL_F −HAL_O < 1.5 was observed for 65% (N=271 cases), and between −2.0 < HAL_F −HAL_O < 2.0 was observed for 78% (327 cases) of the data, as shown in Figure 2C.

Scatter plots for HAL_C v. HAL_F, HAL_C v. HAL_O, and HAL_F v. HAL_O are provided in Figure 3. Linear regression between HAL_C v. HAL_F (Figure 3A) had a coefficient of 1.03 for a coefficient of determination R²=0.89 when the intercept was set to zero, and a coefficient of 0.83 and an intercept of 0.91 for a coefficient of determination R²=0.57. Linear regression between HAL_C v. HAL_O (Figure 3B) had a coefficient of 0.91 for a coefficient of determination R²=0.84 when the intercept was set to zero, and a coefficient of 0.73 and an intercept of 0.85 for a coefficient of determination R²=0.34. Linear regression between HAL_F v. HAL_O (Figure 3C) had a coefficient of 0.84 for a coefficient of determination R²=0.86 when the intercept was set to zero, and a coefficient of 0.72 and an intercept of 0.57 for a coefficient of determination R²=0.40.

Bland-Altman plots for HAL_C −HAL_F, HAL_C−HAL_O, and HAL_F −HAL_O are provided in Figure 4. The plot for the HAL_C −HAL_F had 95% of the data between the upper limit of 3.13 and the lower limit of −2.44, for a range of 5.57 (Figure 4A). The plot for the HAL_C −HAL_O had 95% of the data between the upper limit of 3.26 and the lower limit of −3.72, for a range of 6.12 (Figure 4B). The plot for the HAL_F −HAL_O had 95% of the data between the upper limit of 2.49 and the lower limit of −3.63, for a range of 6.98 (Figure 4C).

The objective of the current study was to compare the different methods for estimating HAL. Although the differences in HAL for the three methods were relatively small on average (< 1), the differences were statistically significant (p<.001). On average the ratings for HAL_C and HAL_F were the closest. A difference within ±1 point on the HAL scale was considered equivalent. The HAL_C and HAL_F ratings were the most consistent; more than half (68%) of all the task ratings were within that range. Most of the data for HAL_C and HAL_F (89%) had differences less than ±2 (Figure 1A). The absolute difference between HAL_C −HAL_F was smallest on average (mean=1.20, SD=0.83, 95% CI=0.11). The Bland-Altman analysis had similar findings, where a majority of the HAL values agreed within a similar margin. HAL_C and HAL_F had the greatest correlation (R²=.89).

All three methods in the current study had better correlations among the three methods than those reported among individual analysts for observed HAL ratings by Spielholz et al. (2008). They compared HAL_O ratings among expert raters and had a Spearman r correlation of 0.67 while the correlation among experts and novice raters was 0.39. These results in the current study indicate that computed HAL values (HAL_C and HAL_F) were the most reliable and better correlated than when multiple observers rated the same videos.

Wurzelbacher et. al. (2010) compared observer rated HAL (HAL_O) and calculated HAL based on single-frame video analysis and the ACGIH TLV integer HAL look-up table (HAL_T). They reported that the Spearman correlation r correlation for HAL_O and HAL_T was 0.49. The percent of exact agreement was 22%, while agreement within ±1 point on the HAL scale was 61%.

The consistency between definition of exertions for obtaining the frequency (F) was likely an important factor in gaining consistency between the calculated HAL measures (HAL_C and HAL_F). Bao et al. (2006b) found that different definitions of repetitive exertions produced different measures of repetitiveness and that under those circumstances their correlations were poor. Although on average the differences between the three methods for estimating HAL were small, the impact of averages on studies involving large cohorts may be less important than for HAL values used by practitioners where unreliable data for a single subject might be more consequential.

Values for HAL_F and HAL_C had good agreement despite that HAL_F was calculated based on frequency (F) in Equation 3 while HAL_C was calculated based on hand speed (S) in Equation 4. The HAL scale is anchored by hand speed and proportion of exertions, which is better aligned with the speed/ duty cycle relationship described in Equation 4 for HAL_C, rather than the frequency/ duty cycle relationship described in Equation 3 for HAL_F. This might have occurred because the origin of Equation 3 (Radwin, et al., 2015) was developed by aligning its values with the TLV HAL table, which is based on F and D. This agreement provides further evidence that frequency measures used in Equation 3 for HAL_F, and speed measures used for HAL_O and HAL_C in Equation 4 are consistent measures. The large variations in HAL_O indicate that the observation method is less reliable than the single-frame or computer vision measures. Comparable agreement might be expected for HAL_T and HAL_C although lookup table values are discrete and lack the precision of an equation.

There are several qualifications that limit these findings. Since the video database was created for a different purpose, many of the videos were unsuitable for computer vision analysis. Although 419 videos out of 1,653 video clips (25%) were available for study, there is no indication that the excluded videos were systematically related to the HAL values. Exclusions were based on independent circumstances such as the videographer’s location, camera movements such as panning or zooming, or extraneous objects interfering with a full video of the workers’ hands. All HAL_O values were evaluated by a single observer, rather than multiple observers. Although the observer may have had a bias, it is assumed the bias was consistent among all HAL_O estimates.

Considering the excluded videos, it is important that in practice, recordings be made appropriately for computer vision analysis. Videos should be focused on the hands from a vantage point perpendicular, or within ±30° of perpendicular to the plane of greatest motion (Chen, et al., 2015) to minimize parallax error. Image motion should be kept at a minimum by fixating on the hands, avoiding panning or zooming, and using fixed focus lenses. The full video frame should be used for recording the hand motion, if possible. Obstructed views of the hands should also be avoided.

Since HAL is related to muscle exertions (muscle contractions), manual single-frame HAL_F can sometimes be considered more accurate than computer vision HAL_C. This is because HAL is related to muscle exertions (muscle contractions) that can occur with movement (dynamic muscle contractions) or without movement (isometric muscle contractions). In some cases, such as when operating a power hand tool, although there is no observable movement during the exertions, their occurrence may be associated with corresponding events such as squeezing a trigger, or the sound made by the tool. In these cases, a human analyst may be able to detect the exertions, where computer vision may not. HAL_F was demonstrated reliable (Wurzelbacher, et al., 2010) and utilized in the ULMSDC prospective studies that demonstrated the predictivity of the TLV^® for HAL (Garg, et al., 2012; Yung et al., 2019; Kapellusch et al., 2014a; Yung et al., 2019; Kapellusch et al., 2014a; Burt, et al., 2013; Kapellusch et al., 2014b), and thus is the most validated method. Since HAL_C is based on a computer algorithm, its re-test reliability should not be an issue, although it should be formally tested in follow-on studies.

While computer vision was utilized in the current study, other researchers have incorporated continuous measurement methods such as inertial measurement unit sensors (Thamsuwan, et al., 2020) which may be an alternative method for making direct HAL measures. These types of measures should be compared with computer vision in future investigations.

Although manual single-frame MVTA analysis is considered more time consuming than automated computer vision analysis, we are not aware of any benchmarks for which to compare times and analysis time was not recorded for the current study. The time to do a single-frame analysis is related to the frequency of exertions and the cycle time. The more frequent exertions occur, or the longer the cycle time, the more time the analyst needs to take to code them. The analysis time also depends on the required accuracy where the video has to pause longer and more often when a more accurate measure is necessary.

The results of the current study suggest that the computer vision methodology mostly yielded comparable results as single-frame video analysis. The potential advantages of an automated method include objectivity and that computer vision analysis has the potential to require considerably less analyst time than single-frame video analysis.

Given that there is external validity for the ACGIH TLV^® for HAL, having more objective and efficient methods of applying the TLV should be useful in practice. Since there is good epidemiologic evidence of an association between the TLV^® for HAL and incident CTS, computer vision shows promise for increasing efficiency and objectivity for assigning HAL ratings, which could improve adoption of the method by practitioners.

This study concludes that computed methods for calculating HAL are more consistent and reliable than observation. Computer vision methods for estimating HAL had better agreement with single-frame methods than with observation.