<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" article-type="research-article"><?properties manuscript?><front><journal-meta><journal-id journal-id-type="nlm-journal-id">9420718</journal-id><journal-id journal-id-type="pubmed-jr-id">38841</journal-id><journal-id journal-id-type="nlm-ta">Proc Hum Factors Ergon Soc Annu Meet</journal-id><journal-id journal-id-type="iso-abbrev">Proc Hum Factors Ergon Soc Annu Meet</journal-id><journal-title-group><journal-title>Proceedings of the Human Factors and Ergonomics Society ... Annual Meeting. Human Factors and Ergonomics Society. Annual Meeting</journal-title></journal-title-group><issn pub-type="ppub">1071-1813</issn><issn pub-type="epub">2169-5067</issn></journal-meta><article-meta><article-id pub-id-type="pmid">29276370</article-id><article-id pub-id-type="pmc">5740231</article-id><article-id pub-id-type="doi">10.1177/1541931213601741</article-id><article-id pub-id-type="manuscript">HHSPA925225</article-id><article-categories><subj-group subj-group-type="heading"><subject>Article</subject></subj-group></article-categories><title-group><article-title>Statistical Prediction of Hand Force Exertion Levels in a Simulated Push Task using Posture Kinematics</article-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Lim</surname><given-names>Sol</given-names></name></contrib><contrib contrib-type="author"><name><surname>D&#x02019;Souza</surname><given-names>Clive</given-names></name></contrib><aff id="A1">Center for Ergonomics, Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI, USA</aff></contrib-group><pub-date pub-type="nihms-submitted"><day>8</day><month>12</month><year>2017</year></pub-date><pub-date pub-type="epub"><day>28</day><month>9</month><year>2017</year></pub-date><pub-date pub-type="ppub"><month>9</month><year>2017</year></pub-date><pub-date pub-type="pmc-release"><day>22</day><month>12</month><year>2017</year></pub-date><volume>61</volume><issue>1</issue><fpage>1031</fpage><lpage>1035</lpage><!--elocation-id from pubmed: 10.1177/1541931213601741--><abstract><p id="P1">This study explored the use of body posture kinematics derived from wearable inertial sensors to estimate force exertion levels in a two-handed isometric pushing and pulling task. A prediction model was developed grounded on the hypothesis that body postures predictably change depending on the magnitude of the exerted force. Five body postural angles, viz., torso flexion, pelvis flexion, lumbar flexion, hip flexion, and upper arm inclination, collected from 15 male participants performing simulated isometric pushing and pulling tasks in the laboratory were used as predictor variables in a statistical model to estimate handle height (shoulder vs. hip) and force intensity level (low vs. high). Individual anthropometric and strength measurements were also included as predictors. A Random Forest algorithm implemented in a two-stage hierarchy correctly classified 77.2% of the handle height and force intensity levels. Results represent early work in coupling unobtrusive, wearable instrumentation with statistical learning techniques to model occupational activities and exposures to biomechanical risk factors <italic>in situ</italic>.</p></abstract></article-meta></front><body><sec sec-type="intro" id="S1"><title>INTRODUCTION</title><p id="P2">Direct measurement of external force demands in ambulatory material handling tasks (such as pushing, pulling, carrying with different load levels) <italic>in situ</italic> remains a challenge for ergonomics analysis. These force estimates typically get used as inputs to biomechanical models for estimating joint loads and assessing injury risk. Towards assessing external loads and kinetics in ambulatory tasks, previous studies have used pressure mapping insoles (<xref rid="R6" ref-type="bibr">Cordero, Koopman, &#x00026; Van Der Helm, 2004</xref>) and instrumented force shoes (<xref rid="R8" ref-type="bibr">Faber, Kingma, Schepers, Veltink, &#x00026; Van Dieen, 2010</xref>) to measure ground reaction forces, instrumented hand gloves to measure grasp forces (<xref rid="R4" ref-type="bibr">Castro &#x00026; Cliquet, 1997</xref>), and electromyography (EMG) to measure muscle activity thereby estimating the magnitude of force exerted (<xref rid="R17" ref-type="bibr">Theado, Knapik, &#x00026; Marras, 2007</xref>). Such methods require trained ergonomists and can be cumbersome and obtrusive.</p><p id="P3">Motion analysis systems comprising body-worn inertial sensors have been used for measuring spatio-temporal gait parameters (<xref rid="R1" ref-type="bibr">Aminian, Najafi, B&#x000fc;la, Leyvraz, &#x00026; Robert, 2002</xref>), joint kinematics (<xref rid="R2" ref-type="bibr">Bernmark &#x00026; Wiktorin, 2002</xref>; <xref rid="R7" ref-type="bibr">El-Gohary &#x00026; McNames, 2012</xref>) and for material handling activity classification (<xref rid="R11" ref-type="bibr">Kim &#x00026; Nussbaum, 2014</xref>). Recent advances in biomechanical analysis techniques have also investigated estimating of joint loads during normal walking using just kinematic data from inertial sensors (Karatsidis et al., 2017).</p><p id="P4">In this paper, we explore the potential use of inertial sensor-based posture kinematics and statistical learning techniques to predict external load conditions, specifically normalized push and pull force levels. Prior ergonomics research has shown that, given certain work constraints, body posture is organized systematically and predictably in response to external force demands (e.g., <xref rid="R9" ref-type="bibr">Hoffman, 2008</xref>; <xref rid="R13" ref-type="bibr">Lim, Case, &#x00026; D&#x02019;Souza, 2016</xref>). We developed and tested a statistical prediction model with a limited set of posture variables from inertial sensors and anthropometry variables to estimate normalized high vs. low force levels and location of force exertion (shoulder vs. hip height) during pushing and pulling task. We focus on the Random Forest technique which yielded the highest prediction accuracy from among five statistical learning techniques that were evaluated.</p></sec><sec sec-type="methods" id="S2"><title>METHODS</title><sec id="S3" sec-type="methods|subjects"><title>Study Participants</title><p id="P5">The study recruited fifteen healthy right-handed male individuals aged between 18 to 35 years old from the university population. Gender and age restriction were applied to minimize variability in task postures. Average (SD) age, height, and weight of participants were 23.9 years (3.7 years), 1762mm (49mm), and 69.55kg (9.30kg) after excluding data from three participants due to instrumentation error. Prior to participation, participants provided written informed consent and were screened for pre-existing back injuries or chronic pain with a body discomfort questionnaire adapted from the body mapping exercise developed by NIOSH (<xref rid="R5" ref-type="bibr">Cohen, 1997</xref>). The study was approved by the university&#x02019;s Institutional Review Board.</p></sec><sec id="S4"><title>Experiment Procedure</title><p id="P6">The experiment had participants exert an isometric horizontal force on an instrumented handle (<xref rid="F1" ref-type="fig">Figure 1</xref>) to achieve and maintain a required target force level (&#x000b1;5%) for a 3s interval in 36 counterbalanced task conditions. Task conditions were varied by manipulating four task parameters, viz., handle height, force intensity, handedness, and force direction. In this paper, we focus on two of the task parameters, i.e., handle height (hip vs. shoulder level) and force intensity (low vs. high). The low-level force intensity was set to 25% and the high-level to 75% of the participant&#x02019;s two-handed maximum push exertion (MVE; <xref rid="R9" ref-type="bibr">Hoffman, 2008</xref>) measured at hip height and averaged over two trials.</p></sec><sec id="S5"><title>Data Processing</title><p id="P7">During the experiment, body posture kinematics were obtained using four commercial data-logging IS devices (YEI Technology, Inc.) attached over the sixth thoracic (T6) vertebra, low-back (L5/S1), lateral aspect of the right upper arm, and lateral aspect of the right thigh using customized Velcro straps (<xref rid="F1" ref-type="fig">Figure 1</xref>). The IS devices recorded triaxial accelerometer, gyroscope, and magnetometer data at 100-Hz sampling frequency. The data was filtered using a second-order low-pass zero-lag Butterworth filter with a 2-Hz cut-off frequency. Three-dimensional segment orientations using IS data were computed using a custom algorithm implemented in MATLAB R2016b (The MathWorks Inc.) and averaged over the 3s task duration (for details see <xref rid="R13" ref-type="bibr">Lim et al., 2016</xref>).</p></sec><sec id="S6"><title>Statistical Data Analysis</title><sec id="S7"><title>Variable selection</title><p id="P8">Three segment postural angles (viz., torso flexion, pelvis flexion, and right upper arm inclination) relative to the reference posture (T pose) and two joint angles (lumbar flexion and right hip flexion) were selected as potential predictor variables. Nineteen anthropometric and strength measurements were also included as predictors. Tests for multicollinearity (i.e., correlation coefficient &#x0003e; 0.90) resulted in 13 variables being excluded from further analysis.</p><p id="P9">The final set comprised eleven variables, viz., five posture variables: torso flexion (TF), pelvis flexion (PF), right upper arm inclination (UA), lumbar joint flexion (LF), and right hip flexion (HF), and six person variables: stature, weight, grip strength (right-hand), push MVE, L5/S1 to floor height, and Greater Trochanter to floor height.</p></sec><sec id="S8"><title>Statistical model development</title><p id="P10">A preliminary analysis was conducted comparing five statistical classification techniques, viz., multinomial logistic regression, linear discriminant analysis, classification and regression trees, random forest, and na&#x000ef;ve bayes in predicting the external force level as a categorical variable with four classes (high force at shoulder height, low force at shoulder height, high force at hip height, and low force at hip height). Among these techniques, the Random Forest had the highest prediction accuracy when estimating the external load level and is the focus of this analysis.</p><p id="P11">Random Forest (RF; <xref rid="R3" ref-type="bibr">Breiman, Friedman, Olshen, &#x00026; Stone., 1984</xref>) is a tree-based statistical learning technique that explores the relationship between a response variable and multiple predictor variables by growing recursive binary partitioning at the nodes of the tree. In contrast to in the classification and regression trees which grow and prune a single tree for prediction, a RF evaluates hundreds of trees with subsets of predictor variables chosen randomly from the full set and averages the prediction result to obtain one final model (<xref rid="R12" ref-type="bibr">Liaw &#x00026; Wiener, 2002</xref>).</p><p id="P12">Two different types of RF algorithms were implemented to predict the four classes (<xref rid="F2" ref-type="fig">Figure 2</xref>). Model-1 was a multiclass prediction model where the algorithm classifies four response classes at once. Model-2 was a two stage hierarchical model comprising a first binary classification model for predicting handle height, and a second stage binary classification model for predicting the force intensity level given handle height. Model-2 was proposed based on prior empirical studies which indicate that changes in handle height induce a greater change in body posture compared to manipulations in the force intensity level (<xref rid="R13" ref-type="bibr">Lim et al., 2016</xref>). Based on the prediction result for handle height from the first stage, the dataset was split into two groups and then subjected to a second stage model for predicting the force intensity level. Model parameters were set as the same for Models 1 and 2, namely, the number of randomly chosen predictors at each split was set as three, and the number of trees for each model was set as 50.</p></sec><sec id="S9"><title>Model Performance</title><p id="P13">A holdout cross validation was performed by randomly assigning 90% of the data as the training set and the remaining 10% as the testing set. This was repeated 20 times for both models. Model performance was evaluated by comparing the average (S.D.) prediction accuracy (i.e., correct prediction vs. misclassifications) between Models 1 and 2. All statistical computations were carried out in the R Statistical Package v.3.3.1 (<xref rid="R14" ref-type="bibr">R Core Team, 2016</xref>).</p></sec></sec></sec><sec sec-type="results" id="S10"><title>RESULTS</title><sec id="S11"><title>Model Comparisons</title><sec id="S12"><title>Model-1: multiclass prediction</title><p id="P14">The average (S.D.) prediction accuracy of the multiclass model was low at 27.2% (9.4%) suggesting that predicting the force intensity and location of force application simultaneously may be challenging.</p></sec><sec id="S13"><title>Model-2: hierarchical prediction</title><p id="P15">The second model was built by having two sequential binary classification models as described in <xref rid="F2" ref-type="fig">Figure 2</xref>. The average (S.D.) prediction accuracy of the overall prediction model was 77.2% (4.4%). Classification on handle height alone was 96.6% (2.1%) accurate on average (S.D.), while the prediction on force intensity was 80.1% (7.8%) accurate. This finding suggests that changes in body kinematics due to the force intensity levels may be subtle and not distinguishable when the data is aggregated over handle height, but is more meaningful when posture changes are compared at the same handle height level. Postural changes between force intensity were greater when the handle height was set at shoulder height, and resulted in a higher prediction accuracy (81.5%) compared to hip height (78.6%).</p><p id="P16">Results from a t-test confirmed that the hierarchical model (Model-2) outperformed the multiclass model (Model-1) in terms of greater prediction accuracy (<italic>t</italic> = 21.67, dof = 26.93, <italic>p</italic> &#x0003c; 0.001) between holdout testing. The hierarchical Model-2 also showed smaller variance in prediction accuracy (S.D. of Model-1 = 9.4% vs. Model-2 = 4.4%) suggesting greater stability.</p></sec></sec><sec id="S14"><title>Variable Importance</title><p id="P17">The relative importance of different variables comprising the hierarchical model (Model-2) was examined by calculating the Gini impurity Index (<xref rid="R16" ref-type="bibr">Strobl, Boulesteix, Zeileis, &#x00026; Hothorn, 2007</xref>), which is the average impurity at a data partition across all classes of the response variable. A greater decrease in the Gini Index including vs. excluding a particular predictor variable from the model suggests a greater importance of that variable.</p><p id="P18"><xref rid="F3" ref-type="fig">Figure 3</xref> shows the top-five important variables in each stage of the final hierarchical model by plotting the relative importance (%) of each variables in the model. The relative importance was calculated as a relative proportion of mean decrease in Gini Index. Torso flexion and pelvis flexion angles were the most important predictors when classifying handle height (<xref rid="F3" ref-type="fig">Figure 3-A</xref>).</p><p id="P19">All five postural angles were almost equally important when predicting the force intensity level at the hip handle height (<xref rid="F3" ref-type="fig">Figure 3-B</xref>). Pelvis flexion was relatively more important than other postural angles when predicting the force intensity level at the shoulder handle height (<xref rid="F3" ref-type="fig">Figure 3-C</xref>). These differences in variable importance between the stage-2 sub-models suggest a need for predicting force exertion levels specific to handle location and not aggregated across handle height conditions.</p></sec></sec><sec id="S15"><title>DISCUSSION AND CONCLUSIONS</title><p id="P20">This study was intended as an initial step to explore the potential of using inertial sensor-derived posture kinematics for load prediction. Understandably, the resulting prediction model is not yet generalizable for predicting pushing and pulling force levels across different worker and task conditions due to its small sample, constrained task conditions, and limited number of sensors. Nevertheless, the statistical prediction models presented indicate that a reasonably accurate binary classification of the exerted hand force levels during two-handed pushing and pulling task can be made solely from inertial sensor-derived posture kinematics. Further, this suggests the potential of using inertial-sensor based force prediction models when direct measurement of forces may be problematic or obtrusive.</p><p id="P21">A hierarchical approach to statistical modeling significantly improved the prediction accuracy compared to predicting multiple response classes at once. This result underscores the importance of empirical knowledge about adaptations in body posture in response to external force demands for developing efficient hierarchies.</p><p id="P22">The relative importance of different variables in the predictive model also provides insight into optimal placement of inertial sensors for posture analysis. For instance, if the pushing and pulling exertions are known to be performed at a fixed handle height in the workplace, then two inertial sensors could suffice (i.e., at T6 and L5/S1). Regardless of the handle height, the three most informative sensor attachment locations were at L5/S1, T6, and the right thigh. This information could serve as useful guidance about optimal placement of body-worn inertial sensors for obtaining the most informative postural kinematics with a minimal set of body-worn sensors.</p><p id="P23">In this analysis, the anthropometry and strength variables were found to be less important compared to posture variables since the response variable consisted of normalized force levels. We expect a greater contribution of these variables if predicting absolute force magnitudes, or if using statistical prediction models where fixed effect variables (e.g., handle height, force level) and random effect variables (e.g., anthropometry, strength measures) are treated differently as in a mixed effects model (e.g., RE-EM tree; <xref rid="R15" ref-type="bibr">Sela &#x00026; Simonoff, 2012</xref>). Either approach would require a larger sample size with diverse demographic and anthropometry characteristics and is the focus of future work.</p></sec></body><back><ack id="S16"><p>Data collection for this study was supported by the training grant T42 OH008455 from the National Institute for Occupational Safety and Health, Centers for Disease Control and Prevention. Data analysis and contents of this publication were developed under a grant from the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR; grant number 90IF0094-01-00). NIDILRR is a Center within the Administration for Community Living (ACL), Department of Health and Human Services (HHS). The contents of this publication do not necessarily reflect the official policies of NIOSH, NIDILRR, ACL, or HHS, nor imply endorsement by the U.S. Government.</p></ack><ref-list><ref id="R1"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Aminian</surname><given-names>K</given-names></name><name><surname>Najafi</surname><given-names>B</given-names></name><name><surname>B&#x000fc;la</surname><given-names>C</given-names></name><name><surname>Leyvraz</surname><given-names>PF</given-names></name><name><surname>Robert</surname><given-names>P</given-names></name></person-group><year>2002</year><article-title>Spatio-temporal parameters of gait measured by an ambulatory system using miniature gyroscopes</article-title><source>Journal of biomechanics</source><volume>35</volume><issue>5</issue><fpage>689</fpage><lpage>699</lpage><pub-id pub-id-type="pmid">11955509</pub-id></element-citation></ref><ref id="R2"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bernmark</surname><given-names>E</given-names></name><name><surname>Wiktorin</surname><given-names>C</given-names></name></person-group><year>2002</year><article-title>A triaxial accelerometer for measuring arm movements</article-title><source>Applied ergonomics</source><volume>33</volume><issue>6</issue><fpage>541</fpage><lpage>547</lpage><pub-id pub-id-type="pmid">12507338</pub-id></element-citation></ref><ref id="R3"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Breiman</surname><given-names>L</given-names></name><name><surname>Friedman</surname><given-names>JH</given-names></name><name><surname>Olshen</surname><given-names>RA</given-names></name><name><surname>Stone</surname><given-names>CJ</given-names></name></person-group><year>1984</year><source>Classification and regression trees</source><publisher-loc>Boca Raton, FL</publisher-loc><publisher-name>Chapman and Hall/CRC</publisher-name></element-citation></ref><ref id="R4"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Castro</surname><given-names>MCF</given-names></name><name><surname>Cliquet</surname><given-names>A</given-names></name></person-group><year>1997</year><article-title>A low-cost instrumented glove for monitoring forces during object manipulation</article-title><source>IEEE Transactions on Rehabilitation Engineering</source><volume>5</volume><issue>2</issue><fpage>140</fpage><lpage>147</lpage><pub-id pub-id-type="pmid">9184900</pub-id></element-citation></ref><ref id="R5"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Cohen</surname><given-names>AL</given-names></name></person-group><year>1997</year><source>Elements of ergonomics programs: a primer based on workplace evaluations of musculoskeletal disorders</source><volume>97</volume><publisher-name>DIANE Publishing</publisher-name></element-citation></ref><ref id="R6"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Cordero</surname><given-names>AF</given-names></name><name><surname>Koopman</surname><given-names>HJFM</given-names></name><name><surname>Van Der Helm</surname><given-names>FCT</given-names></name></person-group><year>2004</year><article-title>Use of pressure insoles to calculate the complete ground reaction forces</article-title><source>Journal of biomechanics</source><volume>37</volume><issue>9</issue><fpage>1427</fpage><lpage>1432</lpage><pub-id pub-id-type="pmid">15275851</pub-id></element-citation></ref><ref id="R7"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>El-Gohary</surname><given-names>M</given-names></name><name><surname>McNames</surname><given-names>J</given-names></name></person-group><year>2012</year><article-title>Shoulder and elbow joint angle tracking with inertial sensors</article-title><source>IEEE Transactions on Biomedical Engineering</source><volume>59</volume><issue>9</issue><fpage>2635</fpage><lpage>2641</lpage><pub-id pub-id-type="pmid">22911538</pub-id></element-citation></ref><ref id="R8"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Faber</surname><given-names>GS</given-names></name><name><surname>Kingma</surname><given-names>I</given-names></name><name><surname>Schepers</surname><given-names>HM</given-names></name><name><surname>Veltink</surname><given-names>PH</given-names></name><name><surname>Van Dieen</surname><given-names>JH</given-names></name></person-group><year>2010</year><article-title>Determination of joint moments with instrumented force shoes in a variety of tasks</article-title><source>Journal of biomechanics</source><volume>43</volume><issue>14</issue><fpage>2848</fpage><lpage>2854</lpage><pub-id pub-id-type="pmid">20674922</pub-id></element-citation></ref><ref id="R9"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Hoffman</surname><given-names>SG</given-names></name></person-group><year>2008</year><article-title>Whole-Body Postures during Standing Hand-Force Exertions: Development of a 3D Biomechanical Posture Prediction Model</article-title><source>Unpublished doctoral dissertation</source><publisher-name>University of Michigan</publisher-name><publisher-loc>Ann Arbor</publisher-loc></element-citation></ref><ref id="R10"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Karatsidis</surname><given-names>A</given-names></name><name><surname>Bellusci</surname><given-names>G</given-names></name><name><surname>Schepers</surname><given-names>HM</given-names></name><name><surname>de Zee</surname><given-names>M</given-names></name><name><surname>Andersen</surname><given-names>MS</given-names></name><name><surname>Veltink</surname><given-names>PH</given-names></name></person-group><year>2016</year><article-title>Estimation of ground reaction forces and moments during gait using only inertial motion capture</article-title><source>Sensors</source><volume>17</volume><issue>1</issue><fpage>75</fpage></element-citation></ref><ref id="R11"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kim</surname><given-names>S</given-names></name><name><surname>Nussbaum</surname><given-names>MA</given-names></name></person-group><year>2014</year><article-title>An evaluation of classification algorithms for manual material handling tasks based on data obtained using wearable technologies</article-title><source>Ergonomics</source><volume>57</volume><issue>7</issue><fpage>1040</fpage><lpage>1051</lpage><pub-id pub-id-type="pmid">24724567</pub-id></element-citation></ref><ref id="R12"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Liaw</surname><given-names>A</given-names></name><name><surname>Wiener</surname><given-names>M</given-names></name></person-group><year>2002</year><article-title>Classification and regression by randomForest</article-title><source>R news</source><volume>2</volume><issue>3</issue><fpage>18</fpage><lpage>22</lpage></element-citation></ref><ref id="R13"><element-citation publication-type="confproc"><person-group person-group-type="author"><name><surname>Lim</surname><given-names>S</given-names></name><name><surname>Case</surname><given-names>A</given-names></name><name><surname>D&#x02019;Souza</surname><given-names>C</given-names></name></person-group><year>2016</year><article-title>Comparative Analysis of Inertial Sensor to Optical Motion Capture System Performance in Push-Pull Exertion Postures</article-title><conf-name>Proceedings of the HFES Annual Meeting</conf-name><volume>60</volume><issue>1</issue><fpage>970</fpage><lpage>974</lpage><publisher-name>SAGE Publications</publisher-name></element-citation></ref><ref id="R14"><element-citation publication-type="book"><collab>R Core Team</collab><year>2016</year><source>R: A language and environment for statistical computing</source><publisher-name>R Foundation for Statistical Computing</publisher-name><publisher-loc>Vienna, Austria</publisher-loc><comment>Downloaded from: <ext-link ext-link-type="uri" xlink:href="http://www.R-project.org/">http://www.R-project.org/</ext-link></comment></element-citation></ref><ref id="R15"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Sela</surname><given-names>RJ</given-names></name><name><surname>Simonoff</surname><given-names>JS</given-names></name></person-group><year>2012</year><article-title>RE-EM trees: a data mining approach for longitudinal and clustered data</article-title><source>Machine learning</source><volume>86</volume><issue>2</issue><fpage>169</fpage><lpage>207</lpage></element-citation></ref><ref id="R16"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Strobl</surname><given-names>C</given-names></name><name><surname>Boulesteix</surname><given-names>AL</given-names></name><name><surname>Zeileis</surname><given-names>A</given-names></name><name><surname>Hothorn</surname><given-names>T</given-names></name></person-group><year>2007</year><article-title>Bias in random forest variable importance measures: Illustrations, sources and a solution</article-title><source>BMC bioinformatics</source><volume>8</volume><issue>1</issue><fpage>25</fpage><pub-id pub-id-type="pmid">17254353</pub-id></element-citation></ref><ref id="R17"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Theado</surname><given-names>EW</given-names></name><name><surname>Knapik</surname><given-names>GG</given-names></name><name><surname>Marras</surname><given-names>WS</given-names></name></person-group><year>2007</year><article-title>Modification of an EMG-assisted biomechanical model for pushing and pulling</article-title><source>International Journal of Industrial Ergonomics</source><volume>37</volume><issue>11</issue><fpage>825</fpage><lpage>83</lpage></element-citation></ref></ref-list></back><floats-group><fig id="F1" orientation="portrait" position="float"><label>Figure 1</label><caption><p>Schematic representation of the experiment apparatus and instrumentation showing anatomical reference locations for the inertial sensors attachment.</p></caption><graphic xlink:href="nihms925225f1"/></fig><fig id="F2" orientation="portrait" position="float"><label>Figure 2</label><caption><p>Structural differences in Model-1: Multiclass prediction with four classes as the response variable (left-panel) and Model-2: Hierarchical structure (right-panel) where handle height is classified at the first stage and then force intensity. Prediction accuracy at each stage is noted under each sub-model (denoted as an oval), and the overall prediction accuracy at the bottom of the panel.</p></caption><graphic xlink:href="nihms925225f2"/></fig><fig id="F3" orientation="portrait" position="float"><label>Figure 3</label><caption><p>Graphs showing the top-five important variables in each stage of the final hierarchical Model-2 (A: handle height at hip vs. shoulder, B: force intensity at hip handle height, C: force intensity at shoulder handle height) by plotting the mean decrease in Gini Index, a measure of relative importance (%) when the corresponding predictor variable is dropped from the model. A greater relative importance suggests greater importance of the predictor variable.</p></caption><graphic xlink:href="nihms925225f3"/></fig></floats-group></article>