<!DOCTYPE article
PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD with MathML3 v1.2 20190208//EN" "JATS-archivearticle1-mathml3.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" article-type="research-article"><?properties manuscript?><front><journal-meta><journal-id journal-id-type="nlm-journal-id">101262796</journal-id><journal-id journal-id-type="pubmed-jr-id">32819</journal-id><journal-id journal-id-type="nlm-ta">J Expo Sci Environ Epidemiol</journal-id><journal-id journal-id-type="iso-abbrev">J Expo Sci Environ Epidemiol</journal-id><journal-title-group><journal-title>Journal of exposure science &#x00026; environmental epidemiology</journal-title></journal-title-group><issn pub-type="ppub">1559-0631</issn><issn pub-type="epub">1559-064X</issn></journal-meta><article-meta><article-id pub-id-type="pmid">32015431</article-id><article-id pub-id-type="pmc">7397525</article-id><article-id pub-id-type="doi">10.1038/s41370-020-0203-9</article-id><article-id pub-id-type="manuscript">HHSPA1608142</article-id><article-categories><subj-group subj-group-type="heading"><subject>Article</subject></subj-group></article-categories><title-group><article-title>TREXMO plus: an advanced self-learning model for occupational exposure assessment</article-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Savic</surname><given-names>Nenad</given-names></name><xref ref-type="aff" rid="A1">1</xref></contrib><contrib contrib-type="author"><name><surname>Lee</surname><given-names>Eun Gyung</given-names></name><xref ref-type="aff" rid="A2">2</xref></contrib><contrib contrib-type="author"><name><surname>Gasic</surname><given-names>Bojan</given-names></name><xref ref-type="aff" rid="A3">3</xref></contrib><contrib contrib-type="author"><name><surname>Vernez</surname><given-names>David</given-names></name><xref ref-type="aff" rid="A1">1</xref></contrib></contrib-group><aff id="A1"><label>1</label>Center for Primary Care and Public Health (Unisant&#x000e9;), University of Lausanne, Route de la Corniche 2, CH-1066 Epalinges-Lausanne, Switzerland</aff><aff id="A2"><label>2</label>Health Effects Laboratory Division (HELD), Exposure Assessment Branch (EAB), National Institute for Occupational Safety and Health (NIOSH), 1095 Willowdale Road, Morgantown, WV 26505, USA</aff><aff id="A3"><label>3</label>Chemicals and Occupational Health Unit, Swiss State Secretariat for Economic Affairs (SECO), Holzikofenweg 36, CH-3003 Bern, Switzerland</aff><author-notes><corresp id="CR1">Nenad Savic, <email>nenad.savic@unisante.ch</email></corresp></author-notes><pub-date pub-type="nihms-submitted"><day>2</day><month>7</month><year>2020</year></pub-date><pub-date pub-type="epub"><day>03</day><month>2</month><year>2020</year></pub-date><pub-date pub-type="ppub"><month>5</month><year>2020</year></pub-date><pub-date pub-type="pmc-release"><day>03</day><month>8</month><year>2020</year></pub-date><volume>30</volume><issue>3</issue><fpage>554</fpage><lpage>566</lpage><!--elocation-id from pubmed: 10.1038/s41370-020-0203-9--><abstract id="ABS1"><p id="P1">In Europe, several occupational exposure models have been developed and are recommended for regulatory exposure assessment. Only some information on the substance of interest (e.g., vapor pressure) and the workplace conditions (e.g., ventilation rate) is required in these models to predict an exposure value that will be later used to characterize the risk. However, it has been shown that models may differ in their predictions and that usually, one of the models best fits a given set of exposure conditions. Unfortunately, there are no clear rules on how to select the best model. In this study, we developed a new modeling approach that together uses the three most popular models, Advanced REACH Tool, Stoffenmanger, and ECETOC TRAv3, to obtain a unique exposure prediction. This approach is an extension of the TREXMO tool, and is called TREXMO+. TREXMO+ applies a machine-learning technique on a set of exposure data with the measured values to split them into smaller subsets, corresponding to exposure conditions sharing similar characteristics. For each subset, TREXMO+ then establishes a regression model with the three REACH tools used as the exposure predictors. The performance of the new model was tested and a comparison was made between the results obtained by TREXMO+ and those obtained by conventional tools. TREXMO+ model was found to be less biased and more accurate than the REACH models. Its prediction differs generally from measurements by a factor of 2&#x02013;3 from measurements, whereas conventional models were found to differ by a factor 2&#x02013;14. However, as the available test dataset is limited, its results will need to be confirmed by larger-scale tests.</p></abstract><kwd-group><kwd>Exposure assessment</kwd><kwd>REACH</kwd><kwd>Advanced REACH tool</kwd><kwd>Stoffenmanager</kwd><kwd>ECETOC TRA</kwd><kwd>TREXMO</kwd></kwd-group></article-meta></front><body><sec id="S1"><title>Introduction</title><p id="P2">In 2007, the European Union parliament adopted a new chemical regulation that concerns Registration, Evaluation, Authorization, and restriction of CHemicals (REACH) [<xref rid="R1" ref-type="bibr">1</xref>]. The purpose of REACH is to improve the protection of human health and the environment against the risk posed by chemicals. To comply with this legislation, for substances manufactured or imported &#x02265;10 tons per year, registrants should conduct a Chemical Safety Assessment (CSA). The CSA includes risk characterization and exposure assessment based on the established exposure scenarios. An exposure scenario is a document describing the conditions that ensure adequate control of risk when a substance is used in a specific task or several processes or uses. These conditions defined for every step of the manufacturing, any anticipated use down in the supply chain, and every substance in a product. Collection of exposure measurements to support a CSA requires a large sampling size and high costs. The sampling size can be further enlarged to cover the between-and within-user variability [<xref rid="R2" ref-type="bibr">2</xref>]. Statistical exposure models, such as ECETOC TRA [<xref rid="R3" ref-type="bibr">3</xref>, <xref rid="R4" ref-type="bibr">4</xref>], on the other hand, present a cheap and fast assessment alternative. Although the exposure measurements are still considered the gold standard and their collection should not be completely replaced [<xref rid="R5" ref-type="bibr">5</xref>], the models can discriminate between well-controlled situations and situations that may pose risk to the human health.</p><p id="P3">Within the context of REACH, several generic exposure models are recommended for the exposure assessment purposes [<xref rid="R1" ref-type="bibr">1</xref>]. Tier 1 models, such as EMKG-EXPO-TOOL [<xref rid="R6" ref-type="bibr">6</xref>], ECETOC TRAv2 [<xref rid="R3" ref-type="bibr">3</xref>], and TRAv3 [<xref rid="R4" ref-type="bibr">4</xref>], are simple screening tools that are supposed to predict more conservative (i.e., protective) estimates, as compared with higher tier tools, such as Advanced REACH Tool (ART) [<xref rid="R7" ref-type="bibr">7</xref>, <xref rid="R8" ref-type="bibr">8</xref>] and Stoffenmanager (SM) [<xref rid="R9" ref-type="bibr">9</xref>, <xref rid="R10" ref-type="bibr">10</xref>]. To calculate a tier 1 estimate, a few exposure parameters regarding substance and workplace properties are required. EMKG-EXPO-TOOL, for example, uses only three parameters, i.e., volatility (for liquids) or dustiness band (for solids), amount of substance, and applied exposure control. SM and ART, however, require more exposure descriptive information to calculate estimates that are supposed to be more precise compared with the results of tier 1 models. ART, for example, besides the vapor pressure (VP) of a substance, requires its concentration and activity coefficient to assess how the substance properties contribute in the final exposure. Also, these higher tier tools provide estimates at different percentiles of the exposure distribution, i.e., 50th, 75th, 90th, 95th, and 99th (only ART).</p><p id="P4">A broad study, known as the eteam study [<xref rid="R11" ref-type="bibr">11</xref>], has extensively compared several tier 1 models, including ECETOC TRAv2 and v3, EMKG-EXPO-TOOL, and SM (tier 1.5), with nearly 4000 exposure measurements [<xref rid="R12" ref-type="bibr">12</xref>]. The authors reported that overall, the tier 1 models performed satisfactorily for a wider range of considered exposure situations (ESs), by overestimating the corresponding exposure measurement data; this was the desired outcome as the tier 1 models are intended to overestimate the measurements and thus behave more protective. For certain exposure conditions, the study also found underestimations of the measured values. In another study [<xref rid="R13" ref-type="bibr">13</xref>], ART (the only tier 2 model of REACH) was compared with almost 600 measurements from Switzerland. By evaluating the fraction of the measurements found within the 90% confidence interval of the 50th and 90th percentiles, Savic et al. [<xref rid="R13" ref-type="bibr">13</xref>] found that ART tends to accurately predict exposures to volatile liquids and powders. The authors also reported that ART overestimates exposures to volatile liquids but underestimates exposures to powders and solids. Similarly, Lee et al. [<xref rid="R14" ref-type="bibr">14</xref>, <xref rid="R15" ref-type="bibr">15</xref>] tested the performance of these models against a U.S. dataset for liquids with VP &#x0003e; 10 Pa and reported results similar to those of the eteam for the tier 1 models and SM. For ART, however, Lee et al. [<xref rid="R15" ref-type="bibr">15</xref>] found a tendency to underestimate exposure measurements. Several other studies also focused on the performance of these models under REACH [<xref rid="R8" ref-type="bibr">8</xref>, <xref rid="R16" ref-type="bibr">16</xref>&#x02013;<xref rid="R18" ref-type="bibr">18</xref>]. Most of these studies showed that the models could be reliable for certain ESs, but in some cases, their algorithms need to be improved. Nevertheless, the models&#x02019; performance remains unknown for a great number of ESs for which no adequate data are available. To the best of our knowledge, none of the published studies has established straightforward rules on which model is the best choice for a given set of ESs.</p><p id="P5">Furthermore, Savic et al. [<xref rid="R19" ref-type="bibr">19</xref>] compared, in silico, the estimates of ART, SM (version 4.0) [<xref rid="R10" ref-type="bibr">10</xref>], and TRAv3 for 300,000 combinations of exposure parameters. The authors addressed ESs for which the modeled exposures can differ by several orders of magnitude, indicating that a wrong selection of model may have serious consequences on assessment of risk. Because no workplace measurements were involved in the inter-model comparison [<xref rid="R19" ref-type="bibr">19</xref>] (i.e., in silico study), it was impossible to determine the actual performance of the three REACH models. Savic et al. [<xref rid="R20" ref-type="bibr">20</xref>] then introduced the usefulness of TREXMO, which promotes and facilitates the simultaneous use of sereval exposure models.</p><p id="P6">The purpose of this study, therefore, was to develop a new modeling approach that combines the exposure outputs of three existing REACH models to derive a refined prediction. The three inhalation models used as the exposure predictors in our model were ART, SM (version 4.0) [<xref rid="R10" ref-type="bibr">10</xref>], and TRAv3. We developed an algorithm that adjusts its prediction coefficients depending on the evaluated exposure conditions. This means that the model will use available exposure data to learn about the models&#x02019; weaknesses and strengths for different ESs. Whenever new exposure data are supplied to the model, it will perform a self-refinement of its prediction coefficients. We hypothesized that this new approach, compared with the three REACH models, would perform better in terms of bias and accuracy with an assumption that those three models were developed using different and independent exposure data. To calculate its estimates, TREXMO+ will only use indirectly the data on which the three models are based. Depending on how similar is a given exposure situation with the data in these three models, the new model will assign them different prediction coefficients.</p></sec><sec id="S2"><title>Methods</title><sec id="S3"><title>TREXMO</title><p id="P7">TREXMO [<xref rid="R20" ref-type="bibr">20</xref>], an integrated tool of six exposure models, i.e., ART, SM, ECETOC TRAv3, MEASE, EMKG-EXPO-TOOL, and EASE, is designed to predict estimates of the six models simultaneously. By starting from a set of the parameters established in one of the models (preferably ART), TREXMO recommends the most appropriate parameters in the other five. For a given ES, six exposure predictions can thus be provided from a single parameter set. TREXMO v2, however, neither provides its own estimates nor recommends which of the six predictions could be the most suitable, regarding a given ES, for the risk characterization. Detailed information is provided in Savic et al. [<xref rid="R20" ref-type="bibr">20</xref>].</p></sec><sec id="S4"><title>TREXMO+</title><p id="P9">This is an advanced model developed based on the inter-model translations established for TREXMO. It was consequently named TREXMO+. This new model is intended to derive its exposure estimates by using three REACH models, i.e., ART, SM, and TRAv3, as its independent predictors. The remaining three models of TREXMO were not considered in this study because of the following reasons:</p><p id="P10">MEASE. Regarding the inhalation exposure, the model predicts the values similar to those of TRAv3. Its contribution in TREXMO+ was thus not expected to improve the performance.</p><p id="P11">EMKG-EXPO-TOOL and EASE. These two models are conceptually much simpler compared with the three considered once. Also, they do not predict the point value estimates, but the range. Moreover, EASE was used as a basis for the development of ECETOC TRA and it can thus be expected that the exposure data, on which the two models are based, is mostly overlapped.</p></sec><sec id="S5"><title>TREXMO+ concept</title><p id="P12">For both ART and SM, the 50th percentile estimates of the exposure distribution is the direct output of their calibrations against the exposure measurements [<xref rid="R8" ref-type="bibr">8</xref>, <xref rid="R10" ref-type="bibr">10</xref>]. According to the ECETOC&#x02019;s technical guidance [<xref rid="R4" ref-type="bibr">4</xref>], the exposure estimates in TRAv3 are considered to represent the 75th percentile estimate of the exposure distribution. Because neither the model calculations nor the supporting material provides a way to obtain the 50th percentile, the 75th percentile estimate was used here. For the model developed in this study, which predicts the geometric mean (GM), the choice of the percentiles of the three REACH models was not crucial, because these models are used only as predictors of the exposure. This is explained in detail next.</p><p id="P13">To predict exposure estimates of the three REACH models, appropriate exposure parameters (such as VP and activity task) must be coded for the given models. To automatize this step, we integrated the parameter translation rules of TREXMO [<xref rid="R20" ref-type="bibr">20</xref>] into our model. ART was used as the starting model as recommended by Savic et al. [<xref rid="R20" ref-type="bibr">20</xref>]. Since some input parameters in ART can be translated into multiple ways (especially for PROC and type of use in TRAv3) and thus lead to different exposure estimates, TREXMO+ for these cases is programmed to require additional expert judgments. Fortunately, the data (<xref rid="T2" ref-type="table">Table 2</xref>) that were used in our study included a PROC value for each ES. For a smaller fraction of ESs, <italic>professional use</italic> (i.e., more conservative option) was selected to represent the type of use for TRAv3, since the corresponding information was missing.</p><p id="P14">The structure of TREXMO+ is shown in <xref rid="F1" ref-type="fig">Fig. 1a</xref>. The algorithm consists of three layers: (1) classification, (2) learning, and (3) assessment.</p><sec id="S6"><title>Classification layer</title><p id="P15">Exposure data containing the exposure parameters coded for ART (+PROC/TRAv3 per ES) are supplied to this layer. This database must not include exposure data that were used to calibrate any of ART, SM, or TRAv3. Exposure data from two different sources were available for the purpose of this study.</p><p id="P16">The classification layer is designed to split the supplied data into similarly exposed groups with the addressed exposure conditions. To classify the data, we used a conditional inference regression tree (RT) [<xref rid="R21" ref-type="bibr">21</xref>]. This is a machine-learning (ML) method [<xref rid="R22" ref-type="bibr">22</xref>] that constructs a tree-like structure from a dataset containing a continuous dependent variable (such as exposure measurement data). The method has been explained in detail by Strasser and Weber [<xref rid="R21" ref-type="bibr">21</xref>]. A brief explanation is provided in the next paragraph. In this study, the RT algorithm was used only to classify the exposure dataset into smaller subsets and not to derive a prediction algorithm. For each subset of the data obtained by using the RT, the corresponding prediction coefficients are established in the next, learning, layer of TREXMO+.</p><p id="P17"><xref rid="F1" ref-type="fig">Figure 1b</xref> illustrates an example of the workflow of the RT algorithm. The most-top, root node (node 1) contains data consisting of two independent variables (<italic>x1</italic> and <italic>x2</italic>) to determine a dependent (or response) variable (<italic>y</italic>) using a function of <italic>f(x1, x2)</italic>. At this point, the response covers a wider range of possibilities between <italic>y1</italic> and <italic>y2</italic>, shown in the bar graph in <xref rid="F1" ref-type="fig">Fig. 1b</xref>. RT tests a null hypothesis of independence of the two (<italic>x</italic>) predictors with the response. As the independence measure, a <italic>p</italic> value corresponding to a given level of confidence (such as 95% confidence interval) was used. If the null hypothesis cannot be rejected, then a binary split is performed. When more splits are possible, the most discriminative one (with the lowest <italic>p</italic> value) is performed. In <xref rid="F1" ref-type="fig">Fig. 1b</xref>, the first, most discriminant, binary split is found for <italic>x1</italic> = <italic>90</italic> (arbitrary number used for explaining the logic). The data are then split in two smaller subsets; node 2 contains the records with <italic>x1 &#x0003c; 90</italic>, while node 3 contains only the data with <italic>x1</italic> &#x02265; <italic>90</italic>. Compared with the root node, the response in these two child (internal) nodes is narrower. Every binary split results in narrower data and the response with smaller standard deviations. The recursive data splitting continues until the null hypothesis cannot be rejected. The nodes that cannot be split are leaf (or terminal) nodes.</p><p id="P18">In this study, the variables (as such, in <xref rid="F1" ref-type="fig">Fig. 1b</xref>), for which the independence with the response was tested, were the exposure determinants of ART. <xref rid="T1" ref-type="table">Table 1</xref> lists all exposure determinants, grouped by the principal modifying factor of ART [<xref rid="R7" ref-type="bibr">7</xref>]. The RT algorithm evaluated how significant these determinants are in predicting the measured values. Only VP and concentration were used as continuous variables. It is important to keep the independence between the used variables. For activity emission potential, for example, the parameter list for underlying determinants (such as <italic>amount of product</italic>) depends on activity class (such as <italic>surface spraying of liquid products</italic>). These underlying determinants were thus not used as the variables in RT.</p></sec><sec id="S7"><title>Learning layer</title><p id="P19">Here, TREXMO+ evaluates the regression coefficients (<italic>w</italic>) in <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref> for each leaf node from the previous layer (see <xref rid="F1" ref-type="fig">Fig. 1a</xref>). It is assumed that the performance of the three models (ART, SM, and TRAv3) varies less over different conditions in the leaf nodes, as compared with data in the root node. For each leaf node, the regression analysis in <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref> is conducted. It uses the three models&#x02019; estimates (<italic>E</italic>) to predict the GM of measurements.
<disp-formula id="FD1"><label>(1)</label><mml:math display="block" id="M1"><mml:mrow><mml:mtext>GM</mml:mtext><mml:mo>=</mml:mo><mml:msub><mml:mi>w</mml:mi><mml:mrow><mml:mtext>ART</mml:mtext></mml:mrow></mml:msub><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mtext>ART</mml:mtext></mml:mrow></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>w</mml:mi><mml:mrow><mml:mtext>SM</mml:mtext></mml:mrow></mml:msub><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mtext>SM</mml:mtext></mml:mrow></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>w</mml:mi><mml:mrow><mml:mtext>TRA</mml:mtext></mml:mrow></mml:msub><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mtext>TRA</mml:mtext></mml:mrow></mml:msub><mml:mo>+</mml:mo><mml:mi>&#x003b5;</mml:mi></mml:mrow></mml:math></disp-formula></p><p id="P20">For <italic>n</italic> number of leaf nodes, this layer outputs a (3 &#x000d7; <italic>n</italic>) matrix containing three regression coefficients (for ART, SM, and TRAv3) per node. These regression coefficients define how a given predictor (i.e., exposure model) determines the GM found in the SUVA and NIOSH datasets. As already mentioned, it is not crucial to know which percentile of the exposure is used for these three predictors. If, for example, the 90th percentile was used for SM (or ART), it would not change the performance of the final prediction of TREXMO+, but only the evaluated regression coefficient of the given model. This means that even if unitless predictors were used, the outcomes from <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref> would be still prediction of GM in mg/m<sup>3</sup>.</p><p id="P21">Since all predictors in <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref> are REACH exposure models, some collinearity could exist between them. To address this issue, variance inflation factor (VIF) [<xref rid="R23" ref-type="bibr">23</xref>, <xref rid="R24" ref-type="bibr">24</xref>] was evaluated for each leaf node as a measure of collinearity between the predictor models. The predictors for which VIF was higher than five [<xref rid="R24" ref-type="bibr">24</xref>] was excluded from <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref> and the regression coefficients were reevaluated for the remaining predictors. Furthermore, R-squared was calculated for each node to show how much of the variance is explained by using the model in <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref>.</p></sec><sec id="S8"><title>Assessment layer</title><p id="P22">For a new ES, this layer predicts an exposure estimate. For the given ES, it requires a set of ART exposure parameters to be provided. Following the rules established by Savic et al. [<xref rid="R20" ref-type="bibr">20</xref>]; TREXMO+ performs the parameter translations to SM and TRAv3 and calculates the three estimates of the three REACH models. In the next step, TREXMO+ calls the RT from its first layer. Based on the provided ART parameters, the algorithm assigns the given ES to an appropriate leaf node. TREXMO+ then loads the corresponding regression coefficients from its second layer and calculates a final TREXMO+ estimate by using <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref>.</p></sec></sec><sec id="S9"><title>Evaluation of RTs and regression coefficients</title><p id="P23"><xref rid="T2" ref-type="table">Table 2</xref> shows the exposure data used in the classification layer of TREXMO+. The complete data were coded using the exposure parameters of ART. Also, a PROC and type of use parameter were assigned to each ES to allow the calculations in TRAv3. For the ESs with more than one exposure measurement, the GM was calculated. 80% of the data was selected into the training set, i.e., the TREXMO+ development, while the remaining data&#x02014;the testing set&#x02014; were used to test the performance of the new models (explained in the next section). The redistribution of exposure data into two subsets (the training set and testing set) was randomly conducted by iterating until an RT with the maximal possible number of the binary splits was achieved. Then, an RT based on the training set was established and compared with the RT from the previous iteration. The goal was to cover as much variance as possible in the training set in order to result in more binary splits. This step is especially important for the exposure types in which fewer data were available (that is, powders and solids) and a single random selection may have resulted in no splits.</p><p id="P24">The ART parameter sets (+PROC and type of use), with their corresponding GMs, were supplied to the classification layer of TREXMO+. One RT and one matrix holding the regression coefficients (<italic>w</italic> in <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref>) were obtained for each of the exposure types, that is, volatile liquids, powders, and solids.</p><p id="P25">It is important to address that the official TREXMO+ tool includes the RTs and the matrices holding the evaluated regression coefficients for the whole data (both training and testing set). The RT algorithm and the coefficient matrices were evaluated for the total data. Again, one RT and one matrix with these coefficients were evaluated for each exposure type, separately. The results are given in <xref rid="SD2" ref-type="supplementary-material">Figs. S1</xref>&#x02013;S3 and <xref rid="SD2" ref-type="supplementary-material">Table S1</xref> (see <xref rid="SD2" ref-type="supplementary-material">Supplementary Information 2</xref>).</p></sec><sec id="S10"><title>TREXMO+ validation</title><p id="P26">For each ES, based on a given set of the exposure parameters, the third layer of TREXMO+ loaded the corresponding coefficients from its second layer. The TREXMO+ predictions were then calculated by using these coefficients with the estimates of ART, SM, and TRAv3, in <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref>.</p><p id="P27">Several statistical analyses were conducted to investigate the performance of TREXMO+ and to compare the results with those predicted individually from three models. Residuals (<xref rid="FD2" ref-type="disp-formula">Eq. 2</xref>), measuring the difference between modeled <inline-formula><mml:math display="inline" id="M2"><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mover accent="true"><mml:mi>y</mml:mi><mml:mo>^</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:math></inline-formula> and individual exposure measurement (i.e., not GM) (<italic>y</italic>), were calculated and plotted against the measurements. According to <xref rid="FD2" ref-type="disp-formula">Eq. 2</xref>, positive residuals mean overestimation, while negative residuals implicate underestimation of the measurements by the modeled predictions.
<disp-formula id="FD2"><label>(2)</label><mml:math display="block" id="M3"><mml:mrow><mml:msub><mml:mi>r</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mtext>log</mml:mtext><mml:mspace width="1pt"/><mml:msub><mml:mover accent="true"><mml:mi>y</mml:mi><mml:mo>^</mml:mo></mml:mover><mml:mi>i</mml:mi></mml:msub><mml:mo>&#x02212;</mml:mo><mml:mtext>log</mml:mtext><mml:mspace width="1pt"/><mml:msub><mml:mi>y</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:math></disp-formula></p><p id="P28">LOcally-WEighted Scatterplot Smoothing (LOWESS) [<xref rid="R25" ref-type="bibr">25</xref>] was used to present the changes of the residuals for different models in the same graph. LOWESS converted residual data points into smoothed lines.</p><p id="P29">Relative bias (<xref rid="FD3" ref-type="disp-formula">Eqs. 3</xref> and <xref rid="FD4" ref-type="disp-formula">4</xref>) and accuracy (<xref rid="FD5" ref-type="disp-formula">Eq. 5</xref>) were also evaluated. Accuracy is calculated as mean absolute error (MAE). While the former shows how much, overall, the models overestimate or underestimate exposure; the latter quantifies the average distance between the model&#x02019;s central estimate and GM of the measurements. Because the bias (<xref rid="FD3" ref-type="disp-formula">Eq. 3</xref>) accounts for the sign of the log-difference, it cannot be used to quantify the average distance between the measured and modeled values.
<disp-formula id="FD3"><label>(3)</label><mml:math display="block" id="M4"><mml:mrow><mml:mtext>bias</mml:mtext><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>n</mml:mi></mml:mfrac><mml:munderover><mml:mstyle><mml:mo>&#x02211;</mml:mo></mml:mstyle><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mtext>log</mml:mtext><mml:mspace width="1pt"/><mml:msub><mml:mover accent="true"><mml:mi>y</mml:mi><mml:mo>^</mml:mo></mml:mover><mml:mi>i</mml:mi></mml:msub><mml:mo>&#x02212;</mml:mo><mml:mtext>log</mml:mtext><mml:mspace width="1pt"/><mml:msub><mml:mi>y</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:math></disp-formula>
<disp-formula id="FD4"><label>(4)</label><mml:math display="block" id="M5"><mml:mrow><mml:mtext>relative</mml:mtext><mml:mspace width="1pt"/><mml:mtext>bias</mml:mtext><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mtext>bias&#x000a0;</mml:mtext></mml:mrow></mml:msup><mml:mo>&#x02212;</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>&#x000d7;</mml:mo><mml:mn>100</mml:mn><mml:mi>%</mml:mi></mml:mrow></mml:math></disp-formula>
<disp-formula id="FD5"><label>(5)</label><mml:math display="block" id="M6"><mml:mrow><mml:mtext>accuracy</mml:mtext><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>n</mml:mi></mml:mfrac><mml:munderover><mml:mstyle><mml:mo>&#x02211;</mml:mo></mml:mstyle><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mo>|</mml:mo><mml:mrow><mml:mtext>log</mml:mtext><mml:mspace width="1pt"/><mml:msub><mml:mover accent="true"><mml:mi>y</mml:mi><mml:mo>^</mml:mo></mml:mover><mml:mi>i</mml:mi></mml:msub><mml:mo>&#x02212;</mml:mo><mml:mtext>log</mml:mtext><mml:mspace width="1pt"/><mml:msub><mml:mrow><mml:mtext>GM</mml:mtext></mml:mrow><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>|</mml:mo></mml:mrow></mml:mrow></mml:math></disp-formula></p><p id="P30">Since the ML algorithm in the classification layer of TREXMO+ can also be used to predict GM, the corresponding relative bias and accuracy were also evaluated.</p><p id="P31">Finally, regression parameters, slope and intercept, were calculated for TREXMO+ and its three predictor models. The log-estimates of each model were plotted against the log-exposure measurements.</p></sec><sec id="S11"><title>Software</title><p id="P32">The entire study was performed in R language and environment for statistical computing [<xref rid="R26" ref-type="bibr">26</xref>], version 3.5.0. To establish the RTs needed for the first layer of TREXMO+, <italic>ctree</italic> function was called from R&#x02019;s <italic>party</italic> package. For the multicollinearity test, <italic>car</italic> package and its function <italic>vif</italic> were used. Furthermore, for the graphs in this paper, <italic>ggplot</italic> function from <italic>ggplot2</italic> package [<xref rid="R27" ref-type="bibr">27</xref>] was used.</p></sec></sec><sec id="S12"><title>Results</title><sec id="S13"><title>Established RTs and regression coefficients</title><p id="P33"><xref rid="F2" ref-type="fig">Figure 2</xref> shows RTs established for the training dataset for volatile liquids (A) and powders (B) (<xref rid="FD1" ref-type="disp-formula">Eq. 1</xref>). For solids, the exposure conditions were insufficiently different to allow for a statistically significant binary split. The iterative random redistribution of the ESs failed to result in a training set with enough variance for a binary split. Solid ESs thus remained in the root node, and no RT is shown for solids in <xref rid="F2" ref-type="fig">Fig. 2</xref>. For liquids, three determinants&#x02014;fugacity, activity (sub)classes, and local controls&#x02014;were found to significantly affect the GM of the measurements. For powders, these determinants were dustiness and concentration.</p><p id="P34"><xref rid="T3" ref-type="table">Table 3</xref> shows the calculated regression coefficients for the three REACH models and different nodes in <xref rid="F2" ref-type="fig">Fig. 2</xref> (<italic>w</italic>; see <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref>). For solids, because no RT was established, one set of these coefficients was obtained. A high collinearity (i.e., VIF &#x0003e; 5) was often found between ART and SM. Therefore, most of the time, only two models (e.g., ART and TRAv3 or SM and TRAv3) were used as predictors in <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref>. Finally, the found R-squared values in <xref rid="T3" ref-type="table">Table 3</xref> varied drastically from node to node. For liquids, for example, while the regression explained 86% for node 6, only 13% was explained for node 3.</p><p id="P35">To illustrate the concept of TREXMO+, residuals with LOWESS smoothing were also evaluated for the training dataset. <xref rid="F3" ref-type="fig">Figure 3</xref> (the training dataset) shows how TREXMO+ balances its estimates by shifting it to a REACH model that locally performs best. Our model, however, kept the overall behavior of the three models. This is especially evident for liquids, where the three REACH models tend to overestimate lower exposures and underestimate higher exposures. Here, TREXMO+ resulted in a similar, but alleviated, trend.</p></sec><sec id="S14"><title>Validation results</title><p id="P36">The RTs in <xref rid="F2" ref-type="fig">Fig. 2</xref> and the corresponding coefficients in <xref rid="T3" ref-type="table">Table 3</xref> were used to calculate the TREXMO+ estimates for the ESs in the testing dataset.</p><p id="P37"><xref rid="F3" ref-type="fig">Figure 3</xref> (the testing dataset) shows trends in the residuals for the three REACH models and TREXMO+. As for the training set, the models tend to overestimate lower exposures and underestimate higher exposures. The TREXMO+ line is the least deviated from the central &#x0201c;zero&#x0201d; line. This means that the mentioned trend is alleviated for TREXMO+. It also means that compared with the three REACH models, the difference between the TREXMO+ estimates and the exposure measurements is smaller.</p><p id="P38">The results found for relative bias (<xref rid="T4" ref-type="table">Table 4</xref>) show that TREXMO+ underestimates the measurements (relative bias from &#x02212;24 to &#x02212;41%). Even lower values (relative bias from &#x02212;49 to &#x02212;90) were found when the ML algorithm was used to assess the exposure. Except for solids, for which ART and SM were found to underestimate the exposures, the REACH models were found to overestimate more frequently the measurements. For powders, for example, the extent and the number of overestimations by ART resulted in a high relative bias (=1004%). <xref rid="F3" ref-type="fig">Figure 3</xref> illustrates this finding (the testing dataset); the ART&#x02019;s line is above the central &#x0201c;zero&#x0201d; line over almost the entire domain.</p><p id="P39">For all three exposure types, TREXMO+ was the most accurate model (<xref rid="T4" ref-type="table">Table 4</xref>). This means that the average difference between the estimates of TREXMO+ and the calculated GM was the smallest. Regarding different exposure forms, this difference was found to be, on average, between 0.31 and 0.48. As the accuracy is expressed on the log-scale (see <xref rid="FD5" ref-type="disp-formula">Eq. 5</xref>), the estimates of TREXMO+ are expected to be two to three times lower or higher than the measured GM. The second most accurate was when using only ML of TREXMO+, for which MAE was 0.31&#x02013;0.61. Only a small difference (MAE, 0.31 vs 0.34) was found between TREXMO+ and ML for solids, which can be explained by the fact that the regression layer failed to perform binary splits for this form of exposure. Of the three REACH models, SM was the most accurate, with a log-difference of 0.53&#x02013;0.66 (or three to five times GM).</p><p id="P40">Finally, the relationship between the modeled and measured exposures is shown in <xref rid="F4" ref-type="fig">Fig. 4</xref> and <xref rid="T5" ref-type="table">Table 5</xref>. The corresponding 90% confidence interval for ART and the 90% confidence interval and prediction interval for TREXMO+ are also plotted in <xref rid="F4" ref-type="fig">Fig. 4</xref>.</p><p id="P41">For liquids, all four models were significantly correlated with the measurements (<italic>p</italic> &#x0003c; 0.05). The variance in the measured exposure was best explained by TREXMO+ (R-squared = 0.41). For powders, the correlation was significant only for TREXMO+, whereas SM and TRAv3 were even negatively correlated with the measurements. For SM, this was also the case for solids (slope = &#x02212;0.08). The relationship between the modeled and measured exposure for this exposure form was significant only for TRAv3 and TREXMO+.</p></sec></sec><sec id="S15"><title>Discussion</title><p id="P42">In this study report, we presented the development and validation of a new, &#x0201c;meta&#x0201d; exposure model, called TREXMO+. This model applies the concept of the multi-model approach of TREXMO [<xref rid="R20" ref-type="bibr">20</xref>]. By using between-model translations to calculate a refined estimate, TREXMO+ calculates exposure for three REACH models: ART, SM, and TRAv3. Furthermore, the model incorporates an RT algorithm to account for different performances of these individual models for different exposure conditions. This concept, in which three existing models are used only as exposure predictors of another model, was expected to perform better than the predictor models alone. A small fraction (20%) of the measured data was used to test the performance of TREXMO+ and confirmed this hypothesis. For all tested criteria, including residuals, bias, accuracy, and regression, TREXMO+ led to improved results over the three REACH models considered.</p><sec id="S16"><title>RTs and regression coefficients</title><p id="P43">The splitting criteria in the established RTs, which were the ART exposure parameters, were in accordance with the results in Savic et al. [<xref rid="R19" ref-type="bibr">19</xref>]. Those authors reported that the parameters of substance properties (such as VP or dustiness), activity type (such as spraying), and local control have the most significant influence on determining measured exposure levels. Moreover, a sensitivity analysis [<xref rid="R28" ref-type="bibr">28</xref>] and a computational comparison [<xref rid="R19" ref-type="bibr">19</xref>] have also shown that these parameters affect most the final models&#x02019; predictions and the differences among the predictions of ART, SM, and TRAv3. Regarding the RTs established for different exposure types, the most binary splits (three) were obtained for volatile liquids. This was driven by three factors: data size, variability between ESs, and how well the exposure parameters account for the variance in the measurements. Nearly 65% of the exposure data in this study addressed the exposure to volatile liquids. The given ESs for liquids covered a wide range of different exposure conditions. For example, the data included substances with VPs from 18 Pa to 59 kPa. Also, all activity (sub)classes of ART were covered. A smaller amount of data and less variable ESs led to a smaller RT (that is, two splits) for powders, in comparison with liquids. The RT did not branch further to include splits for different activities and local controls. Finally, the training data, on which the RTs were based, included only 19 ESs for solids. These data were insufficiently different to result in a significant binary split. Almost all ESs (18 of 19) addressed exposure to wood dusts with the contaminant concentration of 100%. Almost half of these ESs included no local control, whereas exposure in the other half was controlled with <italic>on-tool extraction</italic>.</p><p id="P44">In some cases, the binary splits in <xref rid="F2" ref-type="fig">Fig. 2</xref> were not in accordance with the dimensionless scores assigned to the parameters of ART. It is expected that exposure parameters with substantially different scores would be classified in different nodes. However, the <italic>extremely fine dust</italic> (score = 1.0) category appeared in the same node as <italic>firm granules</italic> (score = 0.01) instead with <italic>fine dust</italic> (score = 0.3). The explanation for this can be found in Savic et al. [<xref rid="R13" ref-type="bibr">13</xref>], where the same SUVA data were used to investigate the performance of ART. In that publication, the authors stated that where the raw data were unclear, the most conservative parameters were assigned. This means that most likely the selection of <italic>extremely fine dusts</italic> was too conservative. A more appropriate parameter would be <italic>coarse dust</italic> for the ESs with <italic>extremely fine dust</italic>.</p><p id="P45">This study also found strong collinearity between ART and SM for certain exposure conditions (<xref rid="T3" ref-type="table">Table 3</xref>). For these two exposure models, similar results were obtained in our previous, in silico, study [<xref rid="R19" ref-type="bibr">19</xref>]. These results could be explained by the fact that both ART and SM follow the same source-receptor approach. Also, this could mean that there is some overlapping between the exposure data used to develop these models.</p></sec><sec id="S17"><title>Model validation</title><p id="P46">Overall, the results showed that TREXMO+ is less biased and more accurate. The estimates of this model better explained the variance (that is, higher R-squared) of the measurements. Somewhat higher underestimations and lower accuracy were found when ML alone was used to predict the exposure. This was the least noticeable for solids, for which no binary split was obtained. However, it is difficult to imagine how the two approaches will differ as more data becomes available. Moreover, the relatively small data size and the similarity between the exposure conditions (both from SUVA and NIOSH) in the training and testing datasets might have biased the final results and conclusions.</p><p id="P47">For all three exposure types considered, the correlation between TREXMO+ and the measurements was significant at the 95% confidence level. This was not the case, however, for the REACH models for powders and solids, for which only TRAv3 had a significant correlation with the measurements. The three REACH models were found to overestimate the measurements. For liquids (excluding SM) and powders, these models were too conservative. With relative bias between 300 and 1000%, powders were the most overestimated. Such overprotectiveness might obscure the practical use of these models.</p><p id="P48">Conversely, TREXMO+ underestimated the measurements. The extent of these underestimations (that is, the magnitude of absolute value of relative bias) was small, as compared with the overestimations obtained for the REACH models. Overall, TREXMO+ was thus the least biased model, a determination which was the goal of this study. This model was more biased than SM only for liquids. According to <xref rid="FD3" ref-type="disp-formula">Eq. 3</xref>, bias is a measure of the average difference between the modeled and measured exposures. If, for example, the number and extent of over- and underestimates is equal, then the bias in <xref rid="FD3" ref-type="disp-formula">Eq. 3</xref> equals zero. However, this does not mean that the model is not locally biased (e.g., for lower exposures). A better representation on how a model performs locally was thus obtained with LOWES smoothing of the residuals in <xref rid="F3" ref-type="fig">Fig. 3</xref>. For liquids, the residuals for TREXMO+ were closer to the measured exposure over the measurements range. For SM, however, the extent of overestimation of lower exposures and underestimation of higher exposures was greater than for our model. The results for accuracy confirmed these findings. It was shown that the estimates of TREXMO+, on average, differ only two to three times from the GM of the measured exposures.</p><p id="P49">Regarding the relationship with the measurements, only the estimates of TREXMO+ were significantly correlated with all three exposure types considered. For powders and solids, SM and TRAv3 (only for powders) were even negatively correlated. However, this may be due to the relatively small number of measurements in the testing data, which were also the most distributed within a small range (0.1&#x02013;10 mg/m<sup>3</sup>).</p><p id="P50">The recommendations on the risk management measures that should be communicated down the supply chain when assessing the exposure by using TREXMO+ should be the same as with ART. This is because the users of TREXMO+ will require the same exposure information in TREXMO+ as in ART. Another reason is that, regarding its concept, TREXMO+ is expected to calculate values that could be seen as tier 2 estimates.</p></sec><sec id="S18"><title>Limitations</title><p id="P51">Two major limitations of this study were the amount of data and the reliability of the established exposure parameters for ART. As already mentioned, most of the ESs addressed exposure to volatile liquids. Although for this exposure type, 37 different ESs were used to investigate the performances of TREXMO+, only 14 powder ESs and 5 solid ESs were in the testing dataset.</p><p id="P52">Regarding the quality of information in the two exposure databases, it was more likely that erroneous selection of input parameters for ART occurred when interpreting the SUVA dataset. This statement is drawn from the fact that only two assessors (one expert and one trainee) interpreted the raw SUVA data. The input parameters for the NIOSH database were coded based on the consensus between six assessors from different agencies [<xref rid="R15" ref-type="bibr">15</xref>]. Nevertheless, this might create a space for mistakes that could lead to, for example, miscoding of the dustiness categories.</p><p id="P53">In addition, the current version of TREXMO+ predicts only the GM exposure values, while the 90th percentile is used under REACH for risk characterization [<xref rid="R1" ref-type="bibr">1</xref>]. When more data becomes available, required statistical analyses will be conducted to provide the parameters for the calculation of higher percentiles in TREXMO+.</p></sec></sec><sec id="S19"><title>Conclusion and outlook</title><p id="P54">Compared with the three REACH models, TREXMO+ performed better with regard to bias, accuracy, and its correlation with measurements. The most important outcome is that the model&#x02019;s predictions differ only by a factor between 2 and 3 in comparison with corresponding exposure measurements, relatively smaller than factors of the three REACH models. The small testing dataset, however, might raise concerns on the validity of these results, especially for powders and solids. More exposure data will thus be needed to further test the performances of TREXMO+ or refine its algorithm. It is important to note that a future refinement of any of the predictor models will also improve the predictions of TREXMO+.</p><p id="P55">The new model solves the problem with the multi-models approach recommended in Savic et al. [<xref rid="R20" ref-type="bibr">20</xref>], in which several exposure estimates are calculated for the same ES. Instead of the three or more exposure values calculated from REACH models, which may differ by several orders of magnitude [<xref rid="R19" ref-type="bibr">19</xref>], TREXMO+ provides a single, refined estimate that corresponds the most to given exposure conditions.</p><p id="P56">TREXMO+ is not limited to the three REACH models (see <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref>) and its input parameters. Estimates of other exposure algorithms, such as physical-chemical models, could be considered in the future for inclusion in TREXMO+. Furthermore, regarding the limited exposure data that was available for this study, the classification layer identified only the exposure parameters of the three REACH models as significant for the exposure assessment. It is, however, important to mention that with more exposure data available, other variables, if found significant, could be used to extend <xref rid="FD1" ref-type="disp-formula">Eq. 1</xref>.</p><p id="P57">By the end of 2019, TREXMO+ will be available as a user-friendly web-application. The corresponding URL address will be shared with all current users of TREXMO v2 and added to the Google search engine. This publicly available freeware will include all data in the SUVA and NIOSH datasets.</p></sec><sec sec-type="supplementary-material" id="SM1"><title>Supplementary Material</title><supplementary-material content-type="local-data" id="SD1"><label>Supplement 1</label><media xlink:href="NIHMS1608142-supplement-Supplement_1.doc" orientation="portrait" id="d39e986" position="anchor"/></supplementary-material><supplementary-material content-type="local-data" id="SD2"><label>Supplement 2</label><media xlink:href="NIHMS1608142-supplement-Supplement_2.doc" orientation="portrait" id="d39e990" position="anchor"/></supplementary-material></sec></body><back><ack id="S20"><title>Acknowledgements</title><p id="P58">The authors are grateful to the Swiss Centre for Applied Human Toxicology (SCAHT) for funding this project and the Swiss Insurance Fund (SUVA) for providing the occupational exposure data.</p><p id="P59">Funding The project was funded by the Swiss Centre for Applied Human Toxicology (SCAHT).</p></ack><fn-group><fn id="FN1"><p id="P60" content-type="publisher-disclaimer">Disclaimer</p><p id="P61" content-type="publisher-disclaimer">The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the National Institute for Occupational Safety and Health, Centers for Disease Control and Prevention. Mention of any company or product does not constitute endorsement by NIOSH/CDC.</p></fn><fn id="FN3"><p id="P62">Code availability</p><p id="P63">To understand how the data was split by using the conditional RT algorithm, the applied R code is given in the <xref rid="SD1" ref-type="supplementary-material">Supplementary Information 1</xref>.</p></fn><fn fn-type="COI-statement" id="FN4"><p id="P64">Conflict of interest The authors declare that they have no conflict of interest.</p></fn><fn id="FN5"><p id="P65">Supplementary information The online version of this article (<ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1038/s41370-020-0203-9">https://doi.org/10.1038/s41370-020-0203-9</ext-link>) contains supplementary material, which is available to authorized users.</p></fn></fn-group><ref-list><title>References</title><ref id="R1"><label>1.</label><mixed-citation publication-type="book"><collab>ECHA</collab>. <source>Guidance on information requirements and chemical safety assessment&#x02014;Chapter R.14: occupational exposure assessment</source>. <publisher-loc>Helsinki (Finland)</publisher-loc>: <publisher-name>European Chemical Agency</publisher-name>; <year>2016</year>.</mixed-citation></ref><ref id="R2"><label>2.</label><mixed-citation publication-type="journal"><name><surname>Kromhout</surname><given-names>H</given-names></name>, <name><surname>Symanski</surname><given-names>E</given-names></name>, <name><surname>Rappaport</surname><given-names>SM</given-names></name>. <article-title>A comprehensive evaluation of within- and between-worker components of occupational exposure to chemical agents</article-title>. <source>Ann Occup Hyg</source>. <year>1993</year>;<volume>37</volume>:<fpage>253</fpage>&#x02013;<lpage>70</lpage>.<pub-id pub-id-type="pmid">8346874</pub-id></mixed-citation></ref><ref id="R3"><label>3.</label><mixed-citation publication-type="book"><collab>ECETOC</collab>. <source>Addendum to ECETOC targeted risk assessment report no</source>. <volume>93</volume>
<publisher-loc>Brussels</publisher-loc>: <publisher-name>European Centre for Ecotoxicology and Toxicology of Chemicals</publisher-name>; <year>2009</year>.</mixed-citation></ref><ref id="R4"><label>4.</label><mixed-citation publication-type="book"><collab>ECETOC</collab>. <source>ECETOC TRA version 3: background and rationale for the improvements&#x02014;technical report no</source>.<volume>114</volume>
<publisher-name>Brussels: European Centre for Ecotoxicology and Toxicology of Chemicals</publisher-name>; <year>2012</year>.</mixed-citation></ref><ref id="R5"><label>5.</label><mixed-citation publication-type="journal"><name><surname>Kromhout</surname><given-names>H</given-names></name>
<article-title>Hygiene without numbers</article-title>. <source>Ann Occup Hyg</source>. <year>2016</year>;<volume>60</volume>:<fpage>403</fpage>&#x02013;<lpage>4</lpage>.<pub-id pub-id-type="pmid">26759168</pub-id></mixed-citation></ref><ref id="R6"><label>6.</label><mixed-citation publication-type="book"><name><surname>Kindler</surname><given-names>P</given-names></name>, <name><surname>Winteler</surname><given-names>R</given-names></name>. <source>Anwendbarkeit von Expositionsmodellen f&#x000fc;r Chemikalien auf Schweizer Verh&#x000e4;ltnisse. Teilproject 1: &#x000dc;berpr&#x000fc;fung der Modelle &#x02018;EASE&#x02019; und &#x02018;EMKG-EXPO-TOOL</source>. <publisher-loc>Z&#x000fc;rich (Switzerland)</publisher-loc>: <publisher-name>Eidgen&#x000f6;ssisches Volkwirtschaftsdepartement EDV; Staatsekretariat f&#x000fc;r Wirtschaft SECO&#x02014;Arbeitsbedingungen; Chemikalien und Arbeit</publisher-name>; <year>2010</year>.</mixed-citation></ref><ref id="R7"><label>7.</label><mixed-citation publication-type="journal"><name><surname>Fransman</surname><given-names>W</given-names></name>, <name><surname>Van Tongeren</surname><given-names>M</given-names></name>, <name><surname>Cherrie</surname><given-names>JW</given-names></name>, <name><surname>Tischer</surname><given-names>M</given-names></name>, <name><surname>Schneider</surname><given-names>T</given-names></name>, <name><surname>Schinkel</surname><given-names>J</given-names></name>, <etal/>
<article-title>Advanced Reach Tool (ART): development of the mechanistic model</article-title>. <source>Ann Occup Hyg</source>. <year>2011</year>;<volume>55</volume>:<fpage>957</fpage>&#x02013;<lpage>79</lpage>.<pub-id pub-id-type="pmid">22003239</pub-id></mixed-citation></ref><ref id="R8"><label>8.</label><mixed-citation publication-type="journal"><name><surname>Schinkel</surname><given-names>J</given-names></name>, <name><surname>Warren</surname><given-names>N</given-names></name>, <name><surname>Fransman</surname><given-names>W</given-names></name>, <name><surname>van Tongeren</surname><given-names>M</given-names></name>, <name><surname>McDonnell</surname><given-names>P</given-names></name>, <name><surname>Voogd</surname><given-names>E</given-names></name>, <etal/>
<article-title>Advanced REACH Tool (ART): calibration of the mechanistic model</article-title>. <source>J Environ Monit</source>. <year>2011</year>;<volume>13</volume>:<fpage>1374</fpage>&#x02013;<lpage>82</lpage>.<pub-id pub-id-type="pmid">21403945</pub-id></mixed-citation></ref><ref id="R9"><label>9.</label><mixed-citation publication-type="journal"><name><surname>Marquart</surname><given-names>H</given-names></name>, <name><surname>Heussen</surname><given-names>H</given-names></name>, <name><surname>Le Feber</surname><given-names>M</given-names></name>, <name><surname>Noy</surname><given-names>D</given-names></name>, <name><surname>Tielemans</surname><given-names>E</given-names></name>, <name><surname>Schinkel</surname><given-names>J</given-names></name>, <etal/>
<article-title>&#x02018;Stoffenmanager&#x02019;, a web-based control banding tool using an exposure process model</article-title>. <source>Ann Occup Hyg</source>. <year>2008</year>;<volume>52</volume>:<fpage>429</fpage>&#x02013;<lpage>41</lpage>.<pub-id pub-id-type="pmid">18587140</pub-id></mixed-citation></ref><ref id="R10"><label>10.</label><mixed-citation publication-type="journal"><name><surname>Schinkel</surname><given-names>J</given-names></name>, <name><surname>Fransman</surname><given-names>W</given-names></name>, <name><surname>Heussen</surname><given-names>H</given-names></name>, <name><surname>Kromhout</surname><given-names>H</given-names></name>, <name><surname>Marquart</surname><given-names>H</given-names></name>, <name><surname>Tielemans</surname><given-names>E</given-names></name>. <article-title>Cross-validation and refinement of the Stoffenmanager as a first tier exposure assessment tool for REACH</article-title>. <source>Occup Environ Med</source>. <year>2010</year>;<volume>67</volume>:<fpage>125</fpage>&#x02013;<lpage>32</lpage>.<pub-id pub-id-type="pmid">19773280</pub-id></mixed-citation></ref><ref id="R11"><label>11.</label><mixed-citation publication-type="journal"><name><surname>Tischer</surname><given-names>M</given-names></name>, <name><surname>Lamb</surname><given-names>J</given-names></name>, <name><surname>Hesse</surname><given-names>S</given-names></name>, <name><surname>van Tongeren</surname><given-names>M</given-names></name>. <article-title>Evaluation of tier one exposure assessment models (ETEAM): project overview and methods</article-title>. <source>Ann Work Expo Health</source>. <year>2017</year>;<volume>61</volume>:<fpage>911</fpage>&#x02013;<lpage>20</lpage>.<pub-id pub-id-type="pmid">29028257</pub-id></mixed-citation></ref><ref id="R12"><label>12.</label><mixed-citation publication-type="journal"><name><surname>van Tongeren</surname><given-names>M</given-names></name>, <name><surname>Lamb</surname><given-names>J</given-names></name>, <name><surname>Cherrie</surname><given-names>JW</given-names></name>, <name><surname>MacCalman</surname><given-names>L</given-names></name>, <name><surname>Basinas</surname><given-names>I</given-names></name>, <name><surname>Hesse</surname><given-names>S</given-names></name>. <article-title>Validation of lower tier exposure tools used for REACH: comparison of tools estimates with available exposure measurements</article-title>. <source>Ann Work Expo Health</source>. <year>2017</year>;<volume>61</volume>:<fpage>921</fpage>&#x02013;<lpage>38</lpage>.<pub-id pub-id-type="pmid">29028246</pub-id></mixed-citation></ref><ref id="R13"><label>13.</label><mixed-citation publication-type="journal"><name><surname>Savic</surname><given-names>N</given-names></name>, <name><surname>Gasic</surname><given-names>B</given-names></name>, <name><surname>Schinkel</surname><given-names>J</given-names></name>, <name><surname>Vernez</surname><given-names>D</given-names></name>. <article-title>Comparing the Advanced REACH Tool&#x02019;s (ART) estimates with Switzerland&#x02019;s occupational exposure data</article-title>. <source>Ann Work Expo Health</source>. <year>2017</year>;<volume>61</volume>:<fpage>954</fpage>&#x02013;<lpage>64</lpage>.<pub-id pub-id-type="pmid">29028254</pub-id></mixed-citation></ref><ref id="R14"><label>14.</label><mixed-citation publication-type="journal"><name><surname>Lee</surname><given-names>EG</given-names></name>, <name><surname>Lamb</surname><given-names>J</given-names></name>, <name><surname>Savic</surname><given-names>N</given-names></name>, <name><surname>Basinas</surname><given-names>I</given-names></name>, <name><surname>Gasic</surname><given-names>B</given-names></name>, <name><surname>Jung</surname><given-names>C</given-names></name>, <etal/>
<article-title>Evaluation of exposure assessment tools under REACH: part I&#x02014; tier 1 tools</article-title>. <source>Ann Work Expo Health</source>. <year>2019</year>;<volume>63</volume>:<fpage>218</fpage>&#x02013;<lpage>29</lpage>.<pub-id pub-id-type="pmid">30534938</pub-id></mixed-citation></ref><ref id="R15"><label>15.</label><mixed-citation publication-type="journal"><name><surname>Lee</surname><given-names>EG</given-names></name>, <name><surname>Lamb</surname><given-names>J</given-names></name>, <name><surname>Savic</surname><given-names>N</given-names></name>, <name><surname>Basinas</surname><given-names>I</given-names></name>, <name><surname>Gasic</surname><given-names>B</given-names></name>, <name><surname>Jung</surname><given-names>C</given-names></name>, <etal/>
<article-title>Evaluation of exposure assessment tools under REACH: part II&#x02014; higher tier tools</article-title>. <source>Ann Work Expo Health</source>. <year>2019</year>;<volume>63</volume>:<fpage>230</fpage>&#x02013;<lpage>41</lpage>.<pub-id pub-id-type="pmid">30535049</pub-id></mixed-citation></ref><ref id="R16"><label>16.</label><mixed-citation publication-type="journal"><name><surname>Koppisch</surname><given-names>D</given-names></name>, <name><surname>Schinkel</surname><given-names>J</given-names></name>, <name><surname>Gabriel</surname><given-names>S</given-names></name>, <name><surname>Fransman</surname><given-names>W</given-names></name>, <name><surname>Tielemans</surname><given-names>E</given-names></name>. <article-title>Use of the MEGA exposure database for the validation of the Stoffenmanager model</article-title>. <source>Ann Occup Hyg</source>. <year>2011</year>;<volume>56</volume>:<fpage>426</fpage>&#x02013;<lpage>39</lpage>.<pub-id pub-id-type="pmid">22064766</pub-id></mixed-citation></ref><ref id="R17"><label>17.</label><mixed-citation publication-type="journal"><name><surname>Kupczewska-Dobecka</surname><given-names>M</given-names></name>, <name><surname>Czerczak</surname><given-names>S</given-names></name>, <name><surname>Jakubowski</surname><given-names>M</given-names></name>. <article-title>Evaluation of the TRA ECETOC model for inhalation workplace exposure to different organic solvents for selected process categories</article-title>. <source>Int J Occup Med Environ Health</source>. <year>2011</year>;<volume>24</volume>:<fpage>208</fpage>&#x02013;<lpage>17</lpage>.<pub-id pub-id-type="pmid">21537889</pub-id></mixed-citation></ref><ref id="R18"><label>18.</label><mixed-citation publication-type="journal"><name><surname>Spinazz&#x000e8;</surname><given-names>A</given-names></name>, <name><surname>Lunghini</surname><given-names>F</given-names></name>, <name><surname>Campagnolo</surname><given-names>D</given-names></name>, <name><surname>Rovelli</surname><given-names>S</given-names></name>, <name><surname>Locatelli</surname><given-names>M</given-names></name>, <name><surname>Cattaneo</surname><given-names>A</given-names></name>, <etal/>
<article-title>Accuracy evaluation of three modelling tools for occupational exposure assessment</article-title>. <source>Ann Work Expo Health</source>. <year>2017</year>;<volume>61</volume>:<fpage>284</fpage>&#x02013;<lpage>98</lpage>.<pub-id pub-id-type="pmid">28355416</pub-id></mixed-citation></ref><ref id="R19"><label>19.</label><mixed-citation publication-type="journal"><name><surname>Savic</surname><given-names>N</given-names></name>, <name><surname>Gasic</surname><given-names>B</given-names></name>, <name><surname>Vernez</surname><given-names>D</given-names></name>. <article-title>ART, Stoffenmanager, and TRA: a systematic comparison of exposure estimates using the TREXMO translation system</article-title>. <source>Ann Work Expo Health</source>. <year>2018</year>;<volume>62</volume>:<fpage>72</fpage>&#x02013;<lpage>87</lpage>.</mixed-citation></ref><ref id="R20"><label>20.</label><mixed-citation publication-type="journal"><name><surname>Savic</surname><given-names>N</given-names></name>, <name><surname>Racordon</surname><given-names>D</given-names></name>, <name><surname>Buchs</surname><given-names>D</given-names></name>, <name><surname>Gasic</surname><given-names>B</given-names></name>, <name><surname>Vernez</surname><given-names>D</given-names></name>. <article-title>TREXMO: a translation tool to support the use of regulatory occupational exposure models</article-title>. <source>Ann Occup Hyg</source>. <year>2016</year>;<volume>60</volume>:<fpage>991</fpage>&#x02013;<lpage>1008</lpage>.<pub-id pub-id-type="pmid">27358294</pub-id></mixed-citation></ref><ref id="R21"><label>21.</label><mixed-citation publication-type="journal"><name><surname>Strasser</surname><given-names>H</given-names></name>, <name><surname>Weber</surname><given-names>C</given-names></name>. <article-title>On the asymptotic theory of permutation statistics</article-title>. <source>Mathematical Methods of Statistics</source>. <year>1999</year>;<volume>8</volume>;<fpage>220</fpage>&#x02013;<lpage>50</lpage>.</mixed-citation></ref><ref id="R22"><label>22.</label><mixed-citation publication-type="journal"><name><surname>Loh</surname><given-names>W-J</given-names></name>. <article-title>Classification and regression trees</article-title>. <source>Wiley Interdiscip Rev: Data Min Knowl Discov</source>. <year>2011</year>;<volume>1</volume>:<fpage>14</fpage>&#x02013;<lpage>23</lpage>.</mixed-citation></ref><ref id="R23"><label>23.</label><mixed-citation publication-type="journal"><name><surname>Garcia</surname><given-names>CB</given-names></name>, <name><surname>Garcia</surname><given-names>J</given-names></name>, <name><surname>Lopez</surname><given-names>MM</given-names></name>, <name><surname>Salmeron</surname><given-names>R</given-names></name>. <article-title>Collinearity: revisiting the variance inflation factor in ridge regression</article-title>. <source>J Appl Stat</source>. <year>2015</year>;<volume>42</volume>:<fpage>648</fpage>&#x02013;<lpage>61</lpage>.</mixed-citation></ref><ref id="R24"><label>24.</label><mixed-citation publication-type="book"><name><surname>James</surname><given-names>G</given-names></name>, <name><surname>Witten</surname><given-names>D</given-names></name>, <name><surname>Hastie</surname><given-names>T</given-names></name>, <name><surname>Tibshirani</surname><given-names>R</given-names></name>. <source>An introduction to statistical learning: with application in R</source>. <publisher-loc>Springer New York</publisher-loc>: <publisher-name>Springer Publishing Company</publisher-name>; <year>2014</year>
<comment>ISBN 1461471370.</comment></mixed-citation></ref><ref id="R25"><label>25.</label><mixed-citation publication-type="journal"><name><surname>Cleveland</surname><given-names>WS</given-names></name>. <article-title>Robust locally weighted regression and smoothing scatterplots</article-title>. <source>J Am Stat Assoc</source>. <year>1979</year>;<volume>74</volume>:<fpage>829</fpage>&#x02013;<lpage>36</lpage>.</mixed-citation></ref><ref id="R26"><label>26.</label><mixed-citation publication-type="book"><collab>R Core Team</collab>. <source>R: a language and environment for statistical computing</source>. <publisher-loc>Vienna, Austria</publisher-loc>: <publisher-name>R Foundation for Statistical Computing</publisher-name>; <year>2018</year>.</mixed-citation></ref><ref id="R27"><label>27.</label><mixed-citation publication-type="book"><name><surname>Wickham</surname><given-names>H</given-names></name>
<source>Elegant graphics for data analysis</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Springer-Verlag</publisher-name>; <year>2016</year> p. <fpage>ggplot2</fpage>.</mixed-citation></ref><ref id="R28"><label>28.</label><mixed-citation publication-type="journal"><name><surname>Riedmann</surname><given-names>R</given-names></name>, <name><surname>Gasic</surname><given-names>B</given-names></name>, <name><surname>Vernez</surname><given-names>D</given-names></name>. <article-title>Sensitivity analysis, dominant factors, and robustness of the ECETOC TRA v3, Stoffenmanager 4.5, and ART 1.5 occupational exposure models</article-title>. <source>Risk Anal</source>. <year>2015</year>;<volume>35</volume>:<fpage>211</fpage>&#x02013;<lpage>25</lpage>.<pub-id pub-id-type="pmid">25616198</pub-id></mixed-citation></ref></ref-list></back><floats-group><fig id="F1" orientation="portrait" position="float"><label>Fig. 1</label><caption><title>Schematic illustration of TREXMO+ algorithm and a regression tree example.</title><p id="P66"><bold>a</bold> - TREXMO+ algorithm and <bold>b</bold> - regression tree.</p></caption><graphic xlink:href="nihms-1608142-f0001"/></fig><fig id="F2" orientation="portrait" position="float"><label>Fig. 2</label><caption><title>Regression trees established for the evaluated exposure forms.</title><p id="P67"><bold>a</bold> Volatile liquids and <bold>b</bold> powders.</p></caption><graphic xlink:href="nihms-1608142-f0002"/></fig><fig id="F3" orientation="portrait" position="float"><label>Fig. 3</label><caption><title>Residuals obtained for the training and testing dataset for the three exposure types.</title><p id="P68">The LOWESS method was applied to plot data points as smoothed lines for the four exposure models.</p></caption><graphic xlink:href="nihms-1608142-f0003"/></fig><fig id="F4" orientation="portrait" position="float"><label>Fig. 4</label><caption><title>Measured versus modeled exposure values for three different exposure types.</title><p id="P69">Red dotted lines (for ART and the inner dotted lines for TREXMO+) illustrate the 90% confidence intervals, whereas the blue dotted lines (only for TREXMO+; the outer lines) illustrate the prediction interval.</p></caption><graphic xlink:href="nihms-1608142-f0004"/></fig><table-wrap id="T1" position="float" orientation="landscape"><label>Table 1</label><caption><p id="P70">ART exposure determinants used for testing independence with the measurements in RTs.</p></caption><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/></colgroup><thead><tr><th align="left" valign="top" rowspan="1" colspan="1">Modifying factor</th><th align="left" valign="top" rowspan="1" colspan="1">RT variables <italic>(example)</italic></th><th align="left" valign="top" rowspan="1" colspan="1">Variable type</th></tr></thead><tbody><tr><td rowspan="3" align="left" valign="top" colspan="1">Substance emission potential</td><td align="left" valign="top" rowspan="1" colspan="1">Vapor pressure<sup><xref rid="TFN2" ref-type="table-fn">a</xref></sup></td><td align="left" valign="top" rowspan="1" colspan="1">Continuous</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Product concentration<sup><xref rid="TFN2" ref-type="table-fn">a</xref></sup></td><td align="left" valign="top" rowspan="1" colspan="1">Continuous</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Dustiness<sup><xref rid="TFN3" ref-type="table-fn">b</xref></sup></td><td align="left" valign="top" rowspan="1" colspan="1">Categorical</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Activity emission potential</td><td align="left" valign="top" rowspan="1" colspan="1">Activity (sub)classes (e.g., <italic>surface spraying of liquid products)</italic></td><td align="left" valign="top" rowspan="1" colspan="1">Categorical</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Local control</td><td align="left" valign="top" rowspan="1" colspan="1">Category of primary local control (e.g., <italic>enclosing LEV)</italic></td><td align="left" valign="top" rowspan="1" colspan="1">Categorical</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Segregation</td><td align="left" valign="top" rowspan="1" colspan="1">Segregation (e.g., <italic>partial segregation)</italic></td><td align="left" valign="top" rowspan="1" colspan="1">Categorical</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Dilution</td><td align="left" valign="top" rowspan="1" colspan="1">Room volume Air-exchange per hour</td><td align="left" valign="top" rowspan="1" colspan="1">Categorical</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Separation</td><td align="left" valign="top" rowspan="1" colspan="1">Separation (e.g., <italic>partial separation with ventilation)</italic></td><td align="left" valign="top" rowspan="1" colspan="1">Categorical</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Surface contamination</td><td align="left" valign="top" rowspan="1" colspan="1">Surface contamination (e.g., <italic>good housekeeping)</italic></td><td align="left" valign="top" rowspan="1" colspan="1">Categorical</td></tr></tbody></table><table-wrap-foot><fn id="TFN1"><p id="P71"><italic>ART</italic> Advanced REACH Tool, <italic>LEV</italic> local exhaust ventilation, <italic>RT</italic> regression tree.</p></fn><fn id="TFN2"><label>a</label><p id="P72">For volatile liquids, fugacity was calculated as log(vapor pressure &#x000d7; concentration).</p></fn><fn id="TFN3"><label>b</label><p id="P73">For powders and solids, log-concentration was calculated.</p></fn></table-wrap-foot></table-wrap><table-wrap id="T2" position="float" orientation="landscape"><label>Table 2</label><caption><p id="P74">Summary of the number of exposure situations (ESs) and exposure measurements (EMs) provided by the Swiss Insurance Fund (SUVA, Switzerland) and the National Institute for Occupational Safety and Health (NIOSH, US).</p></caption><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/></colgroup><thead><tr><th rowspan="2" align="left" valign="top" colspan="1">Exposure type</th><th colspan="2" align="left" valign="top" style="border-bottom: solid 1px" rowspan="1">SUVA</th><th colspan="2" align="left" valign="top" style="border-bottom: solid 1px" rowspan="1">NIOSH</th></tr><tr><th align="left" valign="top" rowspan="1" colspan="1">Total ESs (EM<sup><xref rid="TFN6" ref-type="table-fn">b</xref></sup>)</th><th align="left" valign="top" rowspan="1" colspan="1">Processed ESs<sup><xref rid="TFN5" ref-type="table-fn">a</xref></sup> (EM)</th><th align="left" valign="top" rowspan="1" colspan="1">Total ESs (EM<sup><xref rid="TFN6" ref-type="table-fn">b</xref></sup>)</th><th align="left" valign="top" rowspan="1" colspan="1">Processed ESs (EM)</th></tr></thead><tbody><tr><td align="left" valign="top" rowspan="1" colspan="1">Volatile liquids (VP &#x0003e; 10 Pa)</td><td align="left" valign="top" rowspan="1" colspan="1">146 (346)</td><td align="left" valign="top" rowspan="1" colspan="1">135 (317)</td><td align="left" valign="top" rowspan="1" colspan="1">50 (425)</td><td align="left" valign="top" rowspan="1" colspan="1">50 (425)</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Non-volatile liquids (VP &#x02264; 10 Pa)</td><td align="left" valign="top" rowspan="1" colspan="1">5 (5)</td><td align="left" valign="top" rowspan="1" colspan="1">0</td><td align="left" valign="top" rowspan="1" colspan="1">4 (9)</td><td align="left" valign="top" rowspan="1" colspan="1">0</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Powders (non-abrasive dust)</td><td align="left" valign="top" rowspan="1" colspan="1">59 (113)</td><td align="left" valign="top" rowspan="1" colspan="1">59 (113)</td><td align="left" valign="top" rowspan="1" colspan="1">10 (40)</td><td align="left" valign="top" rowspan="1" colspan="1">10 (40)</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Solids (abrasive dust)</td><td align="left" valign="top" rowspan="1" colspan="1">52 (120)</td><td align="left" valign="top" rowspan="1" colspan="1">24 (57)</td><td align="left" valign="top" rowspan="1" colspan="1">0</td><td align="left" valign="top" rowspan="1" colspan="1">0</td></tr></tbody></table><table-wrap-foot><fn id="TFN4"><p id="P75"><italic>VP</italic> vapor pressure.</p></fn><fn id="TFN5"><label>a</label><p id="P76">Processed data: only ESs that were applicable in all the three REACH models used in the study; in case of non-volatiles, the data were not processed due to the small data size.</p></fn><fn id="TFN6"><label>b</label><p id="P77">SUVA included 1&#x02013;3 exposure measurements per ES, while NIOSH 5&#x02013;12.</p></fn></table-wrap-foot></table-wrap><table-wrap id="T3" position="float" orientation="landscape"><label>Table 3</label><caption><p id="P78">The TREXMO+ coefficients evaluated for the three predictor models and the corresponding R-squared values found for different exposure subsets (i.e., nodes).</p></caption><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/></colgroup><thead><tr><th align="left" valign="top" rowspan="1" colspan="1">Exposure type (ES<sup><xref rid="TFN8" ref-type="table-fn">a</xref></sup>/EM<sup><xref rid="TFN9" ref-type="table-fn">b</xref></sup>)</th><th align="left" valign="top" rowspan="1" colspan="1">Node</th><th align="left" valign="top" rowspan="1" colspan="1">Number of ES</th><th align="left" valign="top" rowspan="1" colspan="1">ART</th><th align="left" valign="top" rowspan="1" colspan="1">SM</th><th align="left" valign="top" rowspan="1" colspan="1">TRAv3</th><th align="left" valign="top" rowspan="1" colspan="1">Error</th><th align="left" valign="top" rowspan="1" colspan="1"><italic>R</italic><sup>2</sup></th></tr></thead><tbody><tr><td rowspan="4" align="left" valign="top" colspan="1">Liquids (185/742)</td><td align="left" valign="top" rowspan="1" colspan="1">3</td><td align="left" valign="top" rowspan="1" colspan="1">40</td><td align="left" valign="top" rowspan="1" colspan="1">0.32</td><td align="left" valign="top" rowspan="1" colspan="1"><sup><xref rid="TFN10" ref-type="table-fn">c</xref></sup></td><td align="left" valign="top" rowspan="1" colspan="1">0.16</td><td align="left" valign="top" rowspan="1" colspan="1">1.31</td><td align="left" valign="top" rowspan="1" colspan="1">0.13</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">4</td><td align="left" valign="top" rowspan="1" colspan="1">11</td><td align="left" valign="top" rowspan="1" colspan="1">0.03</td><td align="left" valign="top" rowspan="1" colspan="1">2.44</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.99</td><td align="left" valign="top" rowspan="1" colspan="1">1.14</td><td align="left" valign="top" rowspan="1" colspan="1">0.69</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">6</td><td align="left" valign="top" rowspan="1" colspan="1">67</td><td align="left" valign="top" rowspan="1" colspan="1">0.36</td><td align="left" valign="top" rowspan="1" colspan="1"><sup><xref rid="TFN10" ref-type="table-fn">c</xref></sup></td><td align="left" valign="top" rowspan="1" colspan="1">0.37</td><td align="left" valign="top" rowspan="1" colspan="1">0.74</td><td align="left" valign="top" rowspan="1" colspan="1">0.82</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">7</td><td align="left" valign="top" rowspan="1" colspan="1">30</td><td align="left" valign="top" rowspan="1" colspan="1"><sup><xref rid="TFN10" ref-type="table-fn">c</xref></sup></td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.51</td><td align="left" valign="top" rowspan="1" colspan="1">0.49</td><td align="left" valign="top" rowspan="1" colspan="1">0.84</td><td align="left" valign="top" rowspan="1" colspan="1">0.30</td></tr><tr><td rowspan="3" align="left" valign="top" colspan="1">Powders (69/153)</td><td align="left" valign="top" rowspan="1" colspan="1">3</td><td align="left" valign="top" rowspan="1" colspan="1">7</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.19</td><td align="left" valign="top" rowspan="1" colspan="1">1.62</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.69</td><td align="left" valign="top" rowspan="1" colspan="1">1.00</td><td align="left" valign="top" rowspan="1" colspan="1">0.54</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">4</td><td align="left" valign="top" rowspan="1" colspan="1">38</td><td align="left" valign="top" rowspan="1" colspan="1">0.18</td><td align="left" valign="top" rowspan="1" colspan="1">0.15</td><td align="left" valign="top" rowspan="1" colspan="1">0.01</td><td align="left" valign="top" rowspan="1" colspan="1">0.68</td><td align="left" valign="top" rowspan="1" colspan="1">0.15</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">5</td><td align="left" valign="top" rowspan="1" colspan="1">10</td><td align="left" valign="top" rowspan="1" colspan="1"><sup><xref rid="TFN10" ref-type="table-fn">c</xref></sup></td><td align="left" valign="top" rowspan="1" colspan="1">0.98</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.88</td><td align="left" valign="top" rowspan="1" colspan="1">0.71</td><td align="left" valign="top" rowspan="1" colspan="1">0.82</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Solids (24/57)</td><td align="left" valign="top" rowspan="1" colspan="1">/</td><td align="left" valign="top" rowspan="1" colspan="1">24</td><td align="left" valign="top" rowspan="1" colspan="1">0.13</td><td align="left" valign="top" rowspan="1" colspan="1"><sup><xref rid="TFN10" ref-type="table-fn">c</xref></sup></td><td align="left" valign="top" rowspan="1" colspan="1">0.39</td><td align="left" valign="top" rowspan="1" colspan="1">0.60</td><td align="left" valign="top" rowspan="1" colspan="1">0.35</td></tr></tbody></table><table-wrap-foot><fn id="TFN7"><p id="P79">The node numbers correspond to the leaf nodes in <xref rid="F2" ref-type="fig">Fig. 2</xref>.</p></fn><fn id="TFN8"><label>a</label><p id="P80">Number of exposure situations.</p></fn><fn id="TFN9"><label>b</label><p id="P81">Number of exposure measurements.</p></fn><fn id="TFN10"><label>c</label><p id="P82">Model not used as predictor due to a high multicollinearity.</p></fn></table-wrap-foot></table-wrap><table-wrap id="T4" position="float" orientation="portrait"><label>Table 4</label><caption><p id="P83">Relative bias and accuracy of the three REACH models, the machine-learning algorithm only and TREXMO+.</p></caption><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/></colgroup><thead><tr><th rowspan="2" align="left" valign="top" colspan="1">Model</th><th colspan="3" align="left" valign="top" style="border-bottom: solid 1px" rowspan="1">Relative bias (%)</th><th colspan="3" align="left" valign="top" style="border-bottom: solid 1px" rowspan="1">Accuracy</th></tr><tr><th align="left" valign="top" rowspan="1" colspan="1">Liquids</th><th align="left" valign="top" rowspan="1" colspan="1">Powders</th><th align="left" valign="top" rowspan="1" colspan="1">Solids</th><th align="left" valign="top" rowspan="1" colspan="1">Liquids</th><th align="left" valign="top" rowspan="1" colspan="1">Powders</th><th align="left" valign="top" rowspan="1" colspan="1">Solids</th></tr></thead><tbody><tr><td align="left" valign="top" rowspan="1" colspan="1">ART</td><td align="left" valign="top" rowspan="1" colspan="1">86</td><td align="left" valign="top" rowspan="1" colspan="1">1004</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;57</td><td align="left" valign="top" rowspan="1" colspan="1">1.00</td><td align="left" valign="top" rowspan="1" colspan="1">0.85</td><td align="left" valign="top" rowspan="1" colspan="1">0.61</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">SM</td><td align="left" valign="top" rowspan="1" colspan="1">9</td><td align="left" valign="top" rowspan="1" colspan="1">325</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;34</td><td align="left" valign="top" rowspan="1" colspan="1">0.66</td><td align="left" valign="top" rowspan="1" colspan="1">0.63</td><td align="left" valign="top" rowspan="1" colspan="1">0.53</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">TRAv3</td><td align="left" valign="top" rowspan="1" colspan="1">410</td><td align="left" valign="top" rowspan="1" colspan="1">818</td><td align="left" valign="top" rowspan="1" colspan="1">171</td><td align="left" valign="top" rowspan="1" colspan="1">1.16</td><td align="left" valign="top" rowspan="1" colspan="1">1.04</td><td align="left" valign="top" rowspan="1" colspan="1">0.45</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">ML</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;49</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;90</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;66</td><td align="left" valign="top" rowspan="1" colspan="1">0.61</td><td align="left" valign="top" rowspan="1" colspan="1">0.58</td><td align="left" valign="top" rowspan="1" colspan="1">0.34</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">TREXMO+</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;41</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;24</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;30</td><td align="left" valign="top" rowspan="1" colspan="1">0.46</td><td align="left" valign="top" rowspan="1" colspan="1">0.48</td><td align="left" valign="top" rowspan="1" colspan="1">0.31</td></tr></tbody></table></table-wrap><table-wrap id="T5" position="float" orientation="portrait"><label>Table 5</label><caption><p id="P84">Regression coefficients obtained for the models.</p></caption><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/></colgroup><thead><tr><th align="left" valign="top" rowspan="1" colspan="1">Exposure type (ES<sup><xref rid="TFN12" ref-type="table-fn">b</xref></sup>/ EM<sup><xref rid="TFN13" ref-type="table-fn">c</xref></sup>)</th><th align="left" valign="top" rowspan="1" colspan="1">Model</th><th align="left" valign="top" rowspan="1" colspan="1">Intercept (a)</th><th align="left" valign="top" rowspan="1" colspan="1">Slope (b)</th><th align="left" valign="top" rowspan="1" colspan="1"><italic>R</italic><sup>2</sup></th><th align="left" valign="top" rowspan="1" colspan="1"><italic>p</italic> value</th></tr></thead><tbody><tr><td rowspan="4" align="left" valign="top" colspan="1">Liquids (185/742)</td><td align="left" valign="top" rowspan="1" colspan="1">ART</td><td align="left" valign="top" rowspan="1" colspan="1">0.47</td><td align="left" valign="top" rowspan="1" colspan="1">0.38</td><td align="left" valign="top" rowspan="1" colspan="1">0.20</td><td align="left" valign="top" rowspan="1" colspan="1">4.6e-9<sup><xref rid="TFN11" ref-type="table-fn">a</xref></sup></td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">SM</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.13</td><td align="left" valign="top" rowspan="1" colspan="1">1.09</td><td align="left" valign="top" rowspan="1" colspan="1">0.34</td><td align="left" valign="top" rowspan="1" colspan="1">1.2e-15<sup><xref rid="TFN11" ref-type="table-fn">a</xref></sup></td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">TRAv3</td><td align="left" valign="top" rowspan="1" colspan="1">0.03</td><td align="left" valign="top" rowspan="1" colspan="1">0.55</td><td align="left" valign="top" rowspan="1" colspan="1">0.15</td><td align="left" valign="top" rowspan="1" colspan="1">2e-6<sup><xref rid="TFN11" ref-type="table-fn">a</xref></sup></td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">TREXMO+</td><td align="left" valign="top" rowspan="1" colspan="1">0.18</td><td align="left" valign="top" rowspan="1" colspan="1">0.92</td><td align="left" valign="top" rowspan="1" colspan="1">0.41</td><td align="left" valign="top" rowspan="1" colspan="1">&#x0003c;2e-16<sup><xref rid="TFN11" ref-type="table-fn">a</xref></sup></td></tr><tr><td rowspan="4" align="left" valign="top" colspan="1">Powders (69/153)</td><td align="left" valign="top" rowspan="1" colspan="1">ART</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.41</td><td align="left" valign="top" rowspan="1" colspan="1">0.37</td><td align="left" valign="top" rowspan="1" colspan="1">0.06</td><td align="left" valign="top" rowspan="1" colspan="1">0.16</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">SM</td><td align="left" valign="top" rowspan="1" colspan="1">0.06</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.18</td><td align="left" valign="top" rowspan="1" colspan="1">0.01</td><td align="left" valign="top" rowspan="1" colspan="1">0.63</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">TRAv3</td><td align="left" valign="top" rowspan="1" colspan="1">0.86</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.97</td><td align="left" valign="top" rowspan="1" colspan="1">0.49</td><td align="left" valign="top" rowspan="1" colspan="1">0.47</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">TREXMO+</td><td align="left" valign="top" rowspan="1" colspan="1">0.12</td><td align="left" valign="top" rowspan="1" colspan="1">0.87</td><td align="left" valign="top" rowspan="1" colspan="1">0.65</td><td align="left" valign="top" rowspan="1" colspan="1">&#x0003c;2e-16<sup><xref rid="TFN11" ref-type="table-fn">a</xref></sup></td></tr><tr><td rowspan="4" align="left" valign="top" colspan="1">Solids (24/57)</td><td align="left" valign="top" rowspan="1" colspan="1">ART</td><td align="left" valign="top" rowspan="1" colspan="1">0.50</td><td align="left" valign="top" rowspan="1" colspan="1">0.16</td><td align="left" valign="top" rowspan="1" colspan="1">0.05</td><td align="left" valign="top" rowspan="1" colspan="1">0.55</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">SM</td><td align="left" valign="top" rowspan="1" colspan="1">0.56</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.08</td><td align="left" valign="top" rowspan="1" colspan="1">0.01</td><td align="left" valign="top" rowspan="1" colspan="1">0.86</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">TRAv3</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.43</td><td align="left" valign="top" rowspan="1" colspan="1">1.00</td><td align="left" valign="top" rowspan="1" colspan="1">0.94</td><td align="left" valign="top" rowspan="1" colspan="1">&#x0003c;1e-5<sup><xref rid="TFN11" ref-type="table-fn">a</xref></sup></td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">TREXMO+</td><td align="left" valign="top" rowspan="1" colspan="1">&#x02212;0.7</td><td align="left" valign="top" rowspan="1" colspan="1">3.01</td><td align="left" valign="top" rowspan="1" colspan="1">0.68</td><td align="left" valign="top" rowspan="1" colspan="1">0.01<sup><xref rid="TFN11" ref-type="table-fn">a</xref></sup></td></tr></tbody></table><table-wrap-foot><fn id="TFN11"><label>a</label><p id="P85">Statistically significant at the 95% confidence level.</p></fn><fn id="TFN12"><label>b</label><p id="P86">Number of exposure situations.</p></fn><fn id="TFN13"><label>c</label><p id="P87">Number of exposure measurements.</p></fn></table-wrap-foot></table-wrap></floats-group></article>