Measurement uncertainty and NIOSH method accuracy range

Details

• Personal Author:
  Bartley, David L. ; Schlecht, Paul C. ; Shulman, Stanley A.
• Description:
  Recently, the ISO Guide to the Expression of Uncertainty in Measurement (GUM) has come close to being universally adopted as the standardized way to characterize and document measurement uncertainty [ISO 2002, 2005, 2009, 2010; Ellison and Williams 2012]. Since the mid-1970s, accuracy criteria have been an integral part of the evaluations of the sampling and analytical methods used by the National Institute for Occupational Safety and Health (NIOSH), the Occupational Safety and Health Administration (OSHA), the Mine Safety and Health Administration (MSHA), and others. NIOSH has previously published extensive discussions addressing the issue of accuracy as a factor in the development, evaluation, and characterization of analytical methodology. Both traditional method accuracy and new measurement uncertainty concepts are intended to communicate measurement limitations to laboratory clients. Naturally, laboratories are interested in how NIOSH accuracy requirements [Busch 1977; NIOSH 1995] relate to measurement uncertainty. This chapter provides guidance for achieving consistency in determining measurement uncertainty by those laboratories using NIOSH methods. Minor modifications to NIOSH accuracy measures, and an expansion of ISO GUM to cover situations unique to workplace atmospheric measurement can improve consistency and utility. See Bartley [2004] for additional information. GUM proposes pooling estimated variance components from diverse error sources. The square root of the pooled variance estimate is termed the combined uncertainty uc. Multiplication of uc by a coverage factor k (generally in the range of 2 to 3) results in an expanded uncertainty U. The purpose of the expanded uncertainty is for each measurement to provide an interval bracketing the measurand (the true value of what is to be measured) to account for errors in both the measurement and the determination of the uncertainty components themselves. ISO GUM is somewhat unclear about the coverage factor k. Furthermore, the coverage factor can be interpreted in several ways. Most straightforward is the limited case where the uncertainty components can be re-evaluated each time the method is used (resulting in k proportional to a Student-t quantile). In this case, the covering intervals bracket the measurand for (for example) 95% of the measurements. Alternatively, the coverage factors based on the Student-t quantile specify intervals containing measurand values at levels of evaluation confidence in the mean (i.e., averaging over many method evaluations). In other words, for roughly 50% of method evaluations, intervals used at each measurement contain the measurand value greater than (for example) 95% of the time. The concept is consistent with the statistical theory of tolerance or prediction intervals. This approach is important to industrial hygiene since workplace air concentrations vary spatially and over time to such a degree that a method cannot be evaluated by simply taking replicate measurements [Vaughan et al. 1990]. However, industrial hygiene measurement methods have traditionally required confidence levels greater than 50% in the method evaluation. Generally, 95% confidence in a method validation is required. The different types of confidence levels are reflected simply in the numerical value and interpretation of the coverage factor. Of equal importance in the industrial hygiene field are details needed to handle systematic error (bias) relative to reference concentration measurements found during method evaluation. For example, the sampling rate of a given diffusive sampler for gases or vapors is generally measured once by the diffusive sampler manufacturer prior to use by multiple clients. As the samplers are not re-calibrated for each use, residual bias exists in the measurements due to uncertainty in sampling rates used [ASTM 2013a]. (NIOSH methods typically do not cite performance for passive samplers because agreement among diffusive monitor manufacturers on test protocols has not yet been achieved, and a system of third party evaluation of diffusive monitor manufacturers sampling rates is not available.) Similarly, the calculation of desorption efficiencies may be performed only once or infrequently and can, therefore, introduce residual bias in measurements that use sorbent-captured samples, e.g., charcoal tubes. In aerosol sampling, detailed knowledge of the particle size-dependent bias of a sampler relative to a sampling convention, such as adopted by ISO/CEN/ACGIH/ASTM [ISO 1995; CEN 1993; ACGIH 2015; ASTM 2013b] for defining respirable dust, is often necessary to judge the usefulness of a given sampler. Each type of aerosol sampler is characterized by specific particle collection characteristics, and some analytical methods (e.g. silica) may also exhibit particle size effects. Typically the issue of aerosol sampler bias is avoided or minimized in the industrial hygiene field by narrowing use to a specific aerosol sampler. For example, common industrial hygiene practice establishes a single sampler type, such as the 1.7 L/min 10-mm nylon Dorr-Oliver cyclone, for respirable dust sampling in a particular application. Sensitivity to other environmental factors, referred to in ISO GUM as influence variables, must be acknowledged. Suppose a sampler is sensitive to temperature changes that are impractical to measure in the field; i.e., sampler estimates are not temperature corrected. Then, suppose during method evaluation in the laboratory, measurement of this sensitivity is combined with knowledge of the expected temperature variation for a given field application. Putting together both would determine the uncertainty associated with the effect. Examples of the important effects of influence variables - such as wind velocity, temperature, pressure, and fluctuating workplace concentrations - on diffusive monitor uptake rates are common.This approach is important to industrial hygiene since workplace air concentrations vary spatially and over time to such a degree that a method cannot be evaluated by simply taking replicate measurements [Vaughan et al. 1990]. However, industrial hygiene measurement methods have traditionally required confidence levels greater than 50% in the method evaluation. Generally, 95% confidence in a method validation is required. The different types of confidence levels are reflected simply in the numerical value and interpretation of the coverage factor. Of equal importance in the industrial hygiene field are details needed to handle systematic error (bias) relative to reference concentration measurements found during method evaluation. For example, the sampling rate of a given diffusive sampler for gases or vapors is generally measured once by the diffusive sampler manufacturer prior to use by multiple clients. As the samplers are not re-calibrated for each use, residual bias exists in the measurements due to uncertainty in sampling rates used [ASTM 2013a]. (NIOSH methods typically do not cite performance for passive samplers because agreement among diffusive monitor manufacturers on test protocols has not yet been achieved, and a system of third party evaluation of diffusive monitor manufacturers sampling rates is not available.) Similarly, the calculation of desorption efficiencies may be performed only once or infrequently and can, therefore, introduce residual bias in measurements that use sorbent-captured samples, e.g., charcoal tubes. In aerosol sampling, detailed knowledge of the particle size-dependent bias of a sampler relative to a sampling convention, such as adopted by ISO/CEN/ACGIH/ASTM [ISO 1995; CEN 1993; ACGIH 2015; ASTM 2013b] for defining respirable dust, is often necessary to judge the usefulness of a given sampler. Each type of aerosol sampler is characterized by specific particle collection characteristics, and some analytical methods (e.g. silica) may also exhibit particle size effects. Typically the issue of aerosol sampler bias is avoided or minimized in the industrial hygiene field by narrowing use to a specific aerosol sampler. For example, common industrial hygiene practice establishes a single sampler type, such as the 1.7 L/min 10-mm nylon Dorr-Oliver cyclone, for respirable dust sampling in a particular application. Sensitivity to other environmental factors, referred to in ISO GUM as influence variables, must be acknowledged. Suppose a sampler is sensitive to temperature changes that are impractical to measure in the field; i.e., sampler estimates are not temperature corrected. Then, suppose during method evaluation in the laboratory, measurement of this sensitivity is combined with knowledge of the expected temperature variation for a given field application. Putting together both would determine the uncertainty associated with the effect. Examples of the important effects of influence variables - such as wind velocity, temperature, pressure, and fluctuating workplace concentrations - on diffusive monitor uptake rates are common.vThis approach is important to industrial hygiene since workplace air concentrations vary spatially and over time to such a degree that a method cannot be evaluated by simply taking replicate measurements [Vaughan et al. 1990]. However, industrial hygiene measurement methods have traditionally required confidence levels greater than 50% in the method evaluation. Generally, 95% confidence in a method validation is required. The different types of confidence levels are reflected simply in the numerical value and interpretation of the coverage factor. Of equal importance in the industrial hygiene field are details needed to handle systematic error (bias) relative to reference concentration measurements found during method evaluation. For example, the sampling rate of a given diffusive sampler for gases or vapors is generally measured once by the diffusive sampler manufacturer prior to use by multiple clients. As the samplers are not re-calibrated for each use, residual bias exists in the measurements due to uncertainty in sampling rates used [ASTM 2013a]. (NIOSH methods typically do not cite performance for passive samplers because agreement among diffusive monitor manufacturers on test protocols has not yet been achieved, and a system of third party evaluation of diffusive monitor manufacturers sampling rates is not available.) Similarly, the calculation of desorption efficiencies may be performed only once or infrequently and can, therefore, introduce residual bias in measurements that use sorbent-captured samples, e.g., charcoal tubes. In aerosol sampling, detailed knowledge of the particle size-dependent bias of a sampler relative to a sampling convention, such as adopted by ISO/CEN/ACGIH/ASTM [ISO 1995; CEN 1993; ACGIH 2015; ASTM 2013b] for defining respirable dust, is often necessary to judge the usefulness of a given sampler. Each type of aerosol sampler is characterized by specific particle collection characteristics, and some analytical methods (e.g. silica) may also exhibit particle size effects. Typically the issue of aerosol sampler bias is avoided or minimized in the industrial hygiene field by narrowing use to a specific aerosol sampler. For example, common industrial hygiene practice establishes a single sampler type, such as the 1.7 L/min 10-mm nylon Dorr-Oliver cyclone, for respirable dust sampling in a particular application. Sensitivity to other environmental factors, referred to in ISO GUM as influence variables, must be acknowledged. Suppose a sampler is sensitive to temperature changes that are impractical to measure in the field; i.e., sampler estimates are not temperature corrected. Then, suppose during method evaluation in the laboratory, measurement of this sensitivity is combined with knowledge of the expected temperature variation for a given field application. Putting together both would determine the uncertainty associated with the effect. Examples of the important effects of influence variables - such as wind velocity, temperature, pressure, and fluctuating workplace concentrations - on diffusive monitor uptake rates are common.bThis approach is important to industrial hygiene since workplace air concentrations vary spatially and over time to such a degree that a method cannot be evaluated by simply taking replicate measurements [Vaughan et al. 1990]. However, industrial hygiene measurement methods have traditionally required confidence levels greater than 50% in the method evaluation. Generally, 95% confidence in a method validation is required. The different types of confidence levels are reflected simply in the numerical value and interpretation of the coverage factor. Of equal importance in the industrial hygiene field are details needed to handle systematic error (bias) relative to reference concentration measurements found during method evaluation. For example, the sampling rate of a given diffusive sampler for gases or vapors is generally measured once by the diffusive sampler manufacturer prior to use by multiple clients. As the samplers are not re-calibrated for each use, residual bias exists in the measurements due to uncertainty in sampling rates used [ASTM 2013a]. (NIOSH methods typically do not cite performance for passive samplers because agreement among diffusive monitor manufacturers on test protocols has not yet been achieved, and a system of third party evaluation of diffusive monitor manufacturers sampling rates is not available.) Similarly, the calculation of desorption efficiencies may be performed only once or infrequently and can, therefore, introduce residual bias in measurements that use sorbent-captured samples, e.g., charcoal tubes. In aerosol sampling, detailed knowledge of the particle size-dependent bias of a sampler relative to a sampling convention, such as adopted by ISO/CEN/ACGIH/ASTM [ISO 1995; CEN 1993; ACGIH 2015; ASTM 2013b] for defining respirable dust, is often necessary to judge the usefulness of a given sampler. Each type of aerosol sampler is characterized by specific particle collection characteristics, and some analytical methods (e.g. silica) may also exhibit particle size effects. Typically the issue of aerosol sampler bias is avoided or minimized in the industrial hygiene field by narrowing use to a specific aerosol sampler. For example, common industrial hygiene practice establishes a single sampler type, such as the 1.7 L/min 10-mm nylon Dorr-Oliver cyclone, for respirable dust sampling in a particular application. Sensitivity to other environmental factors, referred to in ISO GUM as influence variables, must be acknowledged. Suppose a sampler is sensitive to temperature changes that are impractical to measure in the field; i.e., sampler estimates are not temperature corrected. Then, suppose during method evaluation in the laboratory, measurement of this sensitivity is combined with knowledge of the expected temperature variation for a given field application. Putting together both would determine the uncertainty associated with the effect. Examples of the important effects of influence variables - such as wind velocity, temperature, pressure, and fluctuating workplace concentrations - on diffusive monitor uptake rates are common. [Description provided by NIOSH]
• Subjects:
  Aerosols Dust Occupational Health Particulate Matter Safety Solvents Temperature
• Keywords:
  Aerosol Sampling Analytical Methods Diffusion Analysis Gases Industrial Hygiene Particle Size Particle Size Analysis Respirable Dust Sampling Sampling Methods Temperature Effects Vapors

  More +
• Publisher:
  National Institute for Occupational Safety and Health
• Document Type:
  Text
• Genre:
  Book
• Place as Subject:
  Ohio ; OSHA Region 5
• CIO:
  National Institute for Occupational Safety and Health (NIOSH)
• Division:
  DART - Division of Applied Research and Technology
• Topic:
  Workplace Safety & Health
• Location:
  North America
• NIOSHTIC Number:
  nn:20048069
• Citation:
  NIOSH manual of analytical methods, fifth edition. Ashley K, O'Connor PF, eds. Cincinnati, OH: U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, DHHS (NIOSH) Publication No. 2014-151, 2016 Apr; :UA1-UA23
• Editor(s):
  Ashley K; O'Connor PF
• Federal Fiscal Year:
  2016
• Peer Reviewed:
  False
• Source Full Name:
  NIOSH manual of analytical methods, fifth edition
• Collection(s):
  National Institute for Occupational Safety and Health
• Main Document Checksum:
  urn:sha-512:cbfc352a7f322b9cb9dcb53ad7f641685948efc303784022c39ec82cd1114180abb78726c296313947dce4ccf249bd046595c6db88557cc5e59c1ade2f85abf3
• Download URL:
  https://stacks.cdc.gov/view/cdc/202937/cdc_202937_DS1.pdf
• File Type:
  [PDF - 565.02 KB ]