A process is described to assess the commutability of a reference material (RM) intended for use as a calibrator, trueness control, or external quality assessment sample based on the difference in bias between an RM and clinical samples (CSs) measured using 2 different measurement procedures (MPs). This difference in bias is compared with a criterion based on a medically relevant difference between an RM and CS results to make a conclusion regarding commutability. When more than 2 MPs are included, the commutability is assessed pairwise for all combinations of 2 MPs. This approach allows the same criterion to be used for all combinations of MPs included in the assessment. The assessment is based on an error model that allows estimation of various random and systematic sources of error, including those from sample-specific effects of interfering substances. An advantage of this approach is that the difference in bias between an RM and the average bias of CSs at the concentration (i.e., amount of substance present or quantity value) of the RM is determined and its uncertainty estimated. An RM is considered fit for purpose for those MPs for which commutability is demonstrated.

Commutability was defined in part 1 of this series (^{16} and clinical samples (CSs) measured using 2 different measurement procedures (MPs). This difference in bias is compared with a predefined criterion to make a commutability judgment. If more than 2 MPs are included in an assessment, the commutability is assessed pairwise for all combinations of 2 MPs. If 1 of the MPs is a reference measurement procedure (RMP), then commutability of the RM with each of the MPs can be assessed vs the RMP, and pairwise assessment among all MPs is not necessary.

As explained in part 1 of this series (

Currently applied approaches for commutability assessment use criteria based on the statistical distribution of results for CSs observed between pairs of MPs. Linear regression with prediction interval for the CSs has been commonly used to determine whether an RM is commutable (

An advantage of the approach described here is that the difference in bias between an RM and the average bias of CSs at the concentration of the RM is determined. This approach allows more relevant assessment of commutability being suitable for the intended use of an RM with the same criterion being used for all combinations of MPs included in the assessment. Another advantage is that the criterion to make a commutability judgment can be based on medically relevant differences between RM and CS results, which is not the case with the prediction interval approach.

In practice, an assessment of commutability cannot include all possible performance conditions for an MP, such as reagent lots, calibrator lots, and environmental conditions. We must restrict the assessment to measurements under specified conditions. In this report, the specified conditions are 1 run with each of the MPs using 1 lot of reagents and calibrators, and each MP operated and performing according to the specifications of its manufacturer. The conclusions about commutability of the RM are generalized to all future results using other IVD medical devices representative of the same MP with the assumption that other IVD medical devices have equivalent performance when operated under conditions such as different reagent and calibrator lots, maintenance, and operators. Limitations of this assumption were discussed in part 1 of this series (

A worked example and explanation of the example calculations are provided in the Commutability Example Calculations and Commutability Example Explanation sections of the

The experimental design considers the comparison of results,

_{x}

_{y}

The _{d}_{y}_{x}_{e}_{(}_{x}_{)} and _{e}_{(}_{y}_{)}, in the following denoted _{x}_{y}

Ideally, the variation within runs should be completely random, and the SDs _{x}_{y}_{d}_{d}

For commutability assessment, we recommend only 1 run and that the replicate measurements of the CSs are adjacent, i.e., made one after the other, and that position effects are investigated from measurements of the RMs made in different positions. The terms _{d}

Commutability of the RM concerns how close the systematic difference (the bias) between the 2 MPs for the RM is to the average bias for the CSs, _{RM}

For assessment of commutability, we need to specify a maximum value of |_{RM}

The SD of the contributions from the random components

Assuming a normal distribution, about 5% of the CS differences will be larger than 2_{Random}

For assessment of commutability vs _{RM}_{RM}

The RM is commutable when the uncertainty interval _{RM}_{RM}

The RM is noncommutable when the uncertainty interval _{RM}_{RM}

A commutability decision is inconclusive when the uncertainty interval _{RM}_{RM}

When _{RM}_{RM}

The experiment should be designed in such a way that it is possible to obtain a reliable estimate of the value _{RM}

A number,

The number of CSs,

The minimum number of replicates,

Measuring the RMs in different positions makes it possible to investigate position effects and better estimate the uncertainty of the bias for an RM. The uncertainty of the bias estimate depends on the SD of the position effects and the number of degrees of freedom must be adequate for the estimate of this SD. We recommend that the number of positions,

The estimate of the commutability measure, _{RM}_{RM}

The statistical analysis uses difference plots with or without transformation of the data for the statistical analysis. Difference plots are preferred to regression because possible trends in bias or sample-specific effects over the concentration interval are better identified.

For the statistical analysis, it is an advantage if the SDs of the random components are independent of the concentration. This requirement is often approximately satisfied for either concentration or ln(concentration). Other types of transformations can be used, but the advantage of using ln(concentration) is that it is easy to interpret the results. If SDs and differences of ln(concentration) are multiplied by 100, we obtain approximate values of CVs and relative differences in percent for concentration. The decision whether to use concentration or ln(concentration) for the statistical analysis is based on the experimental results. A precision profile as shown in _{x}_{y}

Difference plots are examined for _{i}_{i}_{i}_{i}_{i}_{i}_{i}_{i}_{i}_{i}_{i}_{x}_{y}

The decisions whether to transform and whether to perform the statistical analysis for different concentration intervals are based on subjective judgments. Models and assumptions are always approximations of reality, and it is better to use a subjective judgment than to use a fixed model with no judgment at all.

Outliers are observations that are distant from the main part of the observations. An outlier may be because of occasional problems in an MP, experimental mistakes, mix-up of results, transcription errors, or other types of operator errors. If such a cause is identified, the observation should be corrected or excluded. There may, however, be outliers for which the causes cannot be identified; for instance, when some of the CSs in a comparison of MPs have properties that cause ≥2 separate distributions of the differences between the MPs. To determine how close the bias for the RM is to the average bias for the CSs is not meaningful if the average bias represents 2 populations of CSs. The populations of CSs corresponding to these different distributions should, if possible, be identified (e.g., healthy and diseased donors) and commutability assessments performed for each population. If the outliers are relatively few (<10%), the only reasonable approach often is to exclude them. If the outlier results are not excluded, the calculation of the mean and the SD may be misleading.

Often a visual inspection is sufficient for identification of possible outliers. Obvious outliers can often be identified from, for instance, precision profiles and difference plots. If there are borderline cases, one can perform the analysis both without and with exclusions. If inclusion or exclusion of potential outliers gives essentially the same estimates, the observations can be included.

For each RM, we have _{Pos-mean}_{e}_{Pos-mean}_{e}_{e}

The SD of the position effects is estimated by:

When the value under the root sign is negative, (_{Pos}

The analysis is performed for either concentration or ln-(concentration). In the following, _{i}_{i}_{x}_{y}_{x}_{y}

The differences _{i} = y_{i}_{i}_{i}_{Random}

It is often reasonable to assume that the maximum difference between consecutive concentrations of the CSs corresponds to a maximum change of the bias function, which is small compared with the contribution from the within-run variation to the observed differences _{i}

_{i}_{i}_{i}_{i}_{MSSD}_{Random}_{CS}_{i}_{MSSD}_{B}_{MSSD}_{B}^{2} is approximately normal with mean 1 and

When there are sample-specific effects, the _{MSSD}_{MSSD}

With the suggested experimental design (triplicate measurements), _{MSSD}_{MSSD}_{2} − _{1}, _{4} − _{3}, and so on, it is based on _{MSSD}

In the denominator in _{x}_{y}

An estimate of _{d}

If the expression under the root sign is negative, the estimate _{d} =

When there are no trends, _{B}_{MSSD}

If there are position effects, these effects are included in the estimate _{d}_{d}_{(}_{corr}_{)}]

The commutability assessment and the calculation of the expanded uncertainty _{RM}

The commutability of the RM is assessed as the difference between the bias for the RM and the average bias for the CSs, _{RM}_{RM}_{RM}_{RM}_{RM}

_{Pos-mean}_{(}_{x}_{)} and _{Pos-mean}_{(}_{y}_{)} are the SDs between position means for the RMs defined in the Components of Variation Within Runs Estimated from the RMs section.

The appropriate estimate of _{RM}

_{CS}_{B}

_{RM}_{RM}_{CS}

It is possible to find a concentration interval with

The number

The mean bias of the _{B}_{B}_{MSSD}

A large magnitude in the bias trend indicates a severe problem, and the commutability assessment may not be possible.

It may be tempting to fit a model to

To evaluate commutability, we need the expanded uncertainty _{RM}_{RM}_{RM}_{RM}_{RM}_{Pos-mean}

A suitable way to illustrate the results from a commutability assessment is shown in

In most cases, more than 2 MPs are included in a commutability assessment, and all combinations of 2 MPs are evaluated as pairs as described here and shown in

The approach to commutability assessment described here gives estimates of the differences in bias between RMs and CSs when 2 MPs are compared. The uncertainties of these estimates are also calculated. A single fixed criterion for commutability of an RM can be applied to all combinations of pairs of MPs. The criterion can be selected based on the intended use of an RM as a calibrator, trueness control, or external quality assessment material, and the criterion can be related to the requirements for medical decisions based on the laboratory test results. The commutability assessment determines whether the difference in bias plus its uncertainty fulfills the criterion for a conclusion that an RM is commutable, noncom-mutable, or indeterminate for pairs of MPs. Conclusions regarding suitability for use of an RM can be made by assessing its commutability for all MPs in the assessment as described in part 1 of this series (

Nonstandard abbreviations: RM, reference material; CS, clinical sample; MP, measurement procedure; RMP, reference measurement procedure; IVD, in vitro diagnostic; C, commutability criterion; MSSD, mean square successive difference.

Disclaimer: The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the official views or positions of the Centers for Disease Control and Prevention/Agency for Toxic Substances and Disease Registry.

(A) shows an approximately constant SD over the concentration interval. (B) shows a proportional relationship between SD and concentration.

(A) is from spreadsheet tab CS_Trans found in the

The solid black line is the mean bias between the 2 measurement procedures for the CSs. The red dashed lines are the commutability criteria. The red squares are the mean bias between the 2 MPs for the RMs, and the bars are the uncertainty in the difference in bias between RM and CS mean bias. RM1, RM2, and RM4 are indeterminate; RM3 is commutable; RM5 is noncommutable. Fig. 3 is from the spreadsheet tab CS&RM_Diff found in the

(A) shows a representation of commutability conclusions for all combinations of pairs of MPs. (B) shows a representation when an RMP is available. The notation C, I, N could be replaced with numeric values for difference in bias and its uncertainty if desired.