In general, the main study likelihood piece based on observed data pairs ( Ym∗,Xm∗) (m = 1, …, M) can be expressed as: [5]Lmain=∏m=1Mπ11∗(ym∗xm∗)π01∗((1-xm∗)ym∗)π10∗(xm∗(1-ym∗))π00∗((1-xm∗)(1-ym∗)), where the π^*s take appropriate forms corresponding to different assumptions on the misclassification process as described in Section 2.1 and m denotes for the main study sample. For instance, if parameterizing in terms of SE/SP and allowing differential and dependent misclassification, we have π11∗=SEY11π11SEX1+SEY01π01(1-SPX1)+(1-SPY11)π10SEX0+(1-SPY01)π00(1-SPX0). In contrast, if independence is assumed while preserving differentiality on both variables, π11∗=SEY1π11SEX1+SEY0π01(1-SPX1)+(1-SPY1)π10SEX0+(1-SPY0)π00(1-SPX0). Under the most simplified setting (e.g. Barron, 1977), the simultaneous assumptions of independent and nondifferential misclassification imply that π11∗=SEYπ11SEX+SEYπ01(1-SPX)+(1-SPY)π10SEX+(1-SPY)π00(1-SPX). The other π^*s are derived similarly under each scenario (Tang, 2012). Note that the “main study only” likelihood in eq. [5] is directly applicable solely for sensitivity analysis. We emphasize extensions to accommodate a main/internal validation design in Section 2.5.

We generalize the concept of the matrix method and its extensions (Kleinbaum et al., 1982; Greenland and Kleinbaum, 1983) by flexibly incorporating the full range of possible misclassification models. In general, one is able to relate surrogate and true cell probabilities via the equality Π^* = AΠ, where Π = (π₁₁ π₀₁ π₁₀ π₀₀)′, Π∗=(π11∗π01∗π10∗π00∗)′ and the definition of A varies according to the assumptions made. For differential and dependent misclassification, we derive A in its most general form as follows: A=[SEY11SEX1SEY01(1-SPX1)(1-SPY11)SEX0(1-SPY01)(1-SPX0)SEY10(1-SEX1)SEY00SPX1(1-SPY10)(1-SEX0)(1-SPY00)SPX0(1-SEY11)SEX1(1-SEY01)(1-SPX1)SPY11SEX0SPY01(1-SPX0)(1-SEY10)(1-SEX1)(1-SEY00)SPX1SPY10(1-SEX0)SPY00SPX0]

Under other assumptions, the matrix A can be derived as in Appendix 1. The matrix method identity relies upon inversion of the matrix A in order to obtain the vector Π = A⁻¹Π^*.

The inverse matrix identity directly expresses true cell probabilities as sums of products of surrogate cell probabilities and predictive values. Here, we extend the proposal of Marshall (1990) to a general context with both variables misclassified in a 2 × 2 table. For example, under dependent and differential misclassification, the law of total probability dictates that π11=PPVY11π11∗PPVX1+(1-NPVY11)π10∗PPVX0+PPVY10π01∗(1-NPVX1)+(1-NPVY10)π00∗(1-NPVX0). Packaging linear equations into matrices, the form of the generalized inverse matrix method is as given in Marshall’s original proposal: Π = BΠ^*. However, in our approach, the matrix B takes a more complicated form to accommodate a general misclassification mechanism for both the X and the Y variables: B=[PPVY11PPVX1PPVY10(1-NPVX1)(1-NPVY11)PPVX0(1-NPVY10)(1-NPVX0)PPVY01(1-PPVX1)PPVY00NPVX1(1-NPVY01)(1-PPVX0)(1-NPVY00)NPVX0(1-PPVY11)PPVX1(1-PPVY10)(1-NPVX1)NPVY11PPVX0NPVY10(1-NPVX0)(1-PPVY01)(1-PPVX1)(1-PPVY00)NPVX1NPVY01(1-PPVX0)NPVY00NPVX0]

In contrast to the generalized matrix method, there is no matrix inversion involved in computing the corrected OR through the generalized inverse matrix method. In principle, this could confer a numerical advantage in practice, although again direct use of the identity is generally restricted to the setting of sensitivity analysis.

The estimate of the corrected OR is OR^=π11^π00^π10^π01^. For all of the approaches presented above, estimation of misclassification probabilities is crucial in practice. When possible, we recommend the use of an internal validation subsample randomly selected from one’s current study, for which both true binary variables are measured via gold-standard methods along with the error-prone methods used in the main study. The primary appeal of adopting internal (as opposed to external) validation sampling is the avoidance of the necessity to assume “transportability” of misclassification probabilities (Begg, 1987; Carroll et al., 2006) and the accommodation of more general misclassification mechanisms.

When allowing full generality, that is, dependent and differential misclassification, it can be shown that a full likelihood approach based on the proposed main/internal validation design is equivalent regardless of whether parameterized based on predictive values or SE/SP probabilities (Tang, 2012). There are in total 16 types of validation set records, if validations on X and Y are measured simultaneously for each subject in the subsample. Table 1 shows the likelihood contributions for each validation record type based on both parameterizations. In contrast, the main study likelihood based on (X^*, Y^*) records is given explicitly in eq. [5], that is,

If parameterizing in terms of SE and SP values, all the π^*s are further expanded (see Section 2.2).

The internal validation subsample likelihood is given by

Lval=∏p=116Lvpnvp, where L_vp is the likelihood term corresponding to observation type p in Table 1, while n_vp is the total number of observations of the pth type (p = 1, 2, … 16). Note that the total validation study sample size is nv=∑p=116nvp. The overall likelihood to be maximized is based on a total of M + n_v subjects and is proportional to the product of the main and validation study components, i.e. L_main × L_val.

There are no closed-form solutions for the MLEs based on the overall likelihood written in terms of SE and SP. Interestingly, however, closed forms exist for the predictive value parameterization in the most general case. For example, one can readily verify that

π11∗^=∑i=1M+nvIXi∗=1,Yi∗=1M+nvandPPV^Y11=∑i=1nvIval=1,yi∗=1,yi=1,xi=1,xi∗=1∑i=1nvIval=1,yi∗=1,xi=1,xi∗=1, where the I notation represents an indicator that the conditions described in the subscript are met (Tang, 2012). The MLEs for the πs can then be estimated from the π∗^s,PPV^s, and NPV^s by direct use of the generalized inverse matrix identity of Section 2.4. Because the two parameterizations are equivalent under the circumstance of dependent and differential misclassification, we may also obtain closed-form MLEs for the SE^ and SP^ parameters as functions of the PPV^s and NPV^s in that setting. For example,

The remaining closed-form MLEs are displayed in Appendix 2.

When the misclassification process is not fully general (e.g. assuming independent misclassification and/or nondifferential misclassification of either variable), the equivalence between the likelihoods based on the SE/SP and predictive value parameterizations no longer holds. In such cases, it appears that there are no simple closed forms for likelihood-based SE^s,SP^s, and π̂s. If one supplies the generalized matrix method with data-driven SE and SP estimates that are not MLEs, the corrected OR^ will not be fully efficient. These conclusions are consistent with previous findings in a simpler context, with misclassification of only one variable (Lyles, 2002).

In general, we recommend the use of the ML approach for optimal efficiency and the ease of numerically computing standard errors. Optimizing the full main/internal validation likelihood under either parameterization path is readily achieved by taking advantage of numerical procedures in standard statistical software. As such, we view the matrix and inverse matrix constructs more as instructive identities than as practical analysis tools, unless they are to be used solely for sensitivity analyses. Straightforward multivariate delta-method calculations allow computing the approximate standard error of the corrected log(OR^) based on ML, after obtaining the π̂s and the corresponding numerically-derived Hessian. SAS NLMIXED (SAS Institute, Inc., 2008) programs for accomplishing these tasks are readily available from the first author.

A natural question one might ask is whether measuring (X^*, Y^*) on every subject in addition to (X, Y) yields a different or improved estimate of the true OR characterizing the (X, Y) association. In fact, if (X, Y, X^*, Y^*) is available on all participants, the available information for estimating the OR is equivalent to that contained in the (X, Y) data alone. The overall likelihood then reduces to L_val. In Appendix 3, we show that maximizing the reduced form of the overall likelihood (L_val only) under the most general misclassification model in this situation leads to exactly the same MLEs of the πs as those obtained from analyses ignoring (X^*, Y^*). A similar argument can be readily derived under other types of misclassification models. This finding unsurprisingly suggests that knowing surrogates when gold-standard measures are available on the whole sample does not offer additional value in the estimation of the primary effect (e.g. OR) of interest, which further implies that if gold-standard measures are comparatively affordable compared to surrogates, it is more efficient to evaluate via gold standards only.

While our focus has been on cross-sectional sampling, the case–control sampling scheme is also worthy of discussion. Here, we consider “case–control” studies as those where case oversampling is conducted based on the error-prone responses. In other words, observations with Y^* = 1 (“cases”) are sampled with a greater probability than those with Y^* = 0 (“controls”). Prior work (Greenland and Kleinbaum, 1983) has noted that supplying the population misclassification probabilities to the correction methods will yield invalid estimates; however, with nondifferential misclassification, the validity of the analytic results could be restored by introducing the sampling fraction of cases and controls into the correction. It was also noted in Lyles et al. (2011) that the main/internal validation design is favorable for handling such oversampling under nondifferential misclassification, because it automatically yields estimates of the “operating” misclassification probabilities. Similar findings are observed in the current setting. With oversampling of “cases” (Y^* = 1), the method described in the previous sections yields valid estimation of the OR, as long as misclassification of Y is nondifferential. When the nondifferential misclassification assumption is not met, however, the validity of the estimated OR based on the main/internal validation design does not hold under “case” oversampling. More details can be found in Tang (2012).

When correcting the estimate of the OR, we would ideally choose the misclassification mechanism that generated the observed data. Here, we provide a straightforward model selection procedure to guide practitioners. For ease of discussion, denote the dependent and differential misclassification model as “Model 1”, followed by “Model 2” (the independent and differential misclassification model in Section 2.1.2) and “Model 3” (the completely nondifferential model in Section 2.1.3). Model 1 reflects a fully general misclassification mechanism, while Model 2 can be regarded as a generalization of Marshall’s (1990) framework to the situation when both X and Y are misclassified and Model 3 is a representation of Barron’s (1977) setting.

Define AIC_q = the value of the Akaike Information Criterion (AIC) (Akaike, 1974) upon fitting Model q (q = 1, 2, 3). In practice, we recommend selecting the model that yields the smallest value of AIC, as that criterion is well known to balance between the number of necessary parameters included and the quality of model fit. One may then simply report the results corresponding to the selected model. Although a more accurate standard error for the resulting estimated log(OR) might presumably be obtained via resampling, our empirical studies suggest that it is suitably reliable and computationally efficient to report the standard error from the selected model (see Section 4). We apply this AIC-based approach to real data in the following section, and a program utilizing the SAS NLMIXED procedure to implement the model selection method is available from the first author. For additional comments regarding selection of the misclassification model, see Section 5.

Our first simulation experiment evaluates the performance of the proposed methods under conditions mimicking the HERS example (Section 3). Cell counts were simulated from a multinomial distribution with cell probabilities of (π₁₁ = 0.1146, π₁₀ = 0.2871, π₀₁ = 0.0677, π₀₀ = 0.5306), and main and internal validation sample sizes (n_m = 687, n_v = 219) similar to those observed in the HERS example. Error-prone response Y^* and exposure X^* were generated with misclassification probabilities estimated from the HERS sample based on the fit of Model 1 (data available in Table 7), where the misclassification process was assumed dependent and differential. For each of 500 simulated datasets, we conducted naïve analysis associating Y^* with X^*, true analysis with Y and X, and main/internal validation analyses via Models 1–3.

Table 3 summarizes the results. The naïve analysis yields a result biased away from the null. Model 1 produces the corrected OR estimate closest to the gold-standard OR, with tolerable sacrifice in efficiency. The 95% CI coverage under Model 1 is also excellent. When reducing Model 1 to other simpler versions by assuming independent or nondifferential misclassification, the results are biased, reflecting the fact that the reduced models are not consistent with the data generation process. Note that with the simplest model assuming nondifferential misclassification of both variables (Model 3), the corrected result is similar to the naïve result (in fact, arguably worse). This strongly highlights the importance of internal validation data to permit flexibility in the selected misclassification model.

The corrected results using the generalized matrix methods discussed in Section 2.1.1 agree well with the MLEs, when ML estimates of misclassification probabilities are supplied. However, when simpler crude estimates obtained from the validation subsample are inserted into the generalized matrix method, results are not satisfying, even producing negative estimates of probabilities in some cases (Tang, 2012; results not shown). Thus, in practice, we favor the proposed main/internal validation study-based full ML approach in the interest of obtaining both valid and efficient results.

The results in Section 4.1 suggest the importance of misclassification model selection to ensure the model is specified correctly (or, at least, generally enough). Extensive simulations were performed to evaluate the performance of the proposed AIC-based model selection strategy (Tables 4–6), when the underlying association was negative (Table 4), or moderate positive (Table 5), or strong positive (Table 6). Under various settings, the model was chosen correctly most of the time. For example, under setting 4, the true underlying model from which data were generated was Model 3. Unsurprisingly, the more general Models 1 and 2 yield valid results. However, with the proposed model selection strategy, Model 3 is correctly picked 88.0% of the time, yielding a slight improvement in efficiency relative to Model 1. In contrast, under setting 6, Model 1 is the underlying model; thus, estimates from Models 2 and 3 are not valid. By correctly selecting Model 1, 94.8% of the time, however, the model selection strategy maintained overall validity and achieved satisfactory 95% CI coverage.

The simulation results in Tables 4–6 suggest that AIC is a highly effective criterion for selecting among the alternative misclassification models. The key concern, however, is maintenance of validity in the OR estimate. Since the true misclassification model is unknown, only Model 1 ensures such validity in theory. Thus, whenever the internal validation subsample is of adequate size to support its fit, Model 1 must be viewed as the safest choice. Another argument in favor of Model 1 is the fact that, at least under the simulation conditions examined here, it produced a log(OR) estimate with very similar mean and variance properties to those characterizing the MLEs under simpler true underlying misclassification models.