90096441090EpidemiologyEpidemiologyEpidemiology (Cambridge, Mass.)1044-39831531-548725120106447752810.1097/EDE.0000000000000166HHSPA699854ArticlePotential sensitivity of bias analysis results to incorrect assumptions of nondifferential or differential binary exposures misclassificationJohnsonCandice Y.abFlandersW. DanaaStricklandMatthew J.aHoneinMargaret A.bHowardsPenelope P.aRollins School of Public Health and Laney Graduate School, Emory University, Atlanta, GANational Center on Birth Defects and Developmental Disabilities, Centers for Disease Control and Prevention, Atlanta, GACorresponding author: Candice Y. Johnson, National Institute for Occupational Safety and Health, Centers for Disease Control and Prevention, 1090 Tusculum Ave MS R-15, Cincinnati, OH 45226. Tel: (513) 841-4454. cyjohnson@cdc.gov136201511201401112015256902909Background

Results of bias analyses for exposure misclassification are dependent on assumptions made during analysis. We describe how adjustment for misclassification is affected by incorrect assumptions about whether sensitivity and specificity are the same (nondifferential) or different (differential) for cases and non-cases.

Methods

We adjusted for exposure misclassification using probabilistic bias analysis, under correct and incorrect assumptions about whether exposure misclassification was differential or not. First, we used simulated datasets in which nondifferential and differential misclassification were introduced. Then, we used data on obesity and diabetes from the National Health and Nutrition Examination Survey (NHANES) in which both self-reported (misclassified) and measured (true) obesity were available, using literature estimates of sensitivity and specificity to adjust for bias. The ratio of odds ratio (ROR; observed odds ratio divided by true odds ratio) was used to quantify magnitude of bias, with ROR=1 signifying no bias.

Results

In the simulated datasets, under incorrect assumptions (e.g., assuming nondifferential misclassification when it was truly differential), results were biased, with RORs ranging from 0.18 to 2.46. In NHANES, results adjusted based on incorrect assumptions also produced biased results, with RORs ranging from 1.26 to 1.55; results were more biased when making these adjustments than when using the misclassified exposure values (ROR=0.91).

Conclusions

Making an incorrect assumption about nondifferential or differential exposure misclassification in bias analyses can lead to more biased results than if no adjustment is performed. In our analyses, incorporating uncertainty using probabilistic bias analysis was not sufficient to overcome this problem.

Bias analysis (sensitivity analysis) has been proposed as an improvement over the qualitative descriptions of study limitations and potential sources of bias typically provided by investigators, in which potential effects of systematic error, and not only random error, are quantified.1 The quantitative nature of these analyses allows a more transparent assessment of the potential direction and magnitude of bias and also guards against the tendency of investigators to favor causation over bias as the most likely explanation for observed results.2,3 Some investigators have advocated greater incorporation of quantitative analyses for exposure misclassification and other forms of bias, 4–8 and many examples are now available in the published literature.9–14

Bias analysis for exposure misclassification involves identifying potential sources of misclassification, estimating bias parameters (e.g., sensitivity [Se] and specificity [Sp]) from validation studies or literature reviews, and using this information to adjust study results. Often, this adjustment is accomplished using simple algebraic manipulations of the contingency table. Probabilistic bias analysis extends this basic approach by allowing the investigator to assign a probability distribution to each bias parameter, sample randomly from the distribution, and perform the bias analysis repeatedly to produce a distribution of the adjusted measure of association. These probabilistic methods allow investigators to acknowledge uncertainty in choice of bias parameters and are more frequently used now that they are available in widely used software such as SAS, Stata, and Excel.2,9,15

There is discussion in the literature on choosing values or distributions of sensitivity and specificity for bias analyses of exposure misclassification.2,3 Relatively less emphasis has been given to the importance of correctly specifying in the analysis whether misclassification is nondifferential or differential. In most studies, it is unclear whether nondifferential misclassification (sensitivity and specificity are the same for cases and non-cases) or differential misclassification (sensitivity and specificity differ between cases and non-cases) is the more appropriate assumption, unless internal validation data are available – in which case sensitivity and specificity can be estimated directly, albeit often with error. It has previously been shown that assuming nondifferential misclassification in a bias analysis when misclassification is truly differential can produce a result further from the truth than the unadjusted estimate.16,17 Investigators might be hesitant to assume differential misclassification unless outcome-specific estimates of sensitivity and specificity are available or the investigator has other data specifying how they differ between cases and non-cases. In the literature, there are examples of bias analyses that use assumptions of nondifferential misclassification only13,14 or both nondifferential and differential misclassification.10–12 The potential effects of other types of incorrect assumptions have not been explored in depth.

The purpose of this study is to illustrate the potential sensitivity of bias analysis results to incorrect assumptions of nondifferential or differential misclassification of a binary variable. Through simulations, we create datasets with nondifferential and differential exposure misclassification, and we adjust for this misclassification using correct and incorrect assumptions about misclassification in probabilistic bias analyses. We then use data from the National Health and Nutrition Examination Survey (NHANES) to show how incorrect assumptions can affect the results of a bias analysis in an epidemiologic study. In the simulated datasets and in NHANES, both true and misclassified versions of exposure are known, and so we can evaluate the success of our adjustments.

EXAMPLE 1: SIMULATED DATAMethods

We began with 2 datasets with 10,000 simulated study participants each. In the first, we assigned a 10% disease prevalence and a 10% exposure prevalence among both cases and non-cases (odds ratio (OR) = 1.00). In the second, we assigned a 10% disease prevalence, 12.5% exposure prevalence among cases, and a 10% exposure prevalence among non-cases, to create a weak association between exposure and disease (OR = 1.29) (Table 1). No specific study design is implied in our simulations (i.e, cohort, case-control, cross-sectional), and we use the OR as our measure of association because it can be calculated for all study designs.

We then made 4 copies of each dataset, introducing a different type of exposure misclassification in each (2 nondifferential and 2 differential exposure misclassification scenarios). We used Bernoulli trials to randomly misclassify the simulated study participants under a given sensitivity and specificity. For the 2 nondifferential misclassification datasets (“exactly nondifferential” and “approximately nondifferential” misclassification, to be further discussed below), cases and non-cases were misclassified with Se = 0.90 and Sp = 0.90 independent of disease status. For differential misclassification, we created one dataset in which cases had higher sensitivity and specificity than non-cases (Secase = 0.95, Spcase = 0.95, Senon-case = 0.90, Spnon-case = 0.90) and one in which cases had lower sensitivity and specificity than non-cases (Secase = 0.85, Spcase = 0.85, Senon-case = 0.90, Spnon-case = 0.90). For convenience, misclassification scenarios in which cases have more accurate classification than non-cases will be referred to as “differential A misclassification” and scenarios in which cases have less accurate classification than non-cases will be referred to as “differential B misclassification”.

We then adjusted for exposure misclassification in these datasets to determine if we could obtain a less biased estimate of the true OR, even if using incorrect assumptions about nondifferential or differential misclassification in the bias analysis. To adjust for misclassification, we used a common algebraic method that involves back-calculating expected cell counts for the correctly classified 2 x 2 contingency table given cell counts for the misclassified contingency table and estimates of sensitivity and specificity (Table 2).3 Because the method uses contingency tables to adjust for bias, it can be used for other measures of association calculated from contingency tables, such as the risk difference or risk ratio.

We implemented a probabilistic analysis by specifying triangular distributions for sensitivity and specificity.2,3 Triangular distributions were chosen over other options such as trapezoidal distributions because they allow specification of a single estimate with the highest probability of being chosen, rather than a range of values; this most closely matches our scenarios in which we designated a single value as most likely. Sensitivity and specificity values (as described below) were used as the modes of their respective triangular distributions. The maximum and minimum values for the triangular distributions were assigned to be +/− 0.05 of the mode for sensitivity and specificity. The distributions were truncated when necessary so all values fell between 0.5 and 1.00, inclusive. At each of 10,000 iterations, one value of sensitivity and one value of specificity were randomly chosen from the triangular distributions for cases and again for non-cases to calculate the misclassification-adjusted OR.

For each of the misclassified datasets in the bias analysis, we adjusted for misclassification using 4 assumptions: exactly nondifferential, approximately nondifferential, differential A, and differential B misclassification. We use the term “exactly nondifferential” to mean that we used identical values of sensitivity and specificity for the cases and the non-cases. “Approximately nondifferential” means that the sensitivity and specificity triangular distributions were the same for cases and non-cases, but sensitivity and specificity values were chosen independently from the same triangular distribution, so that values could differ between cases and non-cases by chance (i.e., numerically there might be differential misclassification).

We chose values of sensitivity and specificity for the adjustments to be as close to the true values as possible so as not to conflate the effects of choosing incorrect values of sensitivity and specificity with the effects of making incorrect assumptions about nondifferential or differential misclassification. However, under scenarios where incorrect assumptions were made, no “true” values of sensitivity and specificity existed, and so we had to make alternate assumptions. The values of sensitivity and specificity used in each bias analysis adjustment (see eTable 1) were calculated as follows.

Making correct assumptions

When a correct assumption was made, sensitivity and specificity were calculated directly from the simulated population. For nondifferential misclassification, sensitivity and specificity were calculated from the whole population (cases and non-cases together). For differential A and B misclassification, sensitivity and specificity were calculated separately for cases and non-cases, and these values were used in the analysis.

Making incorrect assumptions

When assuming nondifferential misclassification, sensitivity and specificity were calculated from the whole population (cases and non-cases together). When assuming differential A, sensitivity and specificity for non-cases were calculated directly from the non-cases; sensitivity for cases was assumed to be sensitivity for non-cases + 0.05, and specificity for cases was assumed to be specificity for non-cases + 0.05. When assuming differential B, sensitivity and specificity for non-cases were calculated directly from the non-cases; sensitivity for cases was assumed to be sensitivity for non-cases – 0.05, and specificity for cases was assumed to be specificity for non-cases – 0.05.

We calculated the ratio of the misclassification-adjusted OR to the true OR (calculated using the known true exposure status). We will refer to this metric as the ratio of odds ratios (ROR); ROR=1.00 signifies no bias. Results are presented as the median ROR and 95% simulation interval (SI). The 95% SI represents the 2.5th and 97.5th percentiles of the ROR distribution generated by simulation. If the sensitivity and specificity values chosen in the analysis produced a negative OR, we did not include it in the calculation of the OR, ROR, or 95% SI. All analyses were conducted in SAS version 9.3 (SAS Institute, Cary, NC) (see eAppendix 1 for SAS code).

Results

For both scenarios of OR = 1.00 and OR = 1.29, when correct assumptions about misclassification were made, results were on average unbiased or nearly unbiased (median ROR range = 1.00 to 1.01). The exception was the situation when misclassification was truly approximately nondifferential; in this scenario, no adjustment provided an unbiased result, given our assumptions (Table 3). There was little difference in results between scenarios in which the true association was null or weakly positive, although there was slightly less bias when the association was non-null. When misclassification was truly nondifferential, the assumption of either differential A or differential B misclassification produced biased results on average, but the biases were in different directions. When differential A was the true type of misclassification, all incorrect assumptions underestimated the magnitude of the association. When differential B was the truth, all incorrect assumptions overestimated the association. The 95% SIs included the true (null) value under some assumptions but not others.

Compared with the misclassified (unadjusted) ORs, the misclassification-adjusted ORs were closer to the truth only when the correct assumption was made for exactly nondifferential, differential A, or differential B misclassification (i.e., results were unbiased). Under all incorrect assumptions and for all assumptions when the truth was approximately nondifferential, the adjusted OR was farther from the truth than the unadjusted estimate.

EXAMPLE 2: DATA FROM AN EPIDEMIOLOGIC STUDYMethods

In this example, we investigate an association between obesity and diabetes in NHANES using literature estimates of sensitivity and specificity, thereby combining the effects of making an incorrect assumption of nondifferential and differential misclassification with potential misspecification of the sensitivity and specificity distributions.

We included non-pregnant women aged 18 to 49 participating in NHANES between 1999 and 2010. NHANES uses a complex, multistage, probability sampling design to select participants from the civilian, non-institutionalized population of the United States.18 NHANES participants complete an in-person interview during which they self-report height and weight. One or two weeks later, they visit a mobile examination center during which their height and weight are measured. Women with missing values for self-reported or measured height or weight were excluded from our analysis.

Obesity (exposure) was defined as body mass index ≥ 30 kg/m2, calculated as weight in kilograms divided by squared height in meters. The reference group for all our analyses is non-obese women (body mass index <30 kg/m2). We will refer to obesity status calculated from self-reported height and weight as “self-reported obesity” (misclassified exposure), and obesity status calculated from measured height and weight as “measured obesity” (true exposure). For the purposes of this example, we assume that measured obesity is measured without error.

Self-reported diagnosis of diabetes (outcome) was obtained by questionnaire. In our analysis, no distinction was made between type 1 and type 2 diabetes. Women who reported “borderline” diabetes were categorized as having no diabetes diagnosis. We excluded women with missing data on diabetes status. For simplicity, we assume that diabetes status was reported with no misclassification.

In this analysis, our purpose was to simulate a bias analysis for which values of sensitivity and specificity were obtained through literature review (external validation data). We searched the literature for studies presenting sensitivity and specificity for self-reported compared with measured obesity among adult females in the United States (see eAppendix 2 for literature search details). We excluded estimates from published NHANES data because our purpose was to approximate an adjustment for misclassification when internal validation data were unavailable.

Based on the results of the literature review, we created triangular distributions for Se and Sp (described further in Results). We conducted the bias analysis using the same probabilistic adjustment method previously described and using the same 4 assumptions: exactly nondifferential, approximately nondifferential, differential A, and differential B misclassification. Probabilistic adjustment for misclassification was conducted over 10,000 iterations to generate a distribution of the misclassification-adjusted prevalence odds ratio (POR). Results are presented as the median POR and 95% SI and as median ROR and 95% SI. For simplicity, in this example we did not take into account in the analysis the complex sampling design of NHANES and as such, the results should not be interpreted as being representative of the United States population.

Results

We identified 5 published studies19–23 including estimates of sensitivity and specificity that met inclusion criteria (see eTable 2 for study details). Because most estimates of sensitivity were near 0.90, we chose this as the mode of the triangular distribution and assigned a minimum and maximum of 0.85 and 0.95 to allow for uncertainty. For specificity, we chose a mode of 0.97 based on the average of all estimates and the minimum (0.94) and maximum (1.00) values of the distribution based on the highest and lowest estimates obtained through literature review. No study provided diabetes-specific estimates of sensitivity or specificity.

In our NHANES population of 8,123 women, the POR between measured obesity and diabetes was 6.06 and the misclassified POR between self-reported obesity and diabetes was 5.53 (ROR=0.91). After adjusting for misclassification, misclassification-adjusted median PORs ranged from 6.39 to 9.37 (median ROR range = 1.05 to 1.55) (Figure 1). All adjustments overestimated the magnitude of the association.

The misclassification assumption producing estimates nearest to the truth was differential misclassification A (POR = 6.39 [95% SI = 5.24 – 7.96; ROR = 1.05 [95% SI = 0.87 – 1.31]). The true misclassification type in NHANES most closely resembled differential misclassification A, which explains why this estimate was the least biased. However, our estimates of sensitivity and specificity from literature review were inaccurate. The true sensitivity was 0.09 higher in cases than non-cases (we assumed 0.05 higher) and the true specificity was 0.01 lower in cases than non-cases (we assumed 0.05 higher) (Table 4). This explains why the final estimate was slightly biased.

We repeated the analyses using sensitivity and specificity estimates abstracted from each of the 5 studies identified in the literature review separately (see eTable 3 for sensitivity and specificity values) to determine if any one study could have provided a more accurate estimation of sensitivity and specificity. Under the most accurate assumption (differential misclassification A), median misclassification-adjusted PORs ranged from 6.15 to 7.11 (median ROR range = 1.01 – 1.17) (Figure 2). All included the true POR of 6.06 in the 95% SI.

DISCUSSION

We presented examples in which adjustment for exposure misclassification was undertaken using various assumptions about differential and nondifferential misclassification. Using simulations and data from an epidemiologic study, we found that making incorrect assumptions about exposure misclassification can produce “bias-adjusted” results that are biased, and in some cases more biased than the unadjusted estimates.

Investigators are encouraged to be cautious when presenting and interpreting results from bias analyses because results are valid only if the assumptions used in the analysis at least approximate the truth.2 In discussions of bias analysis in the literature, more emphasis has been given to choosing distributions of sensitivity and specificity for bias analysis than to choosing the correct assumption regarding nondifferential or differential misclassification. This is not surprising, given that outcome-specific sensitivity and specificity can be difficult to find in the literature, thus providing no evidence for suspecting if misclassification might be differential or not. Published validation studies providing such information would be useful contributions to the literature, facilitating the addition of bias analyses to epidemiologic studies by providing estimates of sensitivity and specificity as evidence-based starting points. However, there will be no guarantee that the values of sensitivity and specificity from one study will be generalizable to another.

In our examples, making an incorrect assumption about nondifferential or differential misclassification produced biased results. Taking uncertainty into account in the analysis by assigning probability distributions to Se and Sp was not sufficient to make up for the incorrect assumption, with the 95% SIs often not including the true value. Widening our triangular distributions might have allowed the 95% SI to cover the true value more frequently, although widening the distribution must be balanced against the ability to be sufficiently precise to interpret results from the analysis.

In our analyses, we considered two types of nondifferential exposure misclassification: “exactly” and “approximately” nondifferential misclassification. Exactly nondifferential misclassification (cases and non-cases have the exact same values of sensitivity and specificity) is rare in reality, because even if misclassification operates through a nondifferential mechanism of systematic error, random error will likely make sensitivity and specificity differ between cases and non-cases. We have referred to scenarios in which sensitivity and specificity differ between cases and non-cases by chance as “approximately” nondifferential misclassification, even though numerically this situation might appear to resemble differential misclassification. This is a potentially realistic scenario for many studies, in which the mechanism is nondifferential, but sensitivity and specificity likely differ by chance. In our simulated study results, when approximately nondifferential misclassification was the truth, none of our adjustment assumptions provided unbiased estimates of the magnitude of the association.

In the simulated data, when correct assumptions about nondifferential or differential misclassification were made, the results were on average unbiased even though some individual bias-adjusted ORs deviated substantially from the truth. In these simulations, we knew the true values of sensitivity and specificity in the population and used them in the analysis. If incorrect estimates of sensitivity and specificity were used to adjust for misclassification, results could be biased even when making the correct assumption about misclassification being differential or not. In the NHANES example, our estimates of sensitivity and specificity from literature review were not always accurate. As a result, none of the adjustments produced an unbiased estimate on average. However, the simulations based on 5 literature review estimates of sensitivity and specificity produced RORs ranging from 1.01 to 1.17 under the most appropriate assumption of differential A misclassification. This suggests that, if values of sensitivity and specificity are misspecified but still reasonably close to the true values, then results near to the truth can still be obtained.

We chose NHANES as the source of data for this analysis because both self-reported and measured versions of the obesity variable were available, allowing evaluation of the potential impact on results of making correct and incorrect assumptions. Because we used a specific dataset with its accompanying limitations, results from our analysis might not be generalizable to all bias analysis results. For example, because NHANES participants are likely aware they would be weighed and measured after self-reporting their weight and height, they might have reported their weight and height more accurately than persons who do not know they will be weighted, making it difficult to find suitable estimate of sensitivity and specificity from literature review. In addition, we used unweighted NHANES data in our analysis, and the applicability of literature estimates of sensitivity and specificity to the NHANES unweighted sample is unknown. Also, we used potentially misclassified self-reported diabetes status as the outcome and did not adjust for potential confounders such as age or sex, which could have affected our results. However, all investigators will face similar issues when conducting bias analysis for exposure misclassification. Our ability to produce fairly unbiased estimates when making a correct assumption about the type of misclassification in the face of these limitations should be encouraging.

In the absence of validation data (and even when validation data are available, because results of validation studies are themselves subject to error), the rationale for choosing nondifferential versus differential misclassification in a bias analysis is often left to the investigator’s perception of how misclassification occurred in the study. This commonly consists of a qualitative description of the possible sources of bias without presentation of evidence supporting the decision.24 This is similar to the qualitative discussion of the direction and magnitude of bias that quantitative bias analysis is meant to guard against. Unfortunately, this situation is difficult to avoid because there is rarely sufficient information available to determine whether nondifferential or differential misclassification is most likely for a given study design and method of exposure measurement. For differential misclassification, the magnitude of the difference in sensitivity or specificity between cases and non-cases is typically unknown. Even if a certain misclassification process is strongly suspected (for example, assuming nondifferential misclassification in a prospective cohort study in which exposure is measured before disease occurs), there is no guarantee that this type of misclassification actually occurred in the study.25 By chance, sensitivity and specificity could have differed between cases and non-cases, producing differential misclassification instead of nondifferential misclassification, or vice versa.26 Factors aside from chance are also important. For example, when exposure categories are combined, differential misclassification can be produced even if the measurement error or misclassification process on the original variable was nondifferential.16,25,27 When presenting results, investigators should clearly state that the results of their analysis are valid only if their assumptions were correct. Providing evidence or a rationale to support choice of assumptions would assist the reader in evaluating the likelihood that a correct assumption was made, although it will not be possible to know this with certainty.

An important role for bias analysis in epidemiologic studies is producing ranges of plausible estimates rather than providing a single bias-adjusted effect estimate as the final result. Without knowing whether misclassification was truly differential or nondifferential in our NHANES example, we would have no evidence for choosing the results of one assumption over the others as the most likely. However, we might conclude with some confidence that exposure misclassification does not account for the observed association, given that none of the 95% SIs contained the null value (POR = 1) under any assumptions. In addition, all of the results suggested that exposure misclassification produced a bias towards the null (i.e., the true POR was larger than the misclassified POR), which is valuable information to have even if the exact magnitude of the POR is uncertain. However, it should be noted that we tested only a small number of the possible combinations of sensitivity and specificity distributions and misclassification assumptions (e.g., we did not include scenarios in which sensitivity was higher for cases but specificity was higher for non-cases). Although our analyses correctly predicted that bias was truly toward the null, it is possible that other assumptions would not have produced the same results.

In this study, we presented examples demonstrating that making inaccurate assumptions about nondifferential or differential misclassification has the potential to produce biased results when adjusting for exposure misclassification. In our examples, using an incorrect assumption created more bias than using the unadjusted estimates, highlighting the fact that results from bias analyses do not necessarily represent an improvement over unadjusted estimates, but simply provide a range of plausible estimates under various assumptions. Investigators should recognize the possibility of making incorrect assumptions during adjustment for bias and consider reporting results based on various assumptions about misclassification (i.e., allowing sensitivity and specificity to be the same and different between cases and non-cases), recognizing that the type of misclassification occurring in the analysis is unknown and the choice of assumptions can affect results of the bias analysis. Although this strategy might not provide a single point estimate as the result, bias analysis remains a useful method for providing plausible ranges of the effect estimate in the absence of information on exposure misclassification.

Supplementary MaterialLashTLFinkAKSemi-automated sensitivity analysis to assess systematic errors in observational dataEpidemiology200314445145812843771LashTLFoxMPFinkAKApplying Quantitative Bias Analysis to Epidemiologic DataNew YorkSpringer2009GreenlandSLashTLBias analysisRothmanKJGreenlandSLashTLModern Epidemiology3PhiladelphiaLippincott Williams & Wilkins2008PhillipsCVQuantifying and reporting uncertainty from systematic errorsEpidemiology20031445946612843772SteenlandKGreenlandSMonte Carlo sensitivity analysis and Bayesian analysis of smoking as an unmeasured confounder in a study of silica and lung cancerAm J Epidemiol200416038439215286024GreenlandSMultiple-bias modelling for analysis of observational data (with discussion)J R Stat Soc A2005168267306MaldonadoGAdjusting a relative-risk estimate for study imperfectionsJ Epidemiol Community Health20086265566318559450FoxMPCreating a demand for bias analysis in epidemiologic researchJ Epidemiol Community Health2009639119141660FoxMPLashTLGreenlandSA method to automate probabilistic sensitivity analyses of misclassified binary variablesInt J Epidemiol2005341370137616172102JurekAMMaldonadoGSpectorLGRossJAPericonceptional maternal vitamin supplementation and childhood leukaemia: an uncertainty analysisJ Epidemiol Community Health20096316817218977808JurekAMLashTLMaldonadoGSpecifying exposure classification parameters for sensitivity analysis: family breast cancer historyClin Epidemiol2009110911720865092MacLehoseRFOlshanAFHerringAHHoneinMAShawGMRomittiPANational Birth Defects Prevention StudyBayesian methods for correcting misclassification: an example from birth defects epidemiologyEpidemiology200920273519234399BodnarLMSiega-RizAMSimhanHNDieselJCAbramsBThe impact of exposure misclassification on associations between prepregnancy BMI and adverse pregnancy outcomesObesity (Silver Spring)2010182184219020168307Stott-MillerMHeikeCLKratzMStarrJRIncreased risk of orofacial clefts associated with maternal obesity: case-control study and Monte Carlo-based bias analysisPaediatr Perinat Epidemiol20102450251220670231OrsiniNBelloccoRBottaiMWolkAGreenlandSA tool for deterministic and probabilistic sensitivity analysis of epidemiologic studiesThe Stata Journal200882948FlegalKMKeylPMNietoFJDifferential misclassification arising from nondifferential errors in exposure measurementAm J Epidemiol1991134123312441746532GustafsonPMeasurement Error and Misclassification in Statistics and Epidemiology: Impacts and Bayesian AdjustmentsBoca Raton, FLChapman & Hall/CRC2004Centers for Disease Control and Prevention (CDC), National Center for Health Statistics (NCHS)National Health and Nutrition Examination Survey DataHyattsville, MDU.S. Department of Health and Human Serves, Centers for Disease Control and Prevention1999–2008KrulAJDaanenHAChoiHSelf-reported and measured weight, height and body mass index (BMI) in Italy, the Netherlands and North AmericaEur J Public Health201121441441920089678JohnsonWDBouchardCNewtonRLJrRyanDHKatzmarzykPTEthnic differences in self-reported and measured obesityObesity200917357157719238143Brunner HuberLRValidity of self-reported height and weight in women of reproductive ageMatern Child Health200711137144HussainSTaylorMWaltermaurerEMcCauleyJFordDECampbellJCMcNuttLAComputer-administered screening of reproductive-aged women for diabetes risk in primary care settings, feasibility and acceptability of such screening, and validity of risk assessments based on self-reported weightPreventing Chronic Disease200743A5417572958Nieto-GarciaFJBushTLKeylPMBody mass definitions of obesity: Sensitivity and specificity using self-reported weight and heightEpidemiology1990121461522073502JurekAMMaldonadoGGreenlandSChurchTRExposure-measurement error is frequently ignored when interpreting epidemiologic study resultsEur J Epidemiol20062187187617186399WacholderSDosemeciMLubinJHBlind assignment of exposure does not always prevent differential misclassificationAm J Epidemiol19911344334371877603JurekAMGreenlandSMaldonadoGChurchTRProper interpretation of non-differential misclassification effects: expectations vs observationsInt J Epidemiol20053468068715802377RothmanKJGreenlandSLashTLValidity in epidemiologic studiesRothmanKJGreenlandSLashTLModern Epidemiology3PhiladephiaLippincott Williams & Wilkins2008

Ratio of odds ratios (ROR) and 95% simulation intervals after adjusting for exposure misclassification under one of four assumptions about exposure misclassification in a study of obesity and diabetes in the National Health and Nutrition Examination Survey. The solid horizontal line indicates no bias (ROR = 1.00) and the dotted horizontal line indicates the result when using the misclassified (unadjusted) values in the analysis (ROR = 0.91).

Ratio of odds ratios (ROR) and 95% simulation intervals after adjusting for exposure misclassification in a study of obesity and diabetes in the National Health and Nutrition Examination Survey, using estimates of sensitivity and specificity from 5 studies, under the correct assumption of differential A misclassification. The solid horizontal line indicates no bias (ROR = 1.00) and the dotted horizontal line indicates the result when using the misclassified (unadjusted) values in the analysis (ROR = 0.91).

Contingency Tables for 2 Examples of True Associations Between Exposure and Disease.

DiseaseTrue Odds Ratio = 1.00 ExposureTrue Odds Ratio = 1.29 Exposure
YesNoYesNo
Yes100900125875
No9008,1009008,100

Relationships Between True (Correctly Classified) and Observed (Misclassified) Exposures.

DiseaseTrue ExposureMisclassified Exposure
YesNoYesNo
YesABa = Se1A + (1-Sp1)Bb = (1-Se1)A + Sp1B
NoCDc = Se0C + (1-Sp0)Dd = (1-Se0)C + Sp0D

Abbreviations: Sei, sensitivity in cases (i = 1) and non-cases (i = 0); Spi, specificity in cases (i = 1) and non-cases (i = 0).

Ratio of Misclassification-Adjusted Odds Ratio to True Odds Ratio over 10,000 Simulations when Making Correct and Incorrect Assumptions about Nondifferential and Differential Misclassification.

SimulationTrutha,bAdjustment AssumptioncTrue OR = 1.00True OR = 1.29
Median Ratio of Odds Ratio (95% Simulation Interval)d,e
1Exactly nondifferentialExactly nondifferentialf1.00 (1.00–1.00)1.00 (0.93–1.19)
2Approximately nondifferential1.01 (0.46–2.19)1.01 (0.52–2.13)
3Differential A1.54 (0.89–3.16)1.41 (0.84–2.90)
4Differential B0.43 (0.03–1.22)0.53 (0.11–1.31)
5Approximately nondifferentialExactly nondifferential1.30 (1.21–1.58)1.24 (1.10–1.66)
6Approximately nondifferentialf1.31 (0.68–2.82)1.25 (0.69–2.62)
7Differential A1.84 (1.09–3.83)1.65 (1.00–3.41)
8Differential B0.70 (0.14–1.74)0.77 (0.30–1.77)
9Differential AExactly nondifferential0.56 (0.19–0.69)0.65 (0.54–0.68)
10Approximately nondifferential0.56 (0.10–1.39)0.65 (0.25–1.45)
11Differential Af1.01 (0.49–2.14)1.00 (0.55–2.07)
12Differential B0.18 (0.02–0.65)0.24 (0.02–0.77)
13Differential BExactly nondifferential1.95 (1.67–2.77)1.77 (1.48–2.62)
14Approximately nondifferential1.97 (1.16–3.98)1.78 (1.08–3.58)
15Differential A2.46 (1.53–4.94)2.15 (1.36–4.31)
16Differential Bf1.01 (0.40–2.23)1.00 (0.49–2.14)

Abbreviations: OR, odds ratio.

Nondifferential misclassifications: sensitivity for cases (Secase) = specificity for cases (Spcase) = sensitivity for controls (Senon-case) = specificity for non-cases (Spnon-case) = 0.90. Differential A: Secase = Spcase = 0.95, Senon-case = Spnon-case = 0.90. Differential B: Secase = Spcase = 0.90, Senon-case = Spnon-case = 0.95.

Misclassified ORs for true OR = 1: exactly nondifferential = 1.00, approximately nondifferential = 1.14, differential A = 0.75, differential B = 1.42. Misclassified ORs for true OR = 1.29: exactly nondifferential = 1.12, approximately nondifferential 1.28, differential A = 0.88, differential B = 1.61.

Exactly nondifferential: Se and Sp for cases and non-cases calculated from total population and are the same. Approximately nondifferential: Se and Sp for cases and non-cases drawn from the same distribution but do not necessarily have the same values; the modes of the distributions are the actual values of Se and Sp calculated from the total population. Differential A and B: Se and Sp for non-cases calculated directly from non-cases; for the correct assumption, Se and Sp for cases are the true values for cases; for incorrect assumptions, the modes of the Se and Sp distributions for cases are 0.05 higher (differential A) or lower (diferential B) than non-cases.

Lower and upper bounds are the 2.5th and 97.5th percentiles of the ratio of odds ratio distribution generated by simulation.

Percentage of simulations generating negative ORs: simulation 4 = 8% for OR = 1, <0.1% for OR = 1.29; simulation 8 = <0.1% for OR = 1; simulation 9 = 0.2% for OR = 1; simulation 10 = 0.2% for OR = 1; simulation 12 = 56% for OR = 1, 20% for OR = 1.29.

Correct assumption.

Contingency Tables and True Values of Sensitivity and Specificity of Obesity Classification for a Study of Self-Reported and Measured Obesity and Diabetes, Non-Pregnant Women Aged 18–49, National Health and Nutrition Examination Survey, 1999–2010.

Self-Reported ObesityaMeasured ObesityaSensitivitya,bSpecificitya,c
ObeseNot ObeseObeseNot Obese
All participants Diabetesd2,4375,6862,7605,3632,318/2,7600.835,244/5, 3630.98
 Yes1978921373195/2130.9271/730.97
 No2,2405,5972,5475,2902,123/2,5470.835,173/5, 2900.98

Counts and proportions are unweighted.

Proportion of individuals truly exposed who reported exposure.

Proportion of individuals truly unexposed who reported not being exposed.

Diabetes was self-reported by participants.