The National Birth Defects Prevention Study (NBDPS) is a multi-state, population-based case-control study of major structural birth defects in the United States between 1997 and 2011. Study methods have been described in detail. ²⁷ Briefly, birth defect cases were identified using active surveillance systems in 10 states (Arkansas, California, Georgia, Iowa, Massachusetts, North Carolina, New Jersey, New York, Texas, and Utah). Cases with a known cause (i.e., single gene or chromosomal disorders) were ineligible. Medical records of cases were reviewed for defect confirmation and all cases were classified by clinical geneticists using standard criteria. ²⁸ Eligible cases could be live births, elective terminations (any gestational age), or stillbirths (gestational age at delivery ≥20 weeks’ gestation or birth weight ≥500g if gestational age unknown). Controls were live born infants without major structural or syndromic birth defects randomly selected from the same geographical location and time period as the cases through either birth certificates or birth hospital records.

Mothers of all eligible cases and controls were invited to participate in a telephone interview within 24 months of their estimated date of delivery (“due date”) about demographic, reproductive factors, pregnancy history, health behaviors, and lifestyle characteristics. Mothers of both cases and controls were ineligible to participate in the interview if the infant was not in the mother’s legal custody, the mother was deceased or incarcerated, did not speak English or Spanish, or had already participated in the NBDPS for a previous pregnancy.

Approval for this study was granted by the Institutional Review Board of the Centers for Disease Control and Prevention and Institutional Review Boards at all study sites. Informed consent was obtained from each participant at the time of the interview.

We investigated two primary questions: (i) Do the results of analyses meaningfully differ when conducted among live births only as compared to when stillbirths and terminations are also included? and (ii) Under what conditions do analyses differ meaningfully?

Analyzing all of the large number of specific birth defects and exposures included in the NBDPS for each of these questions was not practical. Therefore, we selected representative birth defects and exposures to evaluate our questions. The strength of selection bias is dependent on how strongly the exposure and birth defect of interest each affect live birth. ²⁹ Therefore, we selected exposures and birth defects which represent the range of strengths of association with survival to live birth. Our process for selecting these is detailed below.

To quantify how the strength of association between birth defects and survival to live birth affects selection bias, we selected four birth defects which represent the full spectrum of survival to live birth: anencephaly, low survival (high stillbirth and termination incidence); spina bifida moderate survival (moderate stillbirth and high termination incidence); omphalocele moderate survival (high stillbirth and moderate termination incidence); cleft palate without cleft lip high survival (low stillbirth and termination incidence).³⁰

To quantify how the strength of the association between exposure and survival to live birth affects selection bias, we selected three established risk factors for birth defects to represent exposures with a range of effects on survival to live birth in cases and controls: Smoking has a moderate association with oral clefts and no known association with termination for birth defects but a moderately increased risk of stillbirth. ^31–36 Antiepileptic drugs (AEDs) are strongly associated with anencephaly and spina bifida and may be moderately associated with termination for birth defects due to increased surveillance for birth defects among AED users; there is no known effect on stillbirth.^37–40 Multifetal pregnancies have a moderate association with anencephaly and omphalocele, and a strong inverse association with termination for birth defects as termination could negatively affect the survival of normally formed co-siblings. ^41,42 However, compared to singleton pregnancies, multifetal pregnancies have a higher risk of stillbirth. ⁴³

From the maternal interview, we obtained information about first trimester maternal smoking (any smoking vs no smoking), first trimester use of any AED (any time in the first trimester vs no use during the three months before and throughout pregnancy), and multifetal pregnancy (multiple vs singleton). AEDs were defined as any medication containing clonazepam, divalproex sodium, gabapentin, oxcarbazepine, phenytoin, phenobarbital, primidone, diazepam, topiramate, levetiracetam, lamotrigine, valproic acid, or carbamazepine. Multifetal pregnancies were based on maternal interview or if a response was missing from the interview, information abstracted from medical or vital records. If more than one infant from a multiple set had eligible birth defects, the oldest eligible infant was included in the study (n=1 cleft palate, both twins had the same defect and were live born).

We used Firth’s penalized logistic regression models to estimate odds ratios (OR) and 95% profile-likelihood confidence intervals (95% CIs).^44,45

We quantified the strength of the relationship between 1. birth defect and live birth by estimating crude ORs for anencephaly, omphalocele, and spina bifida compared to cleft palate (since all controls were live born, we selected the defect with highest survival for comparison) and 2. exposures and live birth by estimating crude ORs for each birth defect-exposure pair.

Selection on survival was introduced by restricting analyses to the following case groups: “live births only”, “live births and stillbirths”, and “live births, stillbirths, and terminations”. We adjusted logistic models for the following a priori selected covariates: maternal age category (<25, 25–34, ≥35 years), maternal race/ethnicity (white non-Hispanic, Black non-Hispanic, Hispanic, Asian/Pacific Islander, Other), and use of a vitamin containing folic acid any time in the first trimester of pregnancy. Covariates were selected based on known associations with the exposures and birth defects as well as a sufficiently high prevalence in the population to provide confounding adjustment given small numbers of exposed birth defects cases. Given that small numbers can lead to substantial random variation, model results were considered to differ if the 95% CI of models among live births, stillbirths, and terminations excluded the OR when restricted to live births only, or to live births and stillbirths.

Mothers missing data on smoking (n=300 control mothers, 92 case mothers) or AED use (n=238 control mothers, 81 case mothers) were excluded from those respective analyses. Cases with an unknown pregnancy outcome (n=10) were excluded from all analyses.

We evaluated the potential effect of clustering by study center using logistic mixed effect models with a random intercept for study center. To simplify analyses, we assumed no misclassification of exposure or outcome and no unmeasured confounding, although these potential sources of bias may be present. ³⁵

During the study, the inclusion of cases by survival outcome differed by center and over time; some centers included only livebirths and others expanded to non-livebirths later in the study period. ²⁷ The main analyses used data from all centers, but we conducted a sensitivity analysis excluding centers and time periods where stillborn infants and terminations were not collected (“Complete Collection Sample”).

Differences in the proportion of birth defect cases terminated among certain sub-populations may affect the degree of selection bias observed. Therefore, we repeated analyses while restricting to the following populations: 1. singleton pregnancies (increased risk of termination for birth defects) and 2. cases with isolated birth defects (decreased risk of termination for birth defects); analyses of multifetal pregnancy were excluded from this sensitivity analysis.⁴⁶ Finally, since interview participation may differ by survival outcome, we compared survival by interview status for each birth defect.

Survival to live birth (Table 1), was lowest for anencephaly (live birth 33%) and highest for cleft palate (live birth 99%). Compared to the odds of live birth for cases with cleft palate, the odds of live birth were substantially lower for anencephaly (OR 0.007, 95% CI 0.004, 0.01), omphalocele (OR 0.09, 95% CI 0.06, 0.15), and spina bifida (OR 0.10, 95% CI 0.06, 0.15).

AEDs were not associated with odds of live birth for any defect (Table 2), but smoking was moderately associated with decreased odds of live birth among anencephaly cases (OR 0.61, 95% CI: 0.34, 1.04) and increased odds among omphalocele cases (OR 2.22, 95% CI 1.10, 5.34). Multifetal pregnancy was strongly associated with increased odds of live birth among both anencephaly (OR 3.15, 95% CI 1.71,5.92) and omphalocele cases (OR 5.22, 95% CI 1.40, 46.78).

Although exposure prevalence varied across individual survival outcomes, there was little change between the proportion of exposed cases among live births compared to among live births, stillbirths, and terminations for most analyses (Table 1). The exception was for cases of anencphaly where we found notable differences for the exposures of multifetal pregnancy (11.9% among live births vs 6.4% among all survival outcomes)and smoking (8.2% among live births vs 11.6% among all survival outcomes.

Except for analyses of anencephaly, estimated aORs among live births did not differ from those among live births and stillbirths, or among all survival outcomes (Figure 2). The estimated association of smoking with anencephaly among live births was further from the null than the estimate among all survival outcomes, but this analysis did not meet our criterion for difference. The aOR of anencephaly associated with multifetal pregnancy was twice as high when restricted to live births (aOR 4.93, 95% CI 3.16, 7.20) compared to among all survival outcomes (aOR 2.44, 95% CI 1.73, 3.35). The point estimate for live births was outside of the 95% CI among all survival outcomes meeting our criteria for difference.

There was no difference in participation by survival for anencephaly. For the other birth defects, participation among live births was about 5% higher than for stillbirth and termination (eTable 1). Cases with multiple birth defects were more common among pregnancies ending in stillbirth or termination than live birth (eTable 2).

We did not identify a difference in the occurrence or magnitude of the bias across included survival outcomes when analyses were restricted to singletons or cases with isolated birth defects (eTables 3 – 8). Exclusion of study sites and time periods when stillbirths and terminations were not collected (“complete collection sample”) had no meaningful impact on our findings. Results of mixed effect models with a random intercept for study center did not differ meaningfully from fixed effect models, but model convergence could not be achieved for all analyses (eTables 6 – 8).

Details of the quantitative bias analysis are presented in the supplement. Most results were robust to the exclusion of stillborn and terminated controls, even when the exposure odds were 5 to 12 times higher than for liveborn controls (Figure 3; eTables 10 - 12). There was little difference in the results of analyses conducted among all survival outcomes for both cases and controls compared to our main analyses (range of difference in ORs 0.00 to 0.60).

We found limited evidence of meaningful live birth bias in analyses of etiological risk factors for birth defects. As expected from the generic DAG, live birth bias was identified when survival to live birth was strongly associated with both the birth defect of interest and exposure among cases with that birth defect. However, when the exposure distribution among live births was similar to that among all survival outcomes – even when the prior conditions were met – any resulting bias led only to small changes in estimates (e.g., analysis of multifetal pregnancy and omphalocele). When the exposure distributions differed substantially between these populations a large bias was noted, as in the analysis of multifetal pregnancy and anencephaly. In this case, only the strength of this association was attenuated (a two-fold reduction), with no change in the direction of the relationship. However, an inversion of effects is possible.

Examination of selection bias within this large, population-based case-control study and inclusion of a quantitative bias analysis allowed us to examine several different scenarios and quantify the bias introduced. Further, we selected exposures with well established relationships to the defects under study, and our results are consistent with previous reports conducted in various settings. ^{34–36,39,41,53} Consistent methods across centers helped to ensure the quality of case ascertainment and classification. Identified controls later found to have major birth defects were included as cases if they met the study criteria or excluded; thus, misclassification of the outcome is expected to be rare. Survival outcome had little effect on interview participation for cases.

Our study has several limitations. Spontaneous abortions before 20 weeks’ gestation were not captured due to both technological and practical limitations. Therefore, we cannot rule out the possibility that exclusion of miscarried cases and controls could lead to meaningful bias; this may explain the protective association found for maternal smoking and anencephaly in our study and others.^16,34,35 Identification of terminated cases by surveillance systems is likely incomplete, but the proportion of missing cases is unknown. ¹⁷ However, our findings and results of our sensitivity analysis suggest that large numbers of terminations need to be missing differentially by exposure status in order to substantially alter our results. Our criterion for defining the presence of bias are stringent and partially dependent on power; thus, we may miss true selection bias. It is possible that the wide confidence interval for the analysis of smoking and anencephaly among live births may have led us to erroneously conclude that there was no selection bias for this exposure and birth defect pairing. However, objective criteria prevent this determination from being simply based on opinion.

Although self-reported medication use is subject to errant recall, antiepileptic drugs have been found to be reported with near-perfect accuracy compared to prenatally reported medication use and dispensing records. ^54,55 Additionally, self-report of smoking during pregnancy correlates well with biomarker-based estimates of smoking, but some exposure misclassification is possible. ^56,57 While recall bias is a concern in retrospective studies of birth defects, bias has been found to manifest only in extreme circumstances. ⁵⁸

Our results suggest that so long as live births are representative of the exposure distribution among all cases which are live born, stillborn, and terminated, live birth bias is unlikely to substantially alter the results of a risk factor analysis for birth defects. Although there were small differences in the exposure distributions for omphalocele cases between live births and all survival outcomes, little difference was seen for spina bifida and cleft palate cases. Consistent with this observation, there was little difference in the ORs estimated for these defects in live births vs all survival outcomes.

As with previous studies, strong selection bias was restricted to analyses of anencephaly. ^16–20,49 Because only 33% of anencephaly cases were live born, this small subset is less likely to be representative of all cases than most birth defects where well over 80% are live born. Therefore, severe live birth bias is most likely to occur among birth defects with a similarly low probability of live birth (e.g., bilateral renal agenesis, trisomy 13). ^30,50 However, the exposure distributions for anencephaly did not always differ between live births and all survival outcomes, suggesting that not all analyses of high mortality birth defects will be affected by live birth bias.

Importantly, however, the above observations can only be said to apply to analyses of fetuses who survive to the point where birth defect diagnosis is possible. Exposure distributions among conceptions miscarried or electively terminated prior to the point of possible birth defect diagnosis (approximately 11 weeks’ gestation) may differ substantially from survivors for whom birth defects are diagnosed or ruled out. ^16,49,51,52 As approximately 20% of pregnancies end in an early loss – a proportion likely higher for more severe birth defects – exclusion of these cases may alter the exposure distribution among all cases. ^11,49 Thus, miscarriages may induce some degree of selection bias which is not accounted for in our study.

We also found that the restriction of NBDPS controls to live births is unlikely to contribute to selection bias in our study or to alter the results of our analyses. Even under extreme assumptions about the strength of the relationship between exposure and stillbirth or second trimester elective termination results were similar. Thus, we believe our main results are robust to the exclusion of electively terminated and stillborn controls.

These results are expected to generalize to studies of administrative data since the NBDPS utilized population-based cumulative incidence sampling to efficiently sample from the underlying cohort. ⁴⁷ NBDPS controls have previously been found to be generally representative of the underlying study populations, except for the prevalence of multifetal pregnancies.⁴⁸ Correcting to the population level in the study base shifted associations with multifetal pregnancy towards the null but did not alter the occurrence of live birth bias. Studies investigating risk factors for chromosomal disorders with high incidence of stillbirth or termination or which investigate associated structural defects in populations including chromosomal disorders risk live birth bias. A low proportion of liveborn cases will result in highly imprecise results and low power. Finally, we cannot be sure that our results apply to all possible exposures. Publication of the associations of various potential risk factors by survival outcome (individually and combined) and birth defect would aid researchers in assessing the potential for live birth bias.