Public health surveillance of key maternal health indicators is critical for monitoring and assessing the effectiveness of policy and programmatic efforts for maternal and infant health. The Pregnancy Risk Assessment Monitoring System (PRAMS) collects state-specific, population-based information on maternal experiences before, during, and shortly after pregnancy and is administered to women 2–6 months after delivery. As of 2014, PRAMS is implemented in 40 states and one city, representing 78 % of all live births in the United States (US) (http://www.cdc.gov/prams/). PRAMS data are widely used by states to identify women and infants at high risk for health problems and to measure progress in improving health.

The PRAMS survey includes, among many indicators, questions that address medical conditions experienced before or during pregnancy, health care utilization and infant health measures. Although several studies in the US have evaluated mothers’ recall of medical conditions occurring before or during pregnancy [1–11]; none have done so with PRAMS data and most evaluated a mother’s recall many years after the pregnancy. Studies focused on recall of infant birth weight and gestational age have found high agreement for birth weight [2, 3, 5, 7, 8, 11, 12] but low agreement for gestational age and preterm delivery [4, 5, 7, 8, 11]. A systematic review of studies validating gestational hypertension found mostly low sensitivity ranging from 31 to 65 % in 3 US-based samples; however, one case–control study estimated a sensitivity of 90 % among controls [1]. Two studies have examined other complications during pregnancy [9, 11]. One of the studies conducted in three Minnesota hospitals among 1,200 women 10–15 years after delivery found low levels of positive agreement for rare complications such as gestational diabetes (GDM), placental abruption, preeclampsia, and placenta previa and moderate agreement for preterm labor [11]. The majority of previous studies are limited by samples that are not population-based and that are outdated [2, 3, 5, 7, 8, 11, 12].

The primary purpose of this analysis was to evaluate the validity of selected self-reported indicators on the PRAMS questionnaire in two different PRAMS sites, one urban with a diverse racial/ethnic population, the other rural with a primarily non-Hispanic white population.

The women included in the NYC and Vermont samples had similar age distributions, with about 70 % aged 25 and older, but differed on all other demographic characteristics (p < 0.05) (Table 1). Women from NYC were more likely than women from Vermont to be non-white or non-Hispanic, to have been enrolled in WIC, to live below 100 % of the FPL, to be uninsured before pregnancy, to be unmarried, and to have less than a high school education.

This study was conducted one of the first evaluations of mothers’ self-report of pregnancy history, complications during pregnancy, health care utilization, and infant indicators on the PRAMS questionnaire. In our rating system, most indicators fell in the moderate range with only two indicators performing poorly overall in both sites. In addition, while many indicators had excellent to moderate specificity, measures of sensitivity and PPV varied with PPV having the largest number of poor ratings. Many of the indicators have a low prevalence, which is more likely to result in having a poor PPV. PPV is lower when there are greater numbers of false positives, which is more likely when the indicator is rare.

Among indicators with overall excellent ratings in one or both sites, prior live birth and breast-feeding have been previously evaluated, and our findings are consistent with the literature [2, 14]. Admissions to the NICU and Medicaid insurance at delivery have not been previously evaluated. Having valid data for Medicaid is especially valuable, given the likely increases in Medicaid coverage in the years ahead.

Our finding that placenta previa and/or placental abruption and urinary tract infection or kidney infection had poor overall ratings in both sites is consistent with the literature [11]. Possible explanations for these results include the wording of the questions and women’s medical knowledge. The PRAMS question asks about “problems with your placenta (such as abruptio placentae or placenta previa).” The question may need to be more specific, provide more explanation of the condition, and ask about each condition separately. In addition, women may have difficulty reporting placenta previa or abruption because they may never have been told they have the condition or clinicians may have used other terminology to explain the condition. Urinary tract infections may not have been reported with high validity because women may fail to remember an infection during pregnancy when asked 2–6 months after delivery, especially one that had minor symptoms. Other sources, such as hospital discharge data, may be better for surveillance of these medical conditions.

Indicators with overall moderate ratings require more thought as to whether the validity is high enough for specific research or surveillance purposes. Two complications, GDM and hypertension, had moderate sensitivity but poor PPV, consistent with most [9, 11] but not all [1] of the literature. PPV was low due to women over-reporting these two conditions. A possible reason for women over-reporting GDM is that diagnosis requires a positive result on two different glucose tests. Women who screened positive on the first test and negative on the second may have misreported that they had GDM. A similar situation may have resulted in the over-reporting of gestational hypertension, as diagnosis requires high blood pressure at two separate visits. Preterm labor had poor PPV in both sites and poor sensitivity in NYC but moderate sensitivity in Vermont. It is common for women to report and even be evaluated for contractions prior to term and yet not have true preterm labor, suggesting it is difficult to collect valid responses from mothers on this condition. Another complication, PPROM, had moderate sensitivity and poor PPV in NYC and excellent sensitivity and moderate PPV in Vermont, suggesting that the validity of mothers’ report of this item may vary by population characteristics.

One surprising result is that mothers in NYC did not report valid information on whether their previous infant was born with LBW. Previous studies consistently have found mothers’ reporting of infant birth weight to be highly correlated with birth weight reported in the medical record when mothers were asked to recall the infant’s birth weight or when asked to recall if the infant was < 2,500 g [2, 3, 5, 7, 8, 11, 12]. In Vermont, this indicator had excellent sensitivity and an overall excellent rating. NYC’s poor overall validity on this indicator suggests that demographic differences may have led to these quality differences in the indicators. Other studies of mother’s recall of a preterm delivery have documented low-to-moderate sensitivity [4, 5, 7, 8, 11], which is consistent with our findings.

Receipt of an HIV test had an overall moderate rating in NYC but an overall poor rating in Vermont. Differences in HIV testing policies may be one explanation for this. A woman’s ability to accurately report receiving an HIV test will be partly dependent upon informed consent procedures. In 2009, Vermont had an “opt-out” HIV testing policy. Women were informed, either in writing or orally, that they would receive an HIV test and that they had an option to decline it. In NYC, doctors were required to give HIV counseling to all pregnant women, and women were required to give written consent to receive an HIV test. In addition, New York has a policy to test all newborns whose mother was not tested during pregnancy, with no maternal consent required, which may affect maternal acceptance of prenatal testing. Thus, this more involved consent process in NYC may have improved women’s reporting. Two previous studies assessed the validity of self-reported HIV testing. One study compared postpartum reporting to medical record and laboratory data in three hospitals in Toronto [15], the other compared self-report of HIV testing history to laboratory data among pregnant women at one hospital [16]. Unlike our study, which found women under-reported HIV testing, both of these studies found women over-reported receiving an HIV test compared to laboratory data.

This study has several noteworthy strengths. It is the first US population-based validity study we are aware of focused on self-reported measures of pregnancy complications and infant outcomes. The inclusion of population-based samples from two diverse populations provided insight on the robustness of findings across varied groups of women. One limitation of this validation study is the possibility that some complications, test results, and maternal behaviors may not have been recorded in the medical record, leading to biased estimates. This limitation is especially relevant for “received an HIV test”, as well as for breastfeeding in the hospital. Evaluations are needed to assess whether these results can be replicated at other PRAMS sites.

Our findings have implications for PRAMS as well as for researchers collecting self-reported information on pregnancy histories. For PRAMS, the findings have contributed to the decision to remove some questions from the core questionnaire including those on placenta previa/placental abruption, preterm labor, urinary tract infection/kidney infection, and received an HIV test. In addition, PRAMS is considering revising questions such as the one on gestational diabetes. For researchers who plan to collect self-reported pregnancy histories, the results provide direction regarding topics that are difficult for women to report valid information, such as placenta previa, and others that are easier, such as Medicaid insurance at delivery and infant was admitted into the NICU. Indicators will need to be considered carefully and researchers need to consider the importance of each validity metric, sensitivity, specificity and PPV. Understanding the strengths and limitations of self-reported indicators on the PRAMS questionnaire will help maternal and child health professionals use PRAMS data appropriately and inform the research efforts of those relying on self-reported information.