Substance use causes avoidable illness and death, including from cancer (1). Smoking tobacco causes lung, liver, and colorectal cancers, among others (2). Moderate to heavy alcohol consumption is associated with oropharyngeal, colorectal, and pancreatic cancers (3). Emerging evidence suggests a positive association between marijuana use and prostate and cervical cancer (4). Despite these risks, substance use is common: 60 million Americans smoke, 14 million are alcohol-dependent, and 14 million use illicit drugs (including marijuana) (1).

Reducing substance use among adolescents is particularly important for preventing cancer. First, lifelong substance use often begins in adolescence (5,6). For example, 88% of adult daily smokers began smoking before age 18 (7). Second, adolescence is a vulnerable period when people are particularly sensitive to substance use (2). Understanding adolescent substance use is therefore crucial to reducing the risk of related cancers.

Monitoring adolescent substance use, however, is challenging. Some adolescents may underreport use because of social desirability or fear of legal consequences (8) and others may overreport use to earn social cache from their peers (8). Studies comparing self-report to biometric measures of substance use have indicated that self-report measures have fair validity, with some adolescents underreporting and some overreporting use (8,9). However, quantifying the degree of uncertainty in estimates of adolescent substance use in surveillance surveys is important for leveraging these estimates for research and intervention purposes. Given that some public health efforts attempt to target adolescents in high-risk geographic areas, the ability to reliably identify which states have the highest or lowest prevalence of substance use may be called into question for several reasons: different surveillance surveys may identify different states; states likely rank differently across behaviors; and statistical uncertainty may undermine strong conclusions about differences across states. Understanding the extent of this problem has implications for surveillance research, specifically for cancer prevention.

We compared and ranked state estimates of past-month cigarette smoking, binge alcohol drinking, and marijuana use among adolescents from 2 population-based surveys: the National Survey of Drug Use and Health (NSDUH) and the Youth Risk Behavior Surveillance System (YRBSS). Both surveys collect substance use data for youths using cross-sectional, multistage probability sampling design, but each survey has its own strengths (10). A major strength of NSDUH is its ability to support estimates for all 50 states and the District of Columbia, and the data are collected every year. A major strength of YRBSS is its large state-level sample size; however, the survey is conducted every other year and not every state participates in the study or achieves adequate response rates. Comparison of the 2 surveys would highlight the consistencies or differences in rankings across different surveys. Calculation of confidence intervals (CIs) around ranks, often overlooked in ranking studies (11,12), allowed us to statistically evaluate consistencies and differences in state ranks within and across surveys and behaviors, and explore implications for cancer prevention.

In this analysis, we demonstrated differences in states’ simulated ranks for adolescent substance use across behaviors (ie, cigarette smoking, binge alcohol drinking, marijuana use) and surveys (ie, NSDUH, YRBSS). These findings highlight the variability that emerges when ranking states on behavioral indicators, partly due to differences in behaviors, variation in surveys, and the inherent uncertainty in the statistical ranking processes.

States ranked high on one adolescent substance use behavior did not necessarily rank high on another behavior, reflecting the distinct patterns and correlates of these behaviors. Correlation coefficients between states’ ranks of different behaviors ranged from −0.17 to 0.29 for NSDUH and from −0.20 to 0.55 for YRBSS. Partly, these differences could be attributable to demographic differences across states. For example, adolescent cigarette smoking and alcohol use are inversely associated with parental socioeconomic status (SES), whereas marijuana use has an inverse U-shaped association with SES; further, these effects are moderated by race/ethnicity (18). A more contextual explanation might focus on state policies that influence these behaviors. States’ policies regulating cigarettes include indoor smoke-free air laws, tobacco taxes, and minimum age purchase of cigarettes, all of which could influence adolescent smoking prevalence (7,19,20). Similarly, emerging policies allowing marijuana use (among adults) may be associated with increased adolescent marijuana use (21). Descriptively, in the current analysis, states that have legalized marijuana tended to have higher estimates of adolescent marijuana use. State-level differences in adolescent substance use could also be related to local norms (22,23) or exposure to mass media campaigns discouraging substance use (24).

Across surveys, states’ estimates and ranks for adolescent substance use were not consistent. First, states’ substance use estimates were up to 2.6 times as high in YRBSS as in NSDUH. In addition, the median substance use estimates in NSDUH were 9.3% to 9.5%, whereas the median estimates in YRBSS were 12.5% to 18.9%, indicating greater variability in substance use in YRBSS than NSDUH. Differences in survey administration mode, sampling frame, and item wording could explain this variability in prevalence estimates (25). In terms of administration, NSDUH is delivered in homes (15) and YRBSS in schools (16); for both surveys, figures of authority may be nearby while the survey is administered (parents or teachers, respectively), which could lead to underreporting (8), but adolescents may overreport substance use in YRBSS to impress their peers. However, one recent study of adolescent self-report of substance use found that survey setting did not introduce bias to responses (26). In terms of sampling frame, NSDUH includes (and YRBSS excludes) adolescents who drop out of school, who in turn have higher levels of substance use (25). In terms of item wording, slight differences could account for some differences in estimates; for example, the cigarette smoking item in NSDUH was “During the past 30 days, have you smoked part or all of a cigarette?” (13) whereas the item in YRBSS was “During the past 30 days, on how many days did you smoke cigarettes?” (14). However, efforts to obtain substance use estimates without using self-report on surveys (eg, through biometric tests) would be burdensome to collect in a population-based survey, and a previous study demonstrated construct validity of self-reported adolescent substance use estimates (27). Further, wide differences in states’ estimates of adolescent substance use translated into wide differences in states’ ranks across surveys. When comparing ranks in NSDUH versus YRBSS, the correlations were 0.64 for cigarette smoking, 0.36 for binge alcohol drinking, and 0.69 for marijuana use. The overall inconsistency is depicted in the YRBSS panels of Figures 1, 2, and 3, which are organized by NSDUH rank and appear fairly scattered. These findings indicate that the biases inflating YRBSS estimates compared with NSDUH estimates do not operate similarly across states; if they did, we would expect to see similar patterns of ranks across surveys. Overall, the discrepancies between estimates and ranks across surveys suggest that biases in data collection exist, which could have important implications for policy setting and intervention development.

Finally, the statistical process of ranking states while accounting for variability in the underlying estimates of adolescent substance use revealed a lack of precision. The ranks’ 95% CIs were wide, especially for states in the middle of each distribution, where CIs spanned 20 or more ranks. Although the first- and last-ranked states were statistically distinguishable, many of the states between the extremes had overlapping CIs. The research literature on ranking states (or other geographic jurisdictions, such as counties) has been engaged in a debate as to whether ranks are “good enough” to present without measures of error (11,28–30), but our findings suggest such a practice overstates the precision of ranks. Presenting ranks without measures of error may be more digestible to the public and to policy makers, but it could lead to overinterpretation of the ranks and misappropriation of public health funding.

Despite these differences, some consistent findings support conclusions about substance use among adolescents across states. Utah consistently ranked last across behaviors and surveys for adolescent substance use, reflecting the relatively low proportions of adolescents engaging in these behaviors. This finding could be related to Utah’s high levels of religiosity (31) — a construct that has been linked to lower prevalence of adolescent substance use (19,32) — and restrictions on the sale and use of the substances under study (21,33). To the extent possible, adapting programs and policies from Utah (or other well-performing states) for adoption in other jurisdictions could reduce the prevalence of adolescent substance use. In addition, we found that correlations of states’ ranks for their overall population with the ranks derived for boys or girls only (0.66–0.96) were similar. Thus, state processes affecting self-reported adolescent substance use appear to do so for boys and girls similarly.

Taken together, these findings have implications for cancer prevention. Given that adolescent substance use prevalence estimates and rankings varied across surveys and behaviors, identifying “high-need” states for additional research or interventions is difficult. That is, a state that ranked poorly for cigarette smoking on one survey did not necessarily rank poorly on the other survey, and it did not necessarily rank poorly for other outcomes. Thus, selecting high-need states for behavior-specific research depends on the survey used, and selecting high-need states for research on multiple substance use behaviors requires care. However, calculating 95% CIs around ranks affords some flexibility because it allows researchers not only to recognize the uncertainty in rankings but also to identify states that have moderate ranks whose CIs include the poorest ranks. For example, Wyoming is ranked 1 and 4 for cigarette smoking and 12 and 5 for binge alcohol drinking in NSDUH and YRBSS, respectively, but its 95% CIs all include at least rank 2, indicating that it is in the top 5% of states in terms of adolescent use of these 2 substances (a pattern that would not have been immediately discernible without 95% CIs). From the perspective of states that performed well on all indicators, Utah was ranked last (indicating low adolescent substance use prevalence) on all behaviors, which could offer some clues for cancer prevention activities. Overall, however, these findings underscore the need for additional research in at least 2 areas: 1) improving surveillance of cancer prevention behaviors, especially as prevalence estimates appear to be sensitive to survey mode, and uncertainty in the state rankings was evident; and 2) hypothesis generation and testing for state characteristics, programs, and policies that can discourage adolescent substance use for the purpose of lifelong cancer prevention.

This study has several limitations. NSDUH and YRBSS both used self-reported measures of substance use, which are subject to biases (8). Not all states participated in YRBSS (16), restricting the range of ranks and the scope of our inferences. However, NSDUH collected data for all states, allowing us to examine the entire United States. In 2015, NSDUH changed their definition of binge drinking for adolescent girls (15), so the estimates of binge drinking included data only from 2011–2014. Finally, we did not examine the emerging use of other substances that might be related to cancer risk among adolescents, such as e-cigarettes (34). In terms of study strengths, our analysis leveraged data from more than half a million participants in 2 nationally representative surveys using different administration modes. This generous sample size allowed us to produce stable state-level estimates of adolescent substance use. Finally, our research points to the need for improved methodology to rank and compare states’ performance on public health indicators.

Adolescent substance use that contributes to cancer risk is relatively common (4.5% to 27.1% across surveys and behaviors). In 2 population-based surveys, we found some consistency in performance for selected states and across subgroups. However, great variability emerged in states’ rankings, potentially due to differences in behaviors, survey methods, and statistical procedures. Generally, we could not distinguish among states’ performance on adolescent substance use with certainty. Yet public health officials may be able to adopt policies and programs in states that had low estimates of substance use (eg, Utah) to reduce adolescent cigarette smoking, binge alcohol drinking, and marijuana use elsewhere. Such a goal is important for reducing morbidity and mortality among adolescents now (eg, from vehicular crashes when the driver is under the influence of alcohol) and as they grow older (eg, from cancers associated with cigarette, alcohol, and marijuana use).