The balance of benefits relative to costs of an intervention or policy can be quantified using either cost-effectiveness analysis (CEA) or cost-benefit analysis (CBA). In CBA health outcomes are expressed in terms of monetary values that are intended to represent the lost societal welfare from premature death or incapacitation whereas in CEA health outcomes are calculated separately from costs. A CEA compares multiple interventions in terms of total outcomes and total costs. An intervention that both costs less and has better outcomes than the next best alternative is said to be cost-saving or dominant, and one that has better outcomes but the additional cost is relatively moderate is considered cost-effective.

It should be noted that cost-effectiveness is not an absolute attribute of an intervention but depends on the comparator. A strategy may be cost-effective in one setting compared with one alternative but not cost-effective in a different context or faced with a different comparator. Economic evaluations are based on the economics principle of the counterfactual, which means that everything, other than the specific intervention being evaluated, is held constant [18].

The first distinction among economic evaluation methods is whether the analysis models health outcomes (reduced deaths or disease) or just numbers of cases detected. Calculating the cost to detect a case tells one nothing about the value of detecting and treating the disease in question and hence is not informative of the balance of costs and outcomes. Partial CEA studies only consider costs and numbers of cases detected, whereas full CEAs calculate both incremental costs and health outcomes [19].

Full CEAs report whether an intervention is either cost-saving, or cost-effective, with an incremental cost-effectiveness ratio (ICER) that is considered favorable. Full CEAs are of two types: those that use “natural” measures of health, such as life-years saved, and those that use summary measures of health such as the quality-adjusted life-year or QALY [20]. CEAs that calculate incremental cost per QALY are also referred to as cost-utility analyses because QALYs are calculated using health utility (or health-related quality of life) scores for health states on a scale of 0 to 1 where 0 represents death and 1 perfect health [21]. Calculating QALYs for pediatric interventions such as NBS is particularly challenging [22–24].

Whether a given ICER is considered to provide good value depends on the decision maker. Decision-making bodies in some countries set threshold values for ICERs or use benchmarks as rough guides to value, particularly in the arena of pharmaceutical coverage decisions. Examples of popular but arbitrary ICER thresholds include $50,000, £29,000, or €30–40,000 per QALY [25–27]. Alternatives to a single threshold include a range of values, e.g., $50,000 and $250,000 per QALY as lower and upper bounds for cost-effectiveness. The World Health Organization has endorsed lower and upper bounds of 1 and 3 times a country's per capita gross domestic product. However, the “revealed preferences” of decision makers in healthcare policy show that interventions with very high ICERs may be considered acceptable if the absolute expenditures are not too high. In particular, covered treatments for rare diseases, including NBS conditions, may exceed $1 million per QALY gained [28].

Many policy makers outside of medicine prefer CBA estimates expressed in terms of money and which allow for comparison across sectors such as health, environment, and transportation. Cost-benefit analyses in the medical arena often rely on the traditional “human capital” approach to estimating the monetary value of avoided death or disability as the discounted present value of the stream of future annual earnings or productivity. That approach is problematic. First, it ignores the costs of creating and maintaining the stock of human capital [29]. Second, some economists have advocated a “friction cost” approach to valuing the loss of a worker as the temporary cost of recruiting and training a replacement; that approach assumes that a child death involves no loss of future productivity [30]. Third, the human capital approach is inconsistent with the welfare theoretic basis of modern CBA, also referred to as benefit-cost analysis (BCA). In particular, it excludes the value that society places on avoidance of pain and suffering and the spillover effects of death and disability on other people. Exclusion of such valuations understates the economic value to society of disease prevention.

Since the 1970s, the dominant approach to CBA/BCA outside of health care involves assessments of consumer “willingness to pay” (WTP) to value health outcomes. More precisely, researchers assess individual WTP to reduce by a small amount the risk of adverse outcomes and aggregate across individuals to value the prevention of those outcomes [31]. Researchers use either stated preference methods (survey data) or revealed preferences from real-world behavior; the latter includes estimates of compensating wage differentials in relation to occupational fatality risks to estimate what is referred to in the United States as the “value of a statistical life” (VSL). Current US regulatory agency practice uses a VSL estimate of roughly $9 million to value an averted death, with a range of empirical estimates of approximately $7 million to $11 million [32–34]. That compares with US human capital estimates of lifetime productivity of a little over $1 million [35]. Stated preference estimates of the value of preventing as statistical fatality (VPF) in Europe are lower [36].

The traditional differences between CEA/CUA and CBA/BCA studies have begun to blur, although those changes have not yet affected the NBS economic evaluation literature. On the one side, many CUA studies also report estimates of net monetary benefit calculated by using multiple ICER threshold values as estimates of decision makers’ WTP for health gains [37]. From the other side, some CBAs calculate the value of a statistical life year (VSLY) by dividing VSL estimates by numbers of discounted life-years and multiplying them by projected life-years [33]. For example, a VSL of $9 million may imply a VSLY of roughly $400,000 depending on the age group. VSLY or VPF-based estimates substantially exceed conventional WTP estimates for QALYs [27,36].

The calculation of cost-effectiveness or cost-benefit ratios can be divided in four parts. The first and most important component of an economic evaluation is the quantification of health impact. Without effectiveness in terms of better health outcomes with screening, it is impossible to demonstrate cost-effectiveness. Closely related to this is the calculation of the net economic benefits of improved health outcomes, including reduced treatment costs. The third component is relatively straightforward: how much does it cost to implement a policy, e.g., the added costs to laboratories, healthcare systems, and public health authorities to conduct screening, assure its quality, follow up infants, and provide diagnoses. The fourth and final component is to place monetary or utility values on the health outcomes. In this paper, we focus on the first two components of cost-effectiveness, quantifying the gains in health outcomes and the magnitude of avoided costs associated with improvements in health outcomes.

To calculate the magnitude of differences in health outcomes that can be attributed to screening requires the assumption of a counterfactual scenario in which the same level of clinical care, including treatment options, are provided for children with and without screening once diagnosed. Many study designs fall short of providing evidence that addresses this criterion. First, the “natural history” of a disorder, i.e., the prognosis in the absence of treatment, is misleading as a comparison, since the availability of treatment must be the same for NBS and non-NBS cohorts to avoid misattribution. Similarly, the use of historical or geographical controls for whom treatment options may have differed can be misleading because it is difficult to separate the impact of screening from differences in clinical care following diagnosis [38,39].

The basic approach to assess net health impacts of NBS in principle is straightforward: compare health outcomes of affected children of the same ages who differed with regard to the timing and type of diagnosis, but were given the same treatments once diagnosed. Specifically, researchers should seek to produce evidence that early versus late diagnosis is associated with markedly better health or developmental outcomes at the same chronological age. This last point is crucial. It is common that infants diagnosed presymptomatically with a genetic disorder based on NBS or family history are healthier at the time of diagnosis than those diagnosed at a later age based on the appearance of symptoms [40,41]. Such findings tell us nothing about the effectiveness of early diagnosis in avoiding the subsequent development of symptoms.

Evidence of effectiveness may come from prospective follow-up of screened and unscreened cohorts for a range of endpoints, which may include survival, avoidance of severe morbidity, and retention of normal neurological function versus intellectual disability. However, long-term outcomes in screened cohorts are generally not available when conditions are being considered for inclusion in NBS panels. Instead, researchers may have data on outcomes among a group of children with a disorder who were diagnosed at various ages. Early diagnosis may occur as a result of a positive family history, typically the experience of an affected older sibling, or through prenatal or neonatal screening. Analysts can stratify their data by age of diagnosis to assess outcomes for early versus late diagnosis. However, it is important to compare outcomes at the same ages in order to avoid bias from age differences in the progression of symptoms. In particular, if outcomes get worse as children get older, children diagnosed as infants will appear healthier than children diagnosed later, on the basis of symptoms, even if there were no effect of early diagnosis.

Another potentially valuable source of information on the impact of early versus late diagnosis is paired sibling cohort studies. Such studies follow cohorts of affected children in which an older sibling was detected based on symptoms and a younger sibling was detected based on testing, usually as a result of the positive family history. One can compare outcomes for the siblings at the same age group. One limitation is small numbers; small differences in absolute magnitude are unlikely to be statistically significant even if large in relative size and clinically important. Investigators who follow conventional or frequentist statistical inference will often dismiss such findings as evidence of no association. However, that is an error of statistical inference. Lack of conclusive evidence of effect is not equivalent to evidence of no effect. All that one can conclude from such an analysis is that it is not possible to precisely estimate the magnitude of effect, if any. It is important to compare new findings with previous findings in terms of the direction and relative magnitudes of association to look for consistency.

It is difficult to reliably ascertain long-term health outcomes for unscreened cohorts. One reason is that in the absence of screening, individuals with a given congenital or genetic condition may not necessarily come to clinical attention. That may happen because in some cases the disorder is subclinical, symptoms are nonspecific, or the condition results in early death without postmortem diagnosis. Another reason is that most conditions detected by NBS are rare; to identify sufficient numbers of cases to assess outcomes may require collecting outcomes data for cohorts based on millions of births, which may be impractical. Furthermore, outcomes for historical cohorts, who did not have access to currently available treatments, typically are worse than expected outcomes with current treatments in the absence of screening [38,39].

Data limitations have important implications for the conduct and interpretation of economic evaluations. On the one hand, extrapolation of data from historical controls to the projection of outcomes in the absence of NBS can substantially overstate the health and economic impacts of NBS. Not only may controls lack access to the same interventions following diagnosis, but trends in health outcomes resulting from improved treatments reduces the magnitude of potential health gains from early detection. On the other hand, lack of long-term follow-up data can lead to the understatement of future health and economic benefits. For example, higher economic productivity resulting from improved child health and nutrition is difficult to model and is often left out of analyses. As a result, estimates of net economic benefit may understate the actual benefits.

PKU is an autosomal recessive disorder which, without treatment, results in intellectual impairment and disability. Prior to the development of dietary treatment for PKU in the 1950s, as many as 95% of individuals with PKU developed severe to profound intellectual disability (with IQ < approximately 50), almost all of whom received residential care [42]. According to a later study, about 95% of untreated individuals with PKU had below normal intelligence, with about 80% in the severely to profoundly affected range [43]. During the mid-1950s, low-phenylalanine dietary treatment was developed and shown to be highly effective in preventing further progression of cognitive decline and to prevent the onset of decline when begun in early infancy among younger siblings of affected children [44,45]. Beginning in the late 1950s, a urine test for PKU was widely used in the United Kingdom to screen infants for PKU during home visits [46].

In 1960, Robert Guthrie developed a highly sensitive and inexpensive semiquantitative bacterial inhibition assay to screen for PKU in DBS that could be used in birthing hospitals. A screening study Guthrie conducted among 3,118 residents of the Newark (New Jersey) State School in 1961 found that 21 had PKU [47]. Following a large-scale pilot screening study in 29 US states, NBS for PKU was quickly adopted in most US states between 1963 and 1967 [46,48]. The rationale was the opportunity to avoid preventable severe disability and provide children and their families with the opportunity of healthy, independent development.

A frequent argument made by advocates of screening newborns for PKU was that it would save taxpayers money by reducing the money spent by states on residential institutions [46]. Subsequently, analysts compared the expected reduction in costs resulting from avoided institutionalization with the cost of screening and treatment [49–52]. In California, detailed cost calculations from the first 2 years of screening showed that the cost per child with PKU detected was $2500 and the cost of dietary treatment for 10 years was approximately $8000 [49]. In comparison, the expected cost of institutionalization over a 30 year period was estimated to be $162,000, for a cost-savings ratio of 15:1. In Canada, Webb suggested that the cost to diagnose and treat one child with PKU for 5 years was $7000, compared with an expected cost of $250,000 to provide lifetime institutional care, a ratio of 36:1 [51].

Other analyses also concluded that screening for PKU would save money, albeit not as dramatically. Steiner and Smith, using data from Mississippi, concluded that screening and treatment for 7 years would cost $56,000 per child with PKU and the avoided cost of institutional care over a 30-year period would be $77,000 per child, a ratio of 1.4:1 [50]. In addition, the authors calculated a benefit-cost ratio, including gain in lifetime productivity as a benefit, of 2.6:1. Van Pelt and Levy used Massachusetts data on screening for PKU and several other metabolic conditions, and reported a cost-savings ratio of 1.8:1; they assumed that just 4 of 7 children with PKU would have required lifetime institutional care [52]. Subsequent economic analyses, whether reported as CEAs or CBAs, have also concluded that screening for PKU is cost-saving or cost-beneficial because of its prevention of severe disability [53–61]. For example, in a 2005 CEA study, Geehoed et al. projected that 64% of children with PKU would experience severe intellectual disability in the absence of NBS, citing two studies reporting data on children or adults with untreated PKU born in the 1950s or earlier [61].

CBAs of PKU screening have relied on the “human capital” approach to estimating the monetary value of avoided death or disability as the discounted present value of the stream of future annual earnings or productivity in addition to avoided costs of institutional care. According to the “friction cost” approach there is no loss of productivity attributable to congenital conditions [30]. Economic analyses of the expected benefits of screening for PKU, although they appear to have been persuasive, were not based on counterfactual comparisons of screened and unscreened cohorts exposed to dietary treatment. Analysts assumed that the natural history of untreated PKU was the appropriate comparison. They therefore used case series of untreated individuals with PKU as the comparison with cohorts of screened children with PKU. It was widely assumed that children with PKU who are not treated soon after birth would develop irreversible severe cognitive impairment and require lifetime institutional care [46].

However, published data available in the late 1960s and early 1970s belied the assumption that late-diagnosed, late-treated children with PKU have the same prognosis as untreated individuals. Specifically, peer-reviewed studies found that many late-treated children had cognitive test scores either in the low-normal range or had scores indicative of mild intellectual disability [62]. For example, two studies published in 1968 both reported that US or UK children who were put on a low-phenylalanine diet after 4–6 months of age had mean IQ scores of 69 or 77, respectively [63,64].

Experts on PKU came to realize that early cognitive deficits in late-diagnosed PKU with prolonged treatment can be partially reversed in many cases [65,66]. In California, adults with PKU who were born after 1965, but were not detected through NBS, had mean IQ scores of 76 if diagnosed at 3–7 years of age, 92 if diagnosed at 1–2 years of age, and 96 if diagnosed and treated at any time in infancy [62,65]. Despite this recognition among PKU clinical specialists, the NBS community and policy analysts continue to cite obsolete estimates of economic benefits that were predicated on the invalid assumption that late treatment is equivalent to no treatment. A CEA study published in a major peer-reviewed journal in 2006 assumed that in the absence of NBS, 95% of children with PKU would experience moderate to severe developmental delay [60].

Screening for PKU may be less likely to be cost saving (in terms of direct costs) than was previously calculated for a few other reasons [16,67]. First, classical PKU is now recognized to be the severe portion of a spectrum of hyperphenylalaninemia, and a large percentage of infants detected as abnormal by the Guthrie test have mild hyperphenylalaninemia and do not benefit from treatment [68]. Second, the per-person cost of treatment for PKU is now much greater than was assumed previously, when it was thought that older children could safely discontinue the unpleasant, arduous, and expensive low-phenylalanine diet. Since the early 1980s, it has been recommended by experts that dietary therapy be pursued for life [46]. Third, individuals with intellectual disability are now much less likely to be institutionalized than was the case historically, resulting in substantially lower direct costs of care.[69,70]. Fourth, children born to mothers with inadequately treated PKU (maternal PKU) are at risk for birth defects and disability. With NBS, more women with PKU have offspring at risk of maternal PKU and the associated costs of lifetime care [71].

On the other hand, the full benefits to society of screening newborns for PKU in avoiding disability and promoting optimal human development could be even larger than previously estimated. In particular, the economic benefits from improved labor productivity due to gains in cognitive ability are large, even for those who would not be classified as disabled. It has been estimated that each 1 IQ point gain raises lifetime earnings by thousands of dollars [72]. Similar methods have been used to evaluate the economic benefit of prevention of iodine deficiency from the societal perspective [73]. However, direct stated preference estimates of WTP to avoid a 6-point loss of IQ in a child are much smaller than the human capital estimates based on expected gains in lifetime earnings [74].

Studies are also needed to quantify other impacts of prompt versus late treatment of PKU such as psychosocial health impacts that can be quantified in terms of QALYs. CUAs of other NBS conditions that result in neurodevelopmental disability have adopted widely varying estimates of utility weights for the calculation of QALY gains from prevention of neurological problems, which calls into question the reliability of the QALY estimates [23].