<!DOCTYPE article
PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD with MathML3 v1.3 20210610//EN" "JATS-archivearticle1-3-mathml3.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" dtd-version="1.3" xml:lang="en" article-type="research-article"><?properties manuscript?><processing-meta base-tagset="archiving" mathml-version="3.0" table-model="xhtml" tagset-family="jats"><restricted-by>pmc</restricted-by></processing-meta><front><journal-meta><journal-id journal-id-type="nlm-journal-id">8406899</journal-id><journal-id journal-id-type="pubmed-jr-id">7945</journal-id><journal-id journal-id-type="nlm-ta">Vaccine</journal-id><journal-id journal-id-type="iso-abbrev">Vaccine</journal-id><journal-title-group><journal-title>Vaccine</journal-title></journal-title-group><issn pub-type="ppub">0264-410X</issn><issn pub-type="epub">1873-2518</issn></journal-meta><article-meta><article-id pub-id-type="pmid">26808648</article-id><article-id pub-id-type="pmc">11345718</article-id><article-id pub-id-type="doi">10.1016/j.vaccine.2016.01.021</article-id><article-id pub-id-type="manuscript">HHSPA2017404</article-id><article-categories><subj-group subj-group-type="heading"><subject>Article</subject></subj-group></article-categories><title-group><article-title>Beyond efficacy: The full public health impact of vaccines</article-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Saadatian-Elahi</surname><given-names>Mitra</given-names></name><xref rid="CR1" ref-type="corresp">*</xref><aff id="A1">Hospices Civils de Lyon, Groupement Hospitalier Edouard Herriot, 5 Place d&#x02019;Arsonval, 69437 Lyon Cedex 03, France</aff></contrib><contrib contrib-type="author"><name><surname>Horstick</surname><given-names>Olaf</given-names></name><aff id="A2">Institute of Public Health, University of Heidelberg, Germany</aff></contrib><contrib contrib-type="author"><name><surname>Breiman</surname><given-names>Robert F.</given-names></name><aff id="A3">Emory Global Health Institute, Emory University, Atlanta, GA, United States</aff></contrib><contrib contrib-type="author"><name><surname>Gessner</surname><given-names>Bradford D.</given-names></name><aff id="A4">Agence de M&#x000e9;decine Pr&#x000e9;ventive, Paris, France</aff></contrib><contrib contrib-type="author"><name><surname>Gubler</surname><given-names>Duane J.</given-names></name><aff id="A5">Duke-NUS Graduate Medical School, 8 College Road, Singapore 169857, Singapore</aff></contrib><contrib contrib-type="author"><name><surname>Louis</surname><given-names>Jacques</given-names></name><aff id="A6">Fondation M&#x000e9;rieux, 17 rue Bourgelat, 69002 Lyon, France</aff></contrib><contrib contrib-type="author"><name><surname>Parashar</surname><given-names>Umesh D.</given-names></name><aff id="A7">National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, United States</aff></contrib><contrib contrib-type="author"><name><surname>Tapia</surname><given-names>Roberto</given-names></name><aff id="A8">Fundaci&#x000f3;n Carlos Slim, Mexico</aff></contrib><contrib contrib-type="author"><name><surname>Picot</surname><given-names>Valentina</given-names></name><aff id="A9">Fondation M&#x000e9;rieux, 17 rue Bourgelat, 69002 Lyon, France</aff></contrib></contrib-group><contrib-group><contrib contrib-type="author"><name><surname>Zinsou</surname><given-names>Jean-Antoine</given-names></name></contrib><contrib contrib-type="author"><name><surname>Nelson</surname><given-names>Christopher B.</given-names></name></contrib><aff id="A10">Sanofi Pasteur, Vaccination Policy Department, 2 Avenue du Pont Pasteur, 69367 Lyon Cedex 07, France</aff></contrib-group><author-notes><corresp id="CR1"><label>*</label>Corresponding author at: Groupement Hospitalier Edouard Herriot, Service d&#x02019;Hygi&#x000e8;ne, Epid&#x000e9;miologie et Pr&#x000e9;vention, B&#x000e2;timent 1, 5, place d&#x02019;Arsonval, 69437 Lyon Cedex 03, France. fax: +33 04 72 11 07 26. <email>mitra.elahi@chu-lyon.fr</email> (M. Saadatian-Elahi)</corresp></author-notes><pub-date pub-type="nihms-submitted"><day>18</day><month>8</month><year>2024</year></pub-date><pub-date pub-type="ppub"><day>24</day><month>2</month><year>2016</year></pub-date><pub-date pub-type="epub"><day>23</day><month>1</month><year>2016</year></pub-date><pub-date pub-type="pmc-release"><day>26</day><month>8</month><year>2024</year></pub-date><volume>34</volume><issue>9</issue><fpage>1139</fpage><lpage>1147</lpage><abstract id="ABS1"><p id="P1">There is an active discussion in the public health community on how to assess and incorporate, in addition to safety and measures of protective efficacy, the full public health value of preventive vaccines into the evidence-based decision-making process of vaccine licensure and recommendations for public health use. The conference &#x0201c;Beyond efficacy: the full public health impact of vaccines in addition to efficacy measures in trials&#x0201d; held in Annecy, France (June 22&#x02013;24, 2015) has addressed this issue and provided recommendations on how to better capture the whole public health impact of vaccines.</p><p id="P2">Using key examples, the expert group stressed that we are in the midst of a new paradigm in vaccine evaluation, where all aspects of public health value of vaccines beyond efficacy should be evaluated. To yield a wider scope of vaccine benefits, additional measures such as vaccine preventable disease incidence, overall efficacy and other outcomes such as under-five mortality or non-etiologically confirmed clinical syndromes should be assessed in addition to traditional efficacy or effectiveness measurements. Dynamic modelling and the use of probe studies should also be considered to provide additional insight to the full public health value of a vaccine. The use of burden reduction and conditional licensure of vaccines based on collection of outcome results should be considered by regulatory agencies.</p></abstract><kwd-group><kwd>Vaccine efficacy</kwd><kwd>Vaccine preventable disease incidence</kwd><kwd>Vaccine effectiveness</kwd><kwd>Vaccine probe analysis</kwd><kwd>Beyond vaccine efficacy</kwd><kwd>Conference report</kwd></kwd-group></article-meta></front><body><sec id="S1"><label>1.</label><title>Introduction</title><p id="P3">Traditionally, vaccine efficacy, i.e. the percentage reduction of disease in a vaccinated group compared to an unvaccinated group, has been used as the primary benchmark for vaccine licensure. However, efficacy provides a measure of proportionate reduction, is limited to etiologically confirmed disease, and focuses on individual level effects; consequently, it does not capture the full public health impact of a vaccination program. In addition to preventing infection in individuals, the ultimate goal of vaccination is to achieve a significant public health impact in the catchment population. Thus, there is a need to provide a broader measure of impact beyond efficacy and safety that encompasses the capacity of a vaccination program to reduce infection transmission, disease burden (incidence, mortality, sequelae), the pressure on health systems and health inequities between populations, as well as measure coverage and mechanisms of action, all of which help determine a vaccine&#x02019;s impact [<xref rid="R1" ref-type="bibr">1</xref>,<xref rid="R2" ref-type="bibr">2</xref>]. Additionally, the full public health impact will require additional measures, such as vaccine preventable disease incidence (VPDI), number needed to vaccinate, and a wider range of outcomes, such as under-five mortality, impact on syndromic disease, and indirect vaccine effects, as well as additional analytic or design strategies, such as dynamic modelling (i.e. statistical approaches used to express and model the behaviour of a system overtime) and the use of probe studies (i.e. that attempt to estimate the impact of vaccines against syndromes or disease states). These issues are often lost in regulatory discussions, where there is a focus on risk: benefit ratios, as measured only by vaccine efficacy and safety. The relevance of these issues is highlighted herein with reference to pneumococcal, rotavirus, malaria and dengue vaccines.</p><p id="P4">To consider approaches to expand regulatory and policy discussions towards integrating disease burden reduction and vaccine efficacy/effectiveness measurements, the Fondation M&#x000e9;rieux organized a conference from June 22&#x02013;24, 2015 entitled: &#x0201c;Beyond efficacy: the full public health impact of vaccines in addition to efficacy measures in trials&#x0201d; in Annecy, France (&#x0201c;Les Pensi&#x000e8;res&#x0201d; Conference Centre). A multi-disciplinary group of experts drawn from academia, industry, international organizations and public health institutes gathered to discuss the public health impact of vaccination on preventable disease burden in the contexts of vaccine licensure, developing evidence-informed immunization program policy for public sector vaccination programs and of developing communication strategies for target populations. Key issues addressed included:
<list list-type="bullet" id="L2"><list-item><p id="P5">The concept of moderately effective vaccines and the limits of vaccine efficacy</p></list-item><list-item><p id="P6">Preventable disease burden outcomes and measurement</p></list-item><list-item><p id="P7">Key examples from the past and potential examples from the future</p></list-item><list-item><p id="P8">The role of modelling and probe studies in assessing preventable disease burden</p></list-item><list-item><p id="P9">The potential for regulatory agencies to consider preventable disease burden as a criterion for vaccine licensure</p></list-item></list></p><p id="P10">This report provides a summary of selected issues discussed by participants, key findings and recommendations for future approaches to addressing the full health impact of vaccines.</p></sec><sec id="S2"><label>2.</label><title>General concepts and methodological approaches</title><p id="P11">In the vaccine licensure pathway, randomized clinical trials, including those used for phase III trials, are designed to assess vaccine efficacy that is defined as: &#x0201c;the proportionate reduction of the incidence of the target infection in vaccinated subjects compared to controls&#x0201d; [<xref rid="R3" ref-type="bibr">3</xref>]. However, it is equally important to assess vaccine effectiveness, generally assessed in phase IV trials, which is defined as the actual performance of a vaccine at population level, or the balance of benefits and risks following introduction of a vaccine into routine immunization programs [<xref rid="R3" ref-type="bibr">3</xref>]. Both vaccine efficacy and effectiveness can be based on individual or cluster-randomized designs and can report direct and indirect effects of vaccines. Direct effect is the direct protective effect in a vaccinated subject. Indirect effects correspond to the reduction of infection or disease transmission in unimmunized subjects due to the presence of immune individuals [<xref rid="R4" ref-type="bibr">4</xref>,<xref rid="R5" ref-type="bibr">5</xref>]. Total vaccine effectiveness is the combined effects of the chosen vaccination strategy and direct protective effect in vaccinated subjects while overall vaccine effectiveness (i.e. herd effect) is the effect of vaccine in the population with immunized and unimmunized subjects as compared to if the population had not had the vaccination strategy [<xref rid="R6" ref-type="bibr">6</xref>] (<xref rid="F1" ref-type="fig">Fig. 1</xref>).</p><p id="P12">Documentation of the overall vaccine effect (i.e. herd effect) is increasingly required as countries introduce new vaccines into their immunization programs. Its assessment is usually implemented post-licensure, but can face difficulties in the developing world due to the lack of adequate infrastructure for immunization records, surveillance and laboratory confirmation of the target disease. In these countries, cluster-randomized or group-randomized studies can be performed to evaluate vaccine effectiveness in parallel to vaccine efficacy during phase III vaccine trials. Schools [<xref rid="R7" ref-type="bibr">7</xref>], communities [<xref rid="R8" ref-type="bibr">8</xref>,<xref rid="R9" ref-type="bibr">9</xref>], dwellings or premises [<xref rid="R10" ref-type="bibr">10</xref>], and contagious geographical neighbourhoods [<xref rid="R11" ref-type="bibr">11</xref>,<xref rid="R12" ref-type="bibr">12</xref>] have been used as clusters in vaccine trials to assess herd protection. Cluster randomization allows more direct examination of the herd effect but requires minimal level of transmission between clusters, knowledge of the population before randomization and larger sample size. Extrapolation of the results to other clusters could be performed by using mathematical modelling.</p><p id="P13">Assessment of vaccine effectiveness characterizes the vaccine performance when implemented in a public health program, but it does not tell the full story of the impact of vaccines on disease burden. Indeed, most studies cannot have etiologic confirmation of 100% of true cases due to the limited sensitivity or specificity of laboratory tests for some pathogens. The inability to accurately document vaccine impact on disease burden using directly measured etiologically-confirmed cases is problematic since policymakers consistently mention the burden of etiologically-confirmed clinical disease as one of the most important factors in priority setting.</p><p id="P14">Measures beyond efficacy, such as vaccine preventable disease incidence (VPDI), may provide further information to inform economic assessment of vaccines. VPDI is defined as: outcome incidence in an unvaccinated population &#x000d7; vaccine effectiveness. It is a combined measure of vaccine effectiveness (or efficacy) and the baseline disease burden [<xref rid="R13" ref-type="bibr">13</xref>]. The measurement of VPDI during clinical trials in addition to traditional efficacy or effectiveness measurements can overcome limitations related to suboptimal sensitivity or lack of diagnostic tests, and allow measurement of the total burden of disease preventable by vaccine regardless of whether disease is etiologically confirmed or clinically suspected. Vaccine efficacy is usually used to confirm that a vaccine works, and thus is best documented against etiologically confirmed disease. By contrast, VPDI is used to estimate total disease burden reduction from a vaccine and is thus optimally calculated from vaccine impact on syndromic disease, as this approach also measures the contribution of the pathogen to the causal chain of illness regardless of where in the chain the pathogen occurs.</p><sec id="S3"><label>2.1.</label><title>Vaccine probe studies</title><p id="P15">Vaccine probe studies emerged in the past 15 years and are particularly useful for pathogens for which the true burden may be hidden due to the absence of accurate laboratory testing or limited sensitivity of available diagnostics Vaccine probe studies can estimate VPDI, as well as the proportion of a syndrome caused by the pathogen, ideally, via randomized clinical trials [<xref rid="R13" ref-type="bibr">13</xref>] (note that while less precise, VPDI can also be estimated post licensure by evaluating changes in outcome incidence during the pre- and post-vaccine period and using time-series analysis can also be used). Probe analysis has been used for several vaccines with known efficacy to assess the total burden of disease preventable by vaccine whether disease burden is etiologically confirmed or documented clinically [<xref rid="R14" ref-type="bibr">14</xref>&#x02013;<xref rid="R16" ref-type="bibr">16</xref>].</p><p id="P16">Another measure of some interest to public health discussions of vaccine utility is the calculation of syndromic etiologic fraction, defined as the vaccine effectiveness against syndromic disease divided by effectiveness against etiologically confirmed disease. For example, in a randomized controlled trial of pregnant women in Bangladesh, inactivated influenza vaccine given to mothers reported an effectiveness among young infants of 62.8% (95%CI:5.0&#x02013;85.4) against etiologically confirmed disease and an effectiveness of 28.9% (95%CI: 6.9&#x02013;45.7) for all clinically documented febrile respiratory infection [<xref rid="R17" ref-type="bibr">17</xref>]. The calculated etiologic fraction would be 28.9%/62.8% = 46%, and provides an estimate of the fraction of febrile respiratory illness due to influenza.</p><p id="P17">Similar calculations can be done for the efficacy of pneumococcal conjugated vaccine (PCV) in reducing radiologically confirmed pneumonia (defined as pleural effusion or alveolar infiltrate) in children. Bacteriologic studies from children with pneumonia [<xref rid="R18" ref-type="bibr">18</xref>&#x02013;<xref rid="R22" ref-type="bibr">22</xref>] reported PCV-vaccine effectiveness against radiologically confirmed pneumonia (i.e., pneumonia with an alveolar consolidation or pleural effusion) ranging from &#x02212;2% in the USA to 37% in the Gambia (<xref rid="T1" ref-type="table">Table 1</xref>). Taking the example of the Gambia, if PCV prevents 50% of cases of invasive pneumococcal disease and 37% of radiologically confirmed pneumonia, then the etiologic fraction of radiologically confirmed pneumonia due to vaccine serotype pneumococcal disease was 74%, assuming that the efficacy against invasive pneumococcal disease and radiologically confirmed pneumococcal pneumonia is similar. If, however, efficacy against severe pneumococcal pneumonia is actually lower than what it is for invasive disease, then the proportion of pneumonias due to pneumococcus would be estimated to be even higher. As for flu in Bangladesh, the vaccine probe approach has provided information beyond that available from measuring vaccine effectiveness alone.</p><p id="P18">Vaccine probe analysis may also add evidence to potential health impact of a vaccine with low efficacy (e.g. malaria) by demonstrating impact against non-specific syndromes (like fever) or outcomes (like hospitalizations, antimicrobial use or clinic visits). Another potential area for the use of vaccine probe analysis is to estimate the overall VPDI in areas where this information is lacking. This would provide the data needed for Ministries of Health and Finance to evaluate the health impact value of introducing a vaccine into an immunization program. Assumptions for a typhoid conjugate vaccine probe study with primary outcome of hospitalization with 5 days of fever showed that such trials are feasible and could overcome many current limitations in estimating overall typhoid burden [<xref rid="R23" ref-type="bibr">23</xref>].</p></sec><sec id="S4"><label>2.2.</label><title>Modelling</title><p id="P19">Many decisions on the introduction of new vaccines into immunization programs are guided by economic analyses. The two main models used to evaluate cost-effectiveness of vaccination programs are static (decision analysis) models and dynamic models (e.g. the SIR Model). The two models are otherwise identical except that the former account only for direct protection of vaccines while the latter take account of changes in infection risk to unvaccinated persons resulting from a decreased population transmission [<xref rid="R25" ref-type="bibr">25</xref>]. As reported by a review of modelling approaches, the vast majority of economic evaluations of vaccines use static models, meaning that they do not take into account the indirect impact of vaccination programs [<xref rid="R24" ref-type="bibr">24</xref>].</p><p id="P20">The importance of incorporating the indirect effect of vaccination in the assessment of vaccine cost-effectiveness has been investigated in several studies that compared dynamic and static approaches. Comparison of the two models with regard to mass immunization of infants with varicella vaccine showed that as compared to static approach, dynamic models predicted a higher number of cases prevented, a concomitant increase in the average age at infection, and inter-epidemic periods [<xref rid="R25" ref-type="bibr">25</xref>]. These effects are common to many vaccine preventable diseases, particularly many childhood diseases that stimulate relatively long-term immunity. Other vaccine (disease) specific effects such as serotype replacement following PCV vaccination [<xref rid="R26" ref-type="bibr">26</xref>] are also important factors that could impact vaccination programs and are not fully captured by static models. Timing of vaccination is another element with major impact on cost-effectiveness of vaccination in the context of outbreaks and again static models are not appropriate for this purpose because optimizing schedules depends on total and not just direct disease prevention. The use of dynamic models also provided evidence that a cholera vaccination coverage of 50% in Bangladesh would reduce the number of cholera cases in unvaccinated subjects by 89% [<xref rid="R27" ref-type="bibr">27</xref>]. Thus, investment in use of dynamic models to capture indirect effects of vaccines, particularly those with high coverage rate, is crucial because they affect distribution of disease in the population and influence optimal vaccination strategy. This will lead to more appropriate decision making worldwide.</p></sec></sec><sec id="S5"><label>3.</label><title>Case studies</title><sec id="S6"><label>3.1.</label><title>Pneumococcal conjugate vaccines</title><p id="P21"><italic toggle="yes">Streptococcus pneumoniae</italic> (<italic toggle="yes">S. pneumoniae</italic>) causes a variety of clinical diseases ranging from non-severe otitis media and sinusitis to invasive pneumococcal diseases (IPD), including bacteremia, septicemia and meningitis with high mortality risk. Nasopharangeal carriage plays an important role in transmission. Prevention of vaccine serotype pneumococcal diseases is promoted by pneumococcal conjugated vaccines (PCV) containing 7 (PCV-7; Prevnar<sup>&#x02122;</sup>), 10 (PCV10 Synflorix<sup>&#x02122;</sup>) or 13 (PCV-13; Prevnar13<sup>&#x02122;</sup>) serotypes. Estimation of the overall burden of disease preventable by vaccine faces some difficulties related to poor sensitivity of blood culture for identifying pneumococcal pneumonia (since most pneumococcal pneumonia is non-bacteremic), etiological assignment (pneumonia) and overlapping clinical presentations (otitis media). The Finnish Invasive Pneumococcal disease vaccine effectiveness trial (FinIP) is a phase III/IV probe vaccine design, cluster-randomized, double-blind trial in children &#x0003c;19 months that aimed at investigating vaccine effectiveness of the pneumococcal Haemophilus influenzae protein D conjugate vaccine (PHID-CV10) and absolute rate reduction against (1) laboratory-confirmed IPD and (2) clinically suspected IPD, hospital-diagnosed pneumonia and otitis media by use of diagnoses coded in hospital discharge and antimicribial purchases registries [<xref rid="R28" ref-type="bibr">28</xref>,<xref rid="R29" ref-type="bibr">29</xref>]. PCV effectiveness was the highest for laboratory-confirmed IPD and decreased substantially for less specific outcomes such as clinical pneumonia and otitis media, while the VPDI value increased in the opposite way. Indeed, the VPDI per 100,000 person-years was 75 for laboratory-confirmed IPD, 205 for non-laboratory-confirmed but clinically suspected IPD, 340 for hospital-diagnosed pneumonia, and 12,000 for any antimicrobial outpatient prescription mainly due to otitis media [<xref rid="R28" ref-type="bibr">28</xref>,<xref rid="R29" ref-type="bibr">29</xref>]. The higher rate of preventable disease among clinical rather than laboratory-confirmed diseases indicate that vaccine has greater public health value and cost-effectiveness than would be estimated from assessment of laboratory-confirmed illness alone. Where laboratory diagnostics are poor, laboratory confirmed disease rates may more substantially underestimate pneumococcal disease burden. In Finland, PCV-10 was introduced to the national immunization program in September 2010 for children born from June 2010 (2 + 1 schedule at 3, 5 and 12 months of age) without catch-up. The impact of this vaccine in Finnish children has been evaluated by a population-based, observational follow-up study of 2 cohorts (eligible and older ineligible children) [<xref rid="R30" ref-type="bibr">30</xref>]. The results showed significant reduction (80%; 95%CI: 72&#x02013;85) in the overall rate of vaccine-serotype IPD among eligible children and a 48% (95%CI: 18&#x02013;69) reduction among ineligible unvaccinated children due to herd immunity. The VPDI value for laboratory-confirmed IPD, non-laboratory-confirmed IPD and pneumonia were 50, 122 and 90 per 100,000 children-years (age 3&#x02013;42 months), respectively. These estimates are lower than what has been reported by the FinIP trial and illustrate the added value of vaccine probe studies and assessment of VPDI for the evaluation of overall vaccine benefits.</p></sec><sec id="S7"><label>3.2.</label><title>Rotavirus vaccines</title><p id="P22">Rotavirus is the leading cause of severe diarrhoea and accounts for about one-third of all diarrhoea-related hospitalizations among children &#x0003c;5 years of age. Two licensed oral rotavirus vaccines (Rotarix and RotaTeq) are currently used in more than 75 countries around the world. Rotavirus vaccine effectiveness against severe rotavirus gastroenteritis in high and middle income countries ranged from 84% to 98% [<xref rid="R31" ref-type="bibr">31</xref>&#x02013;<xref rid="R37" ref-type="bibr">37</xref>]. By contrast, vaccine effectiveness was only moderate (51&#x02013;64%) in Africa and in less developed areas of Asia [<xref rid="R38" ref-type="bibr">38</xref>&#x02013;<xref rid="R40" ref-type="bibr">40</xref>]. Many potential factors such as malnutrition, presence of concomitant infections (in particular enteric infections), and environmental enteropathy could interfere with the performance of these vaccines. Yet even with lowered vaccine effectiveness, the higher disease burden in these settings might mean that vaccine has a greater public health impact in low-income countries than in high-income settings. In this case, VPDI might be a more appropriate marker of the real impact of vaccination program as it is related to the etiologically specific disease incidence. This has been demonstrated by a randomized-placebo controlled trial that showed lower vaccine effectiveness in Malawi than in South Africa (49.4% vs. 72%); while the number of prevented episodes of severe rotavirus gastroenteritis was greater in Malawi than in South Africa (67 vs. 42 per 1000 infants vaccinated per year) [<xref rid="R38" ref-type="bibr">38</xref>]. Moreover, VPDI value in these African countries was greater than those estimated in high and middle income countries where vaccine effectiveness was higher.</p></sec><sec id="S8"><label>3.3.</label><title>Malaria vaccine</title><p id="P23"><italic toggle="yes">Plasmodium falciparum</italic> and other malaria species caused 198 million cases (124&#x02013;284 million) and 584,000 (367,000&#x02013;755,000) deaths in 2013 [<xref rid="R41" ref-type="bibr">41</xref>]. It is a global problem in the tropics but due to the high density of the parasite in Africa, the highest transmission rates and about 90% of mortality are on the African continent [<xref rid="R42" ref-type="bibr">42</xref>]. Currently, the only candidate malaria vaccine close to licensure is RTS,S/AS01 that is directed against the pre-erythrocytic stage of the parasite (i.e. before its entry into the red blood cells). Based on one of the severe malaria definitions used in the trial, vaccine efficacy during the 12 months after vaccination was 38% among infants vaccinated at 6&#x02013;12 weeks of age and 47% among children vaccinated at 5&#x02013;17 months of age. While efficacy was relatively low compared to other vaccines used in African national immunization programs, the high background malaria incidence meant that these efficacies equated to a reduction of 900 and 2300 severe malaria cases per 100,000 study subjects, respectively [<xref rid="R43" ref-type="bibr">43</xref>,<xref rid="R44" ref-type="bibr">44</xref>], a substantially higher severe disease burden reduction than that calculated for some routine infant immunizations [<xref rid="R45" ref-type="bibr">45</xref>]. No definite, protective effect against death was demonstrated in the clinical trials, possibly because almost no malaria deaths occurred in either arm due to improvements in clinical management in the trial setting.</p><p id="P24">There are other arguments backed-up by epidemiological data that suggest RTS,S may have additional benefits. Several studies have reported that malaria infection predisposes the infected individual to bacteraemia. This hypothesis has been tested in a matched case-control study of Kenyan children in which cases were those with invasive bacterial disease [<xref rid="R46" ref-type="bibr">46</xref>]. The authors reported a significant inverse relationship between the malaria-protective phenotype of sickle-cell trait (HbAS) and bacteraemia (odds ratio0.36; 95%CI: 0.20&#x02013;0.65). Beside, longitudinal data [<xref rid="R46" ref-type="bibr">46</xref>] showed that a decline in bacteraemia paralleled a decrease in malaria admission [<xref rid="R47" ref-type="bibr">47</xref>], reinforcing this hypothesis. Malaria reduction may also impact all-cause mortality. Following extensive malaria control interventions in Bioko Island, Equatorial Guinea, under-5 mortality has been reduced by 2/3, from 152 per 1000 births (95%CI = 122&#x02013;186) to 55 per 1000 (95%CI = 38&#x02013;77; hazard ratio = 0.34 [95%CI = 0.23&#x02013;0.49]) [<xref rid="R48" ref-type="bibr">48</xref>]. This study assessed non-vaccine interventions and observed impacts may not necessarily apply to vaccines. Nevertheless, it provides suggestive evidence that an effective malaria vaccine could potentially prevent cases and deaths for which malaria is a sufficient cause and many more deaths for which malaria is a necessary but not sufficient part of the causal chain. In sum, the example of RTS,S provides an important message that even a vaccine with relatively low efficacy could have huge impact on overall population morbidity and mortality, a feature that which has to be taken into consideration in addition to vaccine effectiveness.</p></sec><sec id="S9"><label>3.4.</label><title>Dengue vaccine</title><p id="P25">Dengue is one of the most important emerging infectious diseases of the 21st century. Dengue epidemiology has changed in the past 40 years as the result of expanding geographic distribution of both the viruses and mosquito vectors, increased epidemic activity, the development of hyperendemicity and the emergence of severe disease [<xref rid="R49" ref-type="bibr">49</xref>,<xref rid="R50" ref-type="bibr">50</xref>]. The observed dramatic increase in the frequency and magnitude of epidemic dengue is driven by population growth, urbanization, environmental changes and modern transportation (i.e. globalization). Primary prevention through vector control has failed due to only partially effective prevention and control tools and inappropriate implementation and assessment [<xref rid="R51" ref-type="bibr">51</xref>]. New dengue disease burden estimates put the global at-risk population at 3.6 billion, with 390 million infections and 21,000 deaths annually, the latter being most probably underestimated [<xref rid="R52" ref-type="bibr">52</xref>]. Vaccine development for dengue has been hampered by several factors including the perceived need for a balanced protection against all 4 serotypes, lack of funding and limited understanding of dengue virology and pathogenesis. Currently, three dengue vaccines are in phase II and III clinical trials. One vaccine has completed phase III clinical trials, and showed efficacy against laboratory-confirmed clinical dengue (65%), hospitalized dengue (80%) and severe dengue (93%) [<xref rid="R53" ref-type="bibr">53</xref>]. The vaccine demonstrated variable efficacy against the four dengue serotypes (47&#x02013;83%) but was highly effective (52&#x02013;81%) in persons with previous exposure to dengue [<xref rid="R53" ref-type="bibr">53</xref>]. The majority of children in hyperendemic areas have already been infected with dengue at least once, and since infections with the third and fourth serotypes are generally mild or asymptomatic [<xref rid="R54" ref-type="bibr">54</xref>,<xref rid="R55" ref-type="bibr">55</xref>], a dengue vaccine with this efficacy profile should have a significant public health impact by preventing severe disease, decreasing health care utilization, particularly the hospitalization rate, and by having a priming effect in populations with previous dengue infection. Further study is required to confirm (i) whether the vaccine virus will induce the same level of immunity as natural infection, (ii) the role of virus strains with variable virulence and epidemic potential, (iii) the role of temporal distribution of infection with different serotypes and (iv) the paucity of knowledge about the disease severity in 3rd and 4th dengue infections and the immunity induced by the vaccine in these individuals. While this efficacy profile can help reduce the burden of disease and achieve public health objectives, successful dengue prevention and control is likely to be obtained only by using the vaccine in combination with other new innovative tools in the pipeline as well as already existing tools and strategies [<xref rid="R56" ref-type="bibr">56</xref>] implemented by various public health agencies. Programmatic demonstration projects may be useful to show how the full impact of these coordinated interventions can be achieved. To facilitate this new paradigm for dengue prevention and control, a new initiative, the Partnership for Dengue Control (PDC), was created in July 2013 at an international consensus conference attended by a cross-sectional representation of the global dengue and public health communities, including representatives from academic institutions, international health agencies, NGOs, and the private sector [<xref rid="R56" ref-type="bibr">56</xref>].</p></sec></sec><sec id="S10"><label>4.</label><title>Regulatory challenges</title><p id="P26">Regulatory authorities evaluate the potential benefits of an effective vaccine, against the potential risk of adverse effects following immunization. Blinded randomized controlled trials (RCTs) are considered as the &#x0201c;Gold Standard&#x0201d; for assessing vaccine efficacy because they provide rigorous control for biases and rely on laboratory-confirmed clinical outcomes, at least for a part of the study population. However, they are expensive, and may require large sample size in particular for uncommon outcomes.</p><p id="P27">The European regulatory framework allows the benefit/risk (B/R) assessment of vaccines to rely on post-approval evidence of population benefit in lieu of pre-approval individual efficacy. Indeed, several vaccines have already been approved in Europe in the absence of vaccine efficacy data, relying either on non-inferiority to licensed vaccines (PCV10, PCV13, conjugate serogroup C meningococcal vaccine) or on surrogate markers of protection (influenza, HPV). Effectiveness and vaccine impact data for these vaccines have been post-approval commitments and have been included in label updates. For future vaccines for HIV, dengue, Ebola, malaria, and others, vaccine efficacy may not be easily shown. In these cases, relying on surrogate markers of protection, vaccine effectiveness and impact data are important to be considered in pre- and post-approval phases of vaccine licensure. Mechanisms exist already in Europe for early dialogue with European regulators. The European pilot project on adaptive pathway, set-up by the European Medicine Agency (EMA), supports the selection of a pathway of product development and potential earlier access to medicines through early dialogue involving all stakeholders [<xref rid="R57" ref-type="bibr">57</xref>]. Criteria for candidate selection are (1) an iterative development plan that starts in a well-defined subpopulation for conditional marketing authorization; (2) real world data following approval to supplement clinical trial; (3) input of all stakeholders and (4) unmet medical need. Acceptance (i.e. conditional approval) or rejection in pilot phase has no inference about the final approval. In cases where the vaccine is not intended for European countries (e.g., dengue and malaria), regulatory processes and options exist for European authorities/EMA to make available their vaccine assessment experiences.</p><p id="P28">The US Food and Drug Administration (FDA) has not defined a specific threshold for vaccine efficacy or a particular endpoint. However, there is a possibility for regulatory acceptance of vaccines with moderate efficacy. Vaccines can be licensed on the basis of immune response when correlates of protection are known. Moreover, expedited regulatory pathways are available for vaccines meeting unmet medical needs. The US FDA expedited regulatory pathways to accelerate vaccine licensure are (1) fast track program (e.g. pneumococcal vaccines and HPV vaccine); (2) breakthrough therapy (e.g. meningitis B vaccine); (3) priority review; (4) accelerated approval (e.g. influenza vaccine) and (5) emergency use authorization.</p><p id="P29">These data show that alternative regulatory process already exist and could be used in a more systematic way for vaccine licensure. The use of burden reduction as an addition to vaccine efficacy for vaccines with moderate efficacy or effectiveness is a key issue that should be taken into consideration.</p></sec><sec id="S11"><label>5.</label><title>Conclusions and recommendations</title><p id="P30">There is an active discussion in the public health community on how to assess and incorporate the full public health value of preventive vaccines into the evidence-based decision-making process of vaccine licensure and public health use. As has been highlighted for pneumococcal, rotavirus, malaria, and dengue vaccines, consideration of vaccine preventable disease incidence (VPDI), indirect effects of vaccination, assessment of impact on alternative outcomes such as under age five years mortality, and use of dynamic modelling and innovative study designs such as probe studies, can provide additional insight to the public health value of a vaccine.</p><p id="P31">Similar to influenza vaccines, many current vaccines have moderate efficacy even against etiologically confirmed outcomes. Nevertheless, for vaccines with moderate efficacy or for which all pathogenic strains are not included in the vaccine, a disease burden and economic impact reduction may justify their introduction into national immunization programs or private markets. However, absence of sufficient data may delay vaccine introduction into immunization programs. The slow introduction of Hib vaccine in low and middle income countries is one example. As noted above, vaccine efficacy fails to capture the whole public health impact of vaccines and may be relatively low even when preventable disease burden is high. In this regard, measures beyond efficacy may be more appropriate and could have a role for both vaccine licensure and policy recommendations.</p><p id="P32">Vaccines should be evaluated based on their public health impact against syndromes or disease states, not the pathogen alone. Other parameters may need to be considered for both licensure and policy recommendation. For example, the evaluation of VPDI against clinical and laboratory-confirmed outcomes in addition to vaccine effectiveness against confirmed disease allows a more complete assessment of a vaccines public health value and thus its potential economic impact. VPDI is determined by both disease epidemiology and vaccine performance. Therefore, it may have more variability across settings than vaccine effectiveness (although vaccine effectiveness can vary based on issues such as concurrent malnutrition, HIV infection, and high intestinal microbial load). This could limit the use of VPDI by regulatory agencies. Modelling is also a valuable tool, but dynamic models should be privileged due to their capacity to predict the impact of a vaccine on the overall disease burden.</p><p id="P33">The expert group concluded by stressing that we are in an era of a new paradigm in vaccine evaluation where all aspects of public health value of vaccines beyond efficacy should be evaluated. This new paradigm implies the following:
<list list-type="bullet" id="L4"><list-item><p id="P34">Large scale clinical trials, e.g. RCTs, may reach their feasibility limits.</p></list-item><list-item><p id="P35">Not all benefits of vaccines are fully explained by vaccine effectiveness, an issue of particular importance for vaccines with moderate efficacy. Other parameters and outcomes (VPDI for example) may need consideration for both licensure and policy recommendation.</p></list-item><list-item><p id="P36">Alternative regulatory pathways involving stakeholders should be considered as a process for LMICs.</p></list-item><list-item><p id="P37">A larger number of phase IV trials that are more relevant for the estimation of VPDI should be introduced.</p></list-item><list-item><p id="P38">Conditional licensure of vaccines based on collection of outcome results should be considered. This would lower the financial barriers to development of new vaccines and thus increase portfolio of vaccines to be developed and introduced.</p></list-item><list-item><p id="P39">Adding post-marketing information to the label (e.g. economic impact, disease burden) should be more settled. However, such labelling may not be as applicable for local decision-making by national immunization programs. From this perspective, the broader effect of vaccine on health outcomes when coordinated with other interventions should be considered and coordinated by relevant national agencies.</p></list-item><list-item><p id="P40">While the regulatory process focuses on collecting the minimum information necessary to establish benefit-risk, a substantial body of additional information is necessary to inform policy. To have a wider scope of vaccine benefits, improvement should occur in risk assessment, partnership and coalition building across interventions, and data sharing. The ADVANCE (Accelerated Development of Vaccine benefit-risk Collaboration in Europe) project is a good example of a partnership that was set-up to build an integrated and sustainable framework for continuous vaccine effect monitoring. Moreover, laboratory and capacity building for vaccine impact assessment &#x02013; usually with a solid surveillance foundation &#x02013; at country-level should be reinforced.</p></list-item></list></p></sec></body><back><ack id="S12"><title>Acknowledgements</title><p id="P41">The authors express their gratitude to all speakers who shared their findings. Thanks are also due to Cindy Grasso (meeting coordinator) and the staff of the M&#x000e9;rieux Foundation Conference Center for outstanding local organization. The organization of this meeting was made possible through support from Sanofi Pasteur.</p><sec sec-type="COI-statement" id="S13"><title>Conflict of interest</title><p id="P42">CBN and JAZ are employee of Sanofi Pasteur. BDG works for AMP which receives grant support from Crucell, GSK, Hilleman Laboratories, Merck, Pfizer, and Sanofi Pasteur. Other authors declare that they have no conflicts of interest to report.</p></sec></ack><app-group><app id="APP1"><title>Speakers and Chairs</title><p id="P43">Baylor Norman</p><p id="P44">Biologics Consulting Group, Inc., USA</p><p id="P45">Black Steven</p><p id="P46">Center for Global Health, University of Cincinnati Children&#x02019;s Hospital, USA</p><p id="P47">Breiman Robert</p><p id="P48">Emory Global Health Institute, USA</p><p id="P49">Dellepiane Nora</p><p id="P50">World Health Organization, Switzerland</p><p id="P51">Dull Peter</p><p id="P52">Bill &#x00026; Melinda Gates Foundation, USA</p><p id="P53">Edmunds John</p><p id="P54">London School of Hygiene &#x00026; Tropical Medicine, United Kingdom</p><p id="P55">Gubler Duane J</p><p id="P56">Duke-NUS Graduate Medical School, USA</p><p id="P57">Horstick Olaf</p><p id="P58">Institute of Public Health, University of Heidelberg, Germany</p><p id="P59">Longini Ira</p><p id="P60">University of Florida, USA</p><p id="P61">Lopez Anna-Lena</p><p id="P62">Institute of Child Health and Human Development, Philippines</p><p id="P63">Marsh Kevin</p><p id="P64">African Academy of Sciences, Kenya</p><p id="P65">Martins Helder</p><p id="P66">SIVAC, Mozambique</p><p id="P67">Nelson B Christopher</p><p id="P68">Sanofi Pasteur, France</p><p id="P69">Palu Arto</p><p id="P70">National Institute for Health and Welfare, Finland</p><p id="P71">Parashar Umesh</p><p id="P72">Centre for Disease Control, USA</p><p id="P73">Vandendriessche Frank</p><p id="P74">PhaRA, Belgium</p><p id="P75">Wilder-Smith Annelies</p><p id="P76">Lee Kong Chian School of Medicine, Singapore</p><p id="P77">Zinsou Jean-Antoine</p><p id="P78">Sanofi Pasteur, France</p></app><app id="APP2"><title>Participants</title><p id="P79">Aguado De Ros M. Teresa</p><p id="P80">Vaccines and Immunization Consultant, Switzerland</p><p id="P81">Baehner Frank</p><p id="P82">Takeda Pharmaceuticals International GmbH, Switzerland</p><p id="P83">Batoosingh Karen</p><p id="P84">Dengue Company, Sanofi Pasteur, Denmark</p><p id="P85">Beard Frank</p><p id="P86">National Centre for Immunization Research and Surveillance of Vaccine Preventable Diseases, Australia</p><p id="P87">Botting Carla</p><p id="P88">PATH, USA</p><p id="P89">Chataway Mark</p><p id="P90">Hyderus/Baird&#x02019;s CMC, United kingdom</p><p id="P91">Chotpitayasunondh Tawee</p><p id="P92">Queen Sirikit National Institute of Child Health, Thailand</p><p id="P93">Cohen Jean-Marie Open Rome, France</p><p id="P94">Esparza Jose</p><p id="P95">Institute of Human Virology, University of Maryland School of Medicine, USA</p><p id="P96">Karafillakis Emilie</p><p id="P97">London School of Hygiene and Tropical Medicine, United Kingdom</p><p id="P98">Farlow Andrew</p><p id="P99">University of Oxford, United Kingdom</p><p id="P100">Ferreira Germano</p><p id="P101">P-95 Pharmacovigilance and Epidemiology, Belgium</p><p id="P102">Flasche Stefan</p><p id="P103">LSHTM, United kingdom</p><p id="P104">Fletcher Mark-Andrew</p><p id="P105">Pfizer Vaccines, France</p><p id="P106">Groome Michelle</p><p id="P107">Respiratory and Meningeal Pathogens Research Unit, University of the Witwatersrand, South Africa</p><p id="P108">Jaenisch Thomas</p><p id="P109">Section Clinical Tropical Medicine, Department of Infectious Diseases, Heidelberg University Hospital, Germany</p><p id="P110">Knerer Gerhart</p><p id="P111">Takeda Vaccines, United Kingdom</p><p id="P112">Lang Jean</p><p id="P113">Sanofi Pasteur, France</p><p id="P114">Lee Bennett</p><p id="P115">Takeda Pharmaceuticals International (Vaccines), Switzerland</p><p id="P116">Louis Jaques</p><p id="P117">Fondation M&#x000e9;rieux, France</p><p id="P118">Moulin Anne-Marie</p><p id="P119">CNRS, France</p><p id="P120">Mogasale Vittal</p><p id="P121">International Vaccine Institute, Korea</p><p id="P122">Munier Aline</p><p id="P123">Agence de M&#x000e9;decine Pr&#x000e9;ventive, France</p><p id="P124">Ohannessian Robin</p><p id="P125">Universit&#x000e9; Lyon 1, France</p><p id="P126">Oriol Mathieu Valerie</p><p id="P127">Janssen Infectious Diseases and Vaccines, Netherlands</p><p id="P128">Pagliusi Sonia</p><p id="P129">DCVMN International, Switzerland</p><p id="P130">Picot Valentina</p><p id="P131">Fondation M&#x000e9;rieux, France</p><p id="P132">Pitzer Virginia</p><p id="P133">Yale School of Public Health, USA</p><p id="P134">Rimolo Natalia</p><p id="P135">Fondation M&#x000e9;rieux, France</p><p id="P136">Saadatian-Elahi Mitra</p><p id="P137">Fondation M&#x000e9;rieux, France</p><p id="P138">Seida Ahmed Adel</p><p id="P139">Microbiology and Immunology Department, Cairo University, Egypt seida</p><p id="P140">
<email>aa@cu.edu.eg</email>
</p><p id="P141">Teyssou R&#x000e9;my</p><p id="P142">Fondation M&#x000e9;rieux, France</p><p id="P143">Tin Tin Htar Myint</p><p id="P144">Pfizer Europe, France</p><p id="P145">Tomori Oyewale</p><p id="P146">Nigerian Academy of Science, Nigeria</p><p id="P147">Van Effelterre Thierry</p><p id="P148">GlaxoSmithKline Vaccines, Belgium</p><p id="P149">Weil John</p><p id="P150">Takeda Vaccines, Switzerland</p></app></app-group><ref-list><title>References</title><ref id="R1"><label>[1]</label><mixed-citation publication-type="journal"><name><surname>Imai</surname><given-names>N</given-names></name>, <name><surname>Dorigatti</surname><given-names>I</given-names></name>, <name><surname>Cauchemez</surname><given-names>S</given-names></name>, <name><surname>Ferguson</surname><given-names>NM</given-names></name>. <article-title>Estimating dengue transmission intensity from sero-prevalence surveys in multiple countries</article-title>. <source>PLoS Negl Trop Dis</source>
<year>2015</year>;<volume>9</volume>(<issue>4</issue>):<fpage>e0003719</fpage>, <pub-id pub-id-type="doi">10.1371/journal.pntd.0003719</pub-id>.<pub-id pub-id-type="pmid">25881272</pub-id>
</mixed-citation></ref><ref id="R2"><label>[2]</label><mixed-citation publication-type="journal"><name><surname>Mukandavire</surname><given-names>Z</given-names></name>, <name><surname>Smith</surname><given-names>DL</given-names></name>, <name><surname>Morris</surname><given-names>JG</given-names><suffix>Jr</suffix></name>. <article-title>Cholera in Haiti: reproductive numbers and vaccination coverage estimates</article-title>. <source>Sci Rep</source>
<year>2013</year>;<volume>3</volume>:<fpage>997</fpage>, <pub-id pub-id-type="doi">10.1038/srep00997</pub-id>.<pub-id pub-id-type="pmid">23308338</pub-id>
</mixed-citation></ref><ref id="R3"><label>[3]</label><mixed-citation publication-type="journal"><name><surname>Clemens</surname><given-names>J</given-names></name>, <name><surname>Brenner</surname><given-names>R</given-names></name>, <name><surname>Rao</surname><given-names>M</given-names></name>, <name><surname>Tafari</surname><given-names>N</given-names></name>, <name><surname>Lowe</surname><given-names>C</given-names></name>. <article-title>Evaluating new vaccines for developing countries. Efficacy or effectiveness?</article-title>
<source>JAMA</source>
<year>1996</year>;<volume>275</volume>(<issue>5</issue>):<fpage>390</fpage>&#x02013;<lpage>7</lpage>.<pub-id pub-id-type="pmid">8569019</pub-id>
</mixed-citation></ref><ref id="R4"><label>[4]</label><mixed-citation publication-type="journal"><name><surname>Fine</surname><given-names>P</given-names></name>, <name><surname>Eames</surname><given-names>K</given-names></name>, <name><surname>Heymann</surname><given-names>DL</given-names></name>. <article-title>Herd immunity: a rough guide</article-title>. <source>Clin Infect Dis</source>
<year>2011</year>;<volume>52</volume>(<issue>7</issue>):<fpage>911</fpage>&#x02013;<lpage>6</lpage>, <pub-id pub-id-type="doi">10.1093/cid/cir007</pub-id>.<pub-id pub-id-type="pmid">21427399</pub-id>
</mixed-citation></ref><ref id="R5"><label>[5]</label><mixed-citation publication-type="journal"><name><surname>Clemens</surname><given-names>J</given-names></name>, <name><surname>Shin</surname><given-names>S</given-names></name>, <name><surname>Ali</surname><given-names>M</given-names></name>. <article-title>New approaches to the assessment of vaccine herd protection in clinical trials</article-title>. <source>Lancet Infect Dis</source>
<year>2011</year>;<volume>11</volume>(<issue>6</issue>):<fpage>482</fpage>&#x02013;<lpage>7</lpage>, <pub-id pub-id-type="doi">10.1016/S1473-3099(10)70318-2</pub-id>.<pub-id pub-id-type="pmid">21616458</pub-id>
</mixed-citation></ref><ref id="R6"><label>[6]</label><mixed-citation publication-type="journal"><name><surname>Halloran</surname><given-names>ME</given-names></name>. <article-title>The minicommunity design to assess indirect effects of vaccination</article-title>. <source>Epidemiol Methods</source>
<year>2012</year>;<volume>1</volume>(<issue>1</issue>):<fpage>83</fpage>&#x02013;<lpage>105</lpage>.<pub-id pub-id-type="pmid">23599908</pub-id>
</mixed-citation></ref><ref id="R7"><label>[7]</label><mixed-citation publication-type="journal"><name><surname>Lehtinen</surname><given-names>M</given-names></name>, <name><surname>Apter</surname><given-names>D</given-names></name>, <name><surname>Baussano</surname><given-names>I</given-names></name>, <name><surname>Eriksson</surname><given-names>T</given-names></name>, <name><surname>Natunen</surname><given-names>K</given-names></name>, <name><surname>Paavonen</surname><given-names>J</given-names></name>, <etal/>
<article-title>Characteristics of a cluster-randomized phase IV human papillomavirus vaccination effectiveness trial</article-title>. <source>Vaccine</source>
<year>2015</year>;<volume>33</volume>(<issue>10</issue>):<fpage>1284</fpage>&#x02013;<lpage>90</lpage>, <pub-id pub-id-type="doi">10.1016/j.vaccine.2014.12.019</pub-id>.<pub-id pub-id-type="pmid">25593103</pub-id>
</mixed-citation></ref><ref id="R8"><label>[8]</label><mixed-citation publication-type="journal"><name><surname>Kwong</surname><given-names>JC</given-names></name>, <name><surname>Pereira</surname><given-names>JA</given-names></name>, <name><surname>Quach</surname><given-names>S</given-names></name>, <name><surname>Pellizzari</surname><given-names>R</given-names></name>, <name><surname>Dusome</surname><given-names>E</given-names></name>, <name><surname>Russell</surname><given-names>ML</given-names></name>, <etal/>
<article-title>Public Health Agency of Canada/Canadian Institutes of Health Research Influenza Research Network (PCIRN) Program Delivery and Evaluation Group Randomized evaluation of live attenuated vs. inactivated influenza vaccines in schools (RELATIVES) pilot study: a cluster randomized trial</article-title>. <source>Vaccine</source>
<year>2015</year>;<volume>33</volume>(<issue>4</issue>):<fpage>535</fpage>&#x02013;<lpage>41</lpage>, <pub-id pub-id-type="doi">10.1016/j.vaccine.2014.11.044</pub-id>.<pub-id pub-id-type="pmid">25488331</pub-id>
</mixed-citation></ref><ref id="R9"><label>[9]</label><mixed-citation publication-type="journal"><name><surname>Loeb</surname><given-names>M</given-names></name>, <name><surname>Russell</surname><given-names>ML</given-names></name>, <name><surname>Moss</surname><given-names>L</given-names></name>, <name><surname>Fonseca</surname><given-names>K</given-names></name>, <name><surname>Fox</surname><given-names>J</given-names></name>, <name><surname>Earn</surname><given-names>DJ</given-names></name>, <etal/>
<article-title>Effect of influenza vaccination of children on infection rates in Hutterite communities: a randomized trial</article-title>. <source>JAMA</source>
<year>2010</year>;<volume>303</volume>(<issue>10</issue>):<fpage>943</fpage>&#x02013;<lpage>50</lpage>, <pub-id pub-id-type="doi">10.1001/jama.2010.250</pub-id>.<pub-id pub-id-type="pmid">20215608</pub-id>
</mixed-citation></ref><ref id="R10"><label>[10]</label><mixed-citation publication-type="journal"><name><surname>Sur</surname><given-names>D</given-names></name>, <name><surname>Lopez</surname><given-names>AL</given-names></name>, <name><surname>Kanungo</surname><given-names>S</given-names></name>, <name><surname>Paisley</surname><given-names>A</given-names></name>, <name><surname>Manna</surname><given-names>B</given-names></name>, <name><surname>Ali</surname><given-names>M</given-names></name>, <etal/>
<article-title>Efficacy and safety of a modified killed-whole-cell oral cholera vaccine in India: an interim analysis of a cluster-randomised, double-blind, placebo-controlled trial</article-title>. <source>Lancet</source>
<year>2009</year>;<volume>374</volume>(<issue>9702</issue>):<fpage>1694</fpage>&#x02013;<lpage>702</lpage>, <pub-id pub-id-type="doi">10.1016/S0140-6736(09)61297-6</pub-id>.<pub-id pub-id-type="pmid">19819004</pub-id>
</mixed-citation></ref><ref id="R11"><label>[11]</label><mixed-citation publication-type="journal"><name><surname>Ali</surname><given-names>M</given-names></name>, <name><surname>Sur</surname><given-names>D</given-names></name>, <name><surname>You</surname><given-names>YA</given-names></name>, <name><surname>Kanungo</surname><given-names>S</given-names></name>, <name><surname>Sah</surname><given-names>B</given-names></name>, <name><surname>Manna</surname><given-names>B</given-names></name>, <etal/>
<article-title>Herd protection by a bivalent killed whole-cell oral cholera vaccine in the slums of Kolkata, India</article-title>. <source>Clin Infect Dis</source>
<year>2013</year>;<volume>56</volume>(<issue>8</issue>):<fpage>1123</fpage>&#x02013;<lpage>31</lpage>, <pub-id pub-id-type="doi">10.1093/cid/cit009</pub-id>.<pub-id pub-id-type="pmid">23362293</pub-id>
</mixed-citation></ref><ref id="R12"><label>[12]</label><mixed-citation publication-type="journal"><name><surname>Sur</surname><given-names>D</given-names></name>, <name><surname>Ochiai</surname><given-names>RL</given-names></name>, <name><surname>Bhattacharya</surname><given-names>SK</given-names></name>, <name><surname>Ganguly</surname><given-names>NK</given-names></name>, <name><surname>Ali</surname><given-names>M</given-names></name>, <name><surname>Manna</surname><given-names>B</given-names></name>, <etal/>
<article-title>A cluster randomized effectiveness trial of Vi typhoid vaccine in India</article-title>. <source>N Engl J Med</source>
<year>2009</year>;<volume>361</volume>(<issue>4</issue>):<fpage>335</fpage>&#x02013;<lpage>44</lpage>, <pub-id pub-id-type="doi">10.1056/NEJMoa0807521</pub-id>.<pub-id pub-id-type="pmid">19625715</pub-id>
</mixed-citation></ref><ref id="R13"><label>[13]</label><mixed-citation publication-type="journal"><name><surname>Feikin</surname><given-names>DR</given-names></name>, <name><surname>Scott</surname><given-names>JA</given-names></name>, <name><surname>Gessner</surname><given-names>BD</given-names></name>. <article-title>Use of vaccines as probes to define disease burden</article-title>. <source>Lancet</source>
<year>2014</year>;<volume>383</volume>(<issue>9930</issue>):<fpage>1762</fpage>&#x02013;<lpage>70</lpage>, <pub-id pub-id-type="doi">10.1016/S0140-6736(13)61682-7</pub-id>.<pub-id pub-id-type="pmid">24553294</pub-id>
</mixed-citation></ref><ref id="R14"><label>[14]</label><mixed-citation publication-type="journal"><name><surname>Peto</surname><given-names>TJ</given-names></name>, <name><surname>Mendy</surname><given-names>ME</given-names></name>, <name><surname>Lowe</surname><given-names>Y</given-names></name>, <name><surname>Webb</surname><given-names>EL</given-names></name>, <name><surname>Whittle</surname><given-names>HC</given-names></name>, <name><surname>Hall</surname><given-names>AJ</given-names></name>. <article-title>Efficacy and effectiveness of infant vaccination against chronic hepatitis B in the Gambia Hepatitis Intervention Study (1986&#x02013;90) and in the nationwide immunisation program</article-title>. <source>BMC Infect Dis</source>
<year>2014</year>;<volume>14</volume>:<fpage>7</fpage>, <pub-id pub-id-type="doi">10.1186/1471-2334-14-7</pub-id>.<pub-id pub-id-type="pmid">24397793</pub-id>
</mixed-citation></ref><ref id="R15"><label>[15]</label><mixed-citation publication-type="journal"><name><surname>Chiang</surname><given-names>CJ</given-names></name>, <name><surname>Yang</surname><given-names>YW</given-names></name>, <name><surname>You</surname><given-names>SL</given-names></name>, <name><surname>Lai</surname><given-names>MS</given-names></name>, <name><surname>Chen</surname><given-names>CJ</given-names></name>. <article-title>Thirty-year outcomes of the national hepatitis B immunization program in Taiwan</article-title>. <source>JAMA</source>
<year>2013</year>;<volume>310</volume>(<issue>9</issue>):<fpage>974</fpage>&#x02013;<lpage>6</lpage>, <pub-id pub-id-type="doi">10.1001/jama.2013.276701</pub-id>.<pub-id pub-id-type="pmid">24002285</pub-id>
</mixed-citation></ref><ref id="R16"><label>[16]</label><mixed-citation publication-type="journal"><name><surname>Gessner</surname><given-names>BD</given-names></name>, <name><surname>Sutanto</surname><given-names>A</given-names></name>, <name><surname>Linehan</surname><given-names>M</given-names></name>, <name><surname>Djelantik</surname><given-names>IG</given-names></name>, <name><surname>Fletcher</surname><given-names>T</given-names></name>, <name><surname>Gerudug</surname><given-names>IK</given-names></name>, <etal/>
<article-title>Incidences of vaccine-preventable <italic toggle="yes">Haemophilus influenzae</italic> type b pneumonia and meningitis in Indonesian children: hamlet-randomised vaccine-probe trial</article-title>. <source>Lancet</source>
<year>2005</year>;<volume>365</volume>(<issue>9453</issue>):<fpage>43</fpage>&#x02013;<lpage>52</lpage>.<pub-id pub-id-type="pmid">15643700</pub-id>
</mixed-citation></ref><ref id="R17"><label>[17]</label><mixed-citation publication-type="journal"><name><surname>Zaman</surname><given-names>K</given-names></name>, <name><surname>Roy</surname><given-names>E</given-names></name>, <name><surname>Arifeen</surname><given-names>SE</given-names></name>, <name><surname>Rahman</surname><given-names>M</given-names></name>, <name><surname>Raqib</surname><given-names>R</given-names></name>, <name><surname>Wilson</surname><given-names>E</given-names></name>, <etal/>
<article-title>Effectiveness of maternal influenza immunization in mothers and infants</article-title>. <source>N Engl J Med</source>
<year>2008</year>;<volume>359</volume>(<issue>15</issue>):<fpage>1555</fpage>&#x02013;<lpage>64</lpage>, <pub-id pub-id-type="doi">10.1056/NEJMoa0708630</pub-id>.<pub-id pub-id-type="pmid">18799552</pub-id>
</mixed-citation></ref><ref id="R18"><label>[18]</label><mixed-citation publication-type="journal"><name><surname>Black</surname><given-names>SB</given-names></name>, <name><surname>Shinefield</surname><given-names>HR</given-names></name>, <name><surname>Hansen</surname><given-names>J</given-names></name>, <name><surname>Elvin</surname><given-names>L</given-names></name>, <name><surname>Laufer</surname><given-names>D</given-names></name>, <name><surname>Malinoski</surname><given-names>F</given-names></name>. <article-title>Postlicensure evaluation of the effectiveness of seven valent pneumococcal conjugate vaccine</article-title>. <source>Pediatr Infect Dis J</source>
<year>2001</year>;<volume>20</volume>(<issue>12</issue>):<fpage>1105</fpage>&#x02013;<lpage>7</lpage>.<pub-id pub-id-type="pmid">11740313</pub-id>
</mixed-citation></ref><ref id="R19"><label>[19]</label><mixed-citation publication-type="journal"><name><surname>Klugman</surname><given-names>KP</given-names></name>, <name><surname>Madhi</surname><given-names>SA</given-names></name>, <name><surname>Huebner</surname><given-names>RE</given-names></name>, <name><surname>Kohberger</surname><given-names>R</given-names></name>, <name><surname>Mbelle</surname><given-names>N</given-names></name>, <name><surname>Pierce</surname><given-names>N</given-names></name>, <etal/>
<article-title>A trial of a 9-valent pneumococcal conjugate vaccine in children with and those without HIV infection</article-title>. <source>N Engl J Med</source>
<year>2003</year>;<volume>349</volume>(<issue>14</issue>):<fpage>1341</fpage>&#x02013;<lpage>8</lpage>.<pub-id pub-id-type="pmid">14523142</pub-id>
</mixed-citation></ref><ref id="R20"><label>[20]</label><mixed-citation publication-type="journal"><name><surname>Cutts</surname><given-names>FT</given-names></name>, <name><surname>Zaman</surname><given-names>SM</given-names></name>, <name><surname>Enwere</surname><given-names>G</given-names></name>, <name><surname>Jaffar</surname><given-names>S</given-names></name>, <name><surname>Levine</surname><given-names>OS</given-names></name>, <name><surname>Okoko</surname><given-names>JB</given-names></name>, <etal/>
<article-title>Efficacy of nine-valent pneumococcal conjugate vaccine against pneumonia and invasive pneumococcal disease in The Gambia: randomised, double-blind, placebo-controlled trial</article-title>. <source>Lancet</source>
<year>2005</year>;<volume>365</volume>(<issue>9465</issue>):<fpage>1139</fpage>&#x02013;<lpage>46</lpage>.<pub-id pub-id-type="pmid">15794968</pub-id>
</mixed-citation></ref><ref id="R21"><label>[21]</label><mixed-citation publication-type="journal"><name><surname>Hansen</surname><given-names>J</given-names></name>, <name><surname>Black</surname><given-names>S</given-names></name>, <name><surname>Shinefield</surname><given-names>H</given-names></name>, <name><surname>Cherian</surname><given-names>T</given-names></name>, <name><surname>Benson</surname><given-names>J</given-names></name>, <name><surname>Fireman</surname><given-names>B</given-names></name>, <etal/>
<article-title>Effectiveness of heptavalent pneumococcal conjugate vaccine in children younger than 5 years of age for prevention of pneumonia: updated analysis using World Health Organization standardized interpretation of chest radiographs</article-title>. <source>Pediatr Infect Dis J</source>
<year>2006</year>;<volume>25</volume>(<issue>9</issue>):<fpage>779</fpage>&#x02013;<lpage>81</lpage>.<pub-id pub-id-type="pmid">16940833</pub-id>
</mixed-citation></ref><ref id="R22"><label>[22]</label><mixed-citation publication-type="journal"><name><surname>Lucero</surname><given-names>MG</given-names></name>, <name><surname>Dulalia</surname><given-names>VE</given-names></name>, <name><surname>Nillos</surname><given-names>LT</given-names></name>, <name><surname>Williams</surname><given-names>G</given-names></name>, <name><surname>Parre&#x000f1;o</surname><given-names>RA</given-names></name>, <name><surname>Nohynek</surname><given-names>H</given-names></name>, <etal/>
<article-title>Pneumococcal conjugate vaccines for preventing vaccine-type invasive pneumococcal disease and X-ray defined pneumonia in children less than two years of age</article-title>. <source>Cochrane Database Syst Rev</source>
<year>2009</year>;<volume>4</volume>:<comment>CD004977</comment>, <pub-id pub-id-type="doi">10.1002/14651858.CD004977</pub-id>.</mixed-citation></ref><ref id="R23"><label>[23]</label><mixed-citation publication-type="journal"><name><surname>Gessner</surname><given-names>BD</given-names></name>, <name><surname>Halloran</surname><given-names>ME</given-names></name>, <name><surname>Khan</surname><given-names>I</given-names></name>. <article-title>The case for a typhoid vaccine probe study and overview of design elements</article-title>. <source>Vaccine</source>
<year>2015</year>;<volume>33</volume>(<issue>Suppl. 3</issue>):<fpage>C30</fpage>&#x02013;<lpage>5</lpage>, <pub-id pub-id-type="doi">10.1016/j.vaccine.2015.03.085</pub-id>.<pub-id pub-id-type="pmid">25912286</pub-id>
</mixed-citation></ref><ref id="R24"><label>[24]</label><mixed-citation publication-type="journal"><name><surname>Kim</surname><given-names>SY</given-names></name>, <name><surname>Goldie</surname><given-names>SJ</given-names></name>. <article-title>Cost-effectiveness analyses of vaccination programmes: a focused review of modelling approaches</article-title>. <source>Pharmacoeconomics</source>
<year>2008</year>;<volume>26</volume>(<issue>3</issue>):<fpage>191</fpage>&#x02013;<lpage>215</lpage>.<pub-id pub-id-type="pmid">18282015</pub-id>
</mixed-citation></ref><ref id="R25"><label>[25]</label><mixed-citation publication-type="journal"><name><surname>Brisson</surname><given-names>M</given-names></name>, <name><surname>Edmunds</surname><given-names>WJ</given-names></name>. <article-title>Economic evaluation of vaccination programs: the impact of herd-immunity</article-title>. <source>Med Decis Making</source>
<year>2003</year>;<volume>23</volume>(<issue>1</issue>):<fpage>76</fpage>&#x02013;<lpage>82</lpage>.<pub-id pub-id-type="pmid">12583457</pub-id>
</mixed-citation></ref><ref id="R26"><label>[26]</label><mixed-citation publication-type="journal"><name><surname>Feikin</surname><given-names>DR</given-names></name>, <name><surname>Kagucia</surname><given-names>EW</given-names></name>, <name><surname>Loo</surname><given-names>JD</given-names></name>, <name><surname>Link-Gelles</surname><given-names>R</given-names></name>, <name><surname>Puhan</surname><given-names>MA</given-names></name>, <name><surname>Cherian</surname><given-names>T</given-names></name>, <etal/>
<article-title>Serotype-specific changes in invasive pneumococcal disease after pneumococcal conjugate vaccine introduction: a pooled analysis of multiple surveillance sites</article-title>. <source>PLoS Med</source>
<year>2013</year>;<volume>10</volume>(<issue>9</issue>):<fpage>e1001517</fpage>, <pub-id pub-id-type="doi">10.1371/journal.pmed.1001517</pub-id>.<pub-id pub-id-type="pmid">24086113</pub-id>
</mixed-citation></ref><ref id="R27"><label>[27]</label><mixed-citation publication-type="journal"><name><surname>Longini</surname><given-names>IM</given-names><suffix>Jr</suffix></name>, <name><surname>Nizam</surname><given-names>A</given-names></name>, <name><surname>Ali</surname><given-names>M</given-names></name>, <name><surname>Yunus</surname><given-names>M</given-names></name>, <name><surname>Shenvi</surname><given-names>N</given-names></name>, <name><surname>Clemens</surname><given-names>JD</given-names></name>. <article-title>Controlling endemic cholera with oral vaccines</article-title>. <source>PLoS Med</source>
<year>2007</year>;<volume>4</volume>(<issue>11</issue>):<fpage>e336</fpage>.<pub-id pub-id-type="pmid">18044983</pub-id>
</mixed-citation></ref><ref id="R28"><label>[28]</label><mixed-citation publication-type="journal"><name><surname>Palmu</surname><given-names>AA</given-names></name>, <name><surname>Jokinen</surname><given-names>J</given-names></name>, <name><surname>Borys</surname><given-names>D</given-names></name>, <name><surname>Nieminen</surname><given-names>H</given-names></name>, <name><surname>Ruokokoski</surname><given-names>E</given-names></name>, <name><surname>Siira</surname><given-names>L</given-names></name>, <etal/>
<article-title>Effectiveness of the ten-valent pneumococcal <italic toggle="yes">Haemophilus influenzae</italic> protein D conjugate vaccine (PHiD-CV10) against invasive pneumococcal disease: a cluster randomised trial</article-title>. <source>Lancet</source>
<year>2013</year>;<volume>381</volume>(<issue>9862</issue>):<fpage>214</fpage>&#x02013;<lpage>22</lpage>, <pub-id pub-id-type="doi">10.1016/S0140-6736(12)61854-6</pub-id>.<pub-id pub-id-type="pmid">23158882</pub-id>
</mixed-citation></ref><ref id="R29"><label>[29]</label><mixed-citation publication-type="journal"><name><surname>Palmu</surname><given-names>AA</given-names></name>, <name><surname>Jokinen</surname><given-names>J</given-names></name>, <name><surname>Nieminen</surname><given-names>H</given-names></name>, <name><surname>Syrj&#x000e4;nen</surname><given-names>R</given-names></name>, <name><surname>Ruokokoski</surname><given-names>E</given-names></name>, <name><surname>Puumalainen</surname><given-names>T</given-names></name>, <etal/>
<article-title>Vaccine effectiveness of the pneumococcal <italic toggle="yes">Haemophilus influenzae</italic> protein D conjugate vaccine (PHiD-CV10) against clinically suspected invasive pneumococcal disease: a cluster-randomised trial</article-title>. <source>Lancet Respir Med</source>
<year>2014</year>;<volume>2</volume>(<issue>9</issue>):<fpage>717</fpage>&#x02013;<lpage>27</lpage>, <pub-id pub-id-type="doi">10.1016/S2213-2600(14)70139-0</pub-id>.<pub-id pub-id-type="pmid">25127244</pub-id>
</mixed-citation></ref><ref id="R30"><label>[30]</label><mixed-citation publication-type="journal"><name><surname>Jokinen</surname><given-names>J</given-names></name>, <name><surname>Rinta-Kokko</surname><given-names>H</given-names></name>, <name><surname>Siira</surname><given-names>L</given-names></name>, <name><surname>Palmu</surname><given-names>AA</given-names></name>, <name><surname>Virtanen</surname><given-names>MJ</given-names></name>, <name><surname>Nohynek</surname><given-names>H</given-names></name>, <etal/>
<article-title>Impact of ten-valent pneumococcal conjugate vaccination on invasive pneumococcal disease in Finnish children &#x02013; a population-based study</article-title>. <source>PLOS ONE</source>
<year>2015</year>;<volume>10</volume>(<issue>3</issue>):<fpage>e0120290</fpage>, <pub-id pub-id-type="doi">10.1371/journal.pone.0120290</pub-id>.<pub-id pub-id-type="pmid">25781031</pub-id>
</mixed-citation></ref><ref id="R31"><label>[31]</label><mixed-citation publication-type="journal"><name><surname>Payne</surname><given-names>DC</given-names></name>, <name><surname>Boom</surname><given-names>JA</given-names></name>, <name><surname>Staat</surname><given-names>MA</given-names></name>, <name><surname>Edwards</surname><given-names>KM</given-names></name>, <name><surname>Szilagyi</surname><given-names>PG</given-names></name>, <name><surname>Klein</surname><given-names>EJ</given-names></name>, <etal/>
<article-title>Effectiveness of pentavalent and monovalent rotavirus vaccines in concurrent use among US children &#x0003c;5 years of age, 2009&#x02013;2011</article-title>. <source>Clin Infect Dis</source>
<year>2013</year>;<volume>57</volume>(<issue>1</issue>):<fpage>13</fpage>&#x02013;<lpage>20</lpage>, <pub-id pub-id-type="doi">10.1093/cid/cit164</pub-id>.<pub-id pub-id-type="pmid">23487388</pub-id>
</mixed-citation></ref><ref id="R32"><label>[32]</label><mixed-citation publication-type="journal"><name><surname>Cortese</surname><given-names>MM</given-names></name>, <name><surname>Immergluck</surname><given-names>LC</given-names></name>, <name><surname>Held</surname><given-names>M</given-names></name>, <name><surname>Jain</surname><given-names>S</given-names></name>, <name><surname>Chan</surname><given-names>T</given-names></name>, <name><surname>Grizas</surname><given-names>AP</given-names></name>, <etal/>
<article-title>Effectiveness of monovalent and pentavalent rotavirus vaccine</article-title>. <source>Pediatrics</source>
<year>2013</year>;<volume>132</volume>(<issue>1</issue>):<fpage>e25</fpage>&#x02013;<lpage>33</lpage>, <pub-id pub-id-type="doi">10.1542/peds.2012-3804</pub-id>.<pub-id pub-id-type="pmid">23776114</pub-id>
</mixed-citation></ref><ref id="R33"><label>[33]</label><mixed-citation publication-type="journal"><name><surname>Cortese</surname><given-names>MM</given-names></name>, <name><surname>Leblanc</surname><given-names>J</given-names></name>, <name><surname>White</surname><given-names>KE</given-names></name>, <name><surname>Jerris</surname><given-names>RC</given-names></name>, <name><surname>Stinchfield</surname><given-names>P</given-names></name>, <name><surname>Preston</surname><given-names>KL</given-names></name>, <etal/>
<article-title>Leveraging state immunization information systems to measure the effectiveness of rotavirus vaccine</article-title>. <source>Pediatrics</source>
<year>2011</year>;<volume>128</volume>(<issue>6</issue>):<fpage>e1474</fpage>&#x02013;<lpage>81</lpage>, <pub-id pub-id-type="doi">10.1542/peds.2011-1006</pub-id>.<pub-id pub-id-type="pmid">22084328</pub-id>
</mixed-citation></ref><ref id="R34"><label>[34]</label><mixed-citation publication-type="journal"><name><surname>Staat</surname><given-names>MA</given-names></name>, <name><surname>Payne</surname><given-names>DC</given-names></name>, <name><surname>Donauer</surname><given-names>S</given-names></name>, <name><surname>Weinberg</surname><given-names>GA</given-names></name>, <name><surname>Edwards</surname><given-names>KM</given-names></name>, <name><surname>Szilagyi</surname><given-names>PG</given-names></name>, <etal/>
<article-title>Effectiveness of pentavalent rotavirus vaccine against severe disease</article-title>. <source>Pediatrics</source>
<year>2011</year>;<volume>128</volume>(<issue>2</issue>):<fpage>e267</fpage>&#x02013;<lpage>75</lpage>, <pub-id pub-id-type="doi">10.1542/peds.2010-3722</pub-id>.<pub-id pub-id-type="pmid">21768317</pub-id>
</mixed-citation></ref><ref id="R35"><label>[35]</label><mixed-citation publication-type="journal"><name><surname>Boom</surname><given-names>JA</given-names></name>, <name><surname>Tate</surname><given-names>JE</given-names></name>, <name><surname>Sahni</surname><given-names>LC</given-names></name>, <name><surname>Rench</surname><given-names>MA</given-names></name>, <name><surname>Hull</surname><given-names>JJ</given-names></name>, <name><surname>Gentsch</surname><given-names>JR</given-names></name>, <etal/>
<article-title>Effectiveness of pentavalent rotavirus vaccine in a large urban population in the United States</article-title>. <source>Pediatrics</source>
<year>2010</year>;<volume>125</volume>(<issue>2</issue>):<fpage>e199</fpage>&#x02013;<lpage>207</lpage>, <pub-id pub-id-type="doi">10.1542/peds.2009-1021</pub-id>.<pub-id pub-id-type="pmid">20083525</pub-id>
</mixed-citation></ref><ref id="R36"><label>[36]</label><mixed-citation publication-type="journal"><name><surname>Vesikari</surname><given-names>T</given-names></name>, <name><surname>Karvonen</surname><given-names>A</given-names></name>, <name><surname>Prymula</surname><given-names>R</given-names></name>, <name><surname>Schuster</surname><given-names>V</given-names></name>, <name><surname>Tejedor</surname><given-names>JC</given-names></name>, <name><surname>Cohen</surname><given-names>R</given-names></name>, <etal/>
<article-title>Efficacy of human rotavirus vaccine against rotavirus gastroenteritis during the first 2 years of life in European infants: randomised, double-blind controlled study</article-title>. <source>Lancet</source>
<year>2007</year>;<volume>370</volume>(<issue>9601</issue>):<fpage>1757</fpage>&#x02013;<lpage>63</lpage>.<pub-id pub-id-type="pmid">18037080</pub-id>
</mixed-citation></ref><ref id="R37"><label>[37]</label><mixed-citation publication-type="journal"><name><surname>Vesikari</surname><given-names>T</given-names></name>, <name><surname>Matson</surname><given-names>DO</given-names></name>, <name><surname>Dennehy</surname><given-names>P</given-names></name>, <name><surname>Van Damme</surname><given-names>P</given-names></name>, <name><surname>Santosham</surname><given-names>M</given-names></name>, <name><surname>Rodriguez</surname><given-names>Z</given-names></name>, <etal/>
<article-title>Safety and efficacy of a pentavalent human-bovine (WC3) reassortant rotavirus vaccine</article-title>. <source>N Engl J Med</source>
<year>2006</year>;<volume>354</volume>(<issue>1</issue>):<fpage>23</fpage>&#x02013;<lpage>33</lpage>.<pub-id pub-id-type="pmid">16394299</pub-id>
</mixed-citation></ref><ref id="R38"><label>[38]</label><mixed-citation publication-type="journal"><name><surname>Madhi</surname><given-names>SA</given-names></name>, <name><surname>Cunliffe</surname><given-names>NA</given-names></name>, <name><surname>Steele</surname><given-names>D</given-names></name>, <name><surname>Witte</surname><given-names>D</given-names></name>, <name><surname>Kirsten</surname><given-names>M</given-names></name>, <name><surname>Louw</surname><given-names>C</given-names></name>, <etal/>
<article-title>Effect of human rotavirus vaccine on severe diarrhea in African infants</article-title>. <source>N Engl J Med</source>
<year>2010</year>;<volume>362</volume>(<issue>4</issue>):<fpage>289</fpage>&#x02013;<lpage>98</lpage>, <pub-id pub-id-type="doi">10.1056/NEJMoa0904797</pub-id>.<pub-id pub-id-type="pmid">20107214</pub-id>
</mixed-citation></ref><ref id="R39"><label>[39]</label><mixed-citation publication-type="journal"><name><surname>Zaman</surname><given-names>K</given-names></name>, <name><surname>Dang</surname><given-names>DA</given-names></name>, <name><surname>Victor</surname><given-names>JC</given-names></name>, <name><surname>Shin</surname><given-names>S</given-names></name>, <name><surname>Yunus</surname><given-names>M</given-names></name>, <name><surname>Dallas</surname><given-names>MJ</given-names></name>, <etal/>
<article-title>Efficacy of pentavalent rotavirus vaccine against severe rotavirus gastroenteritis in infants in developing countries in Asia: a randomized, double-blind, placebo-controlled trial</article-title>. <source>Lancet</source>
<year>2010</year>;<volume>376</volume>(<issue>9741</issue>):<fpage>615</fpage>&#x02013;<lpage>23</lpage>, <pub-id pub-id-type="doi">10.1016/S0140-6736(10)60755-6</pub-id>.<pub-id pub-id-type="pmid">20692031</pub-id>
</mixed-citation></ref><ref id="R40"><label>[40]</label><mixed-citation publication-type="journal"><name><surname>Armah</surname><given-names>GE</given-names></name>, <name><surname>Sow</surname><given-names>SO</given-names></name>, <name><surname>Breiman</surname><given-names>RF</given-names></name>, <name><surname>Dallas</surname><given-names>MJ</given-names></name>, <name><surname>Tapia</surname><given-names>MD</given-names></name>, <name><surname>Feikin</surname><given-names>DR</given-names></name>, <etal/>
<article-title>Efficacy of pentavalent rotavirus vaccine against severe rotavirus gastroenteritis in infants in developing countries in sub-Saharan Africa: a randomized, double-blind, placebo-controlled trial</article-title>. <source>Lancet</source>
<year>2010</year>;<volume>376</volume>(<issue>9741</issue>):<fpage>606</fpage>&#x02013;<lpage>14</lpage>, <pub-id pub-id-type="doi">10.1016/S0140-6736(10)60889-6</pub-id>.<pub-id pub-id-type="pmid">20692030</pub-id>
</mixed-citation></ref><ref id="R41"><label>[41]</label><mixed-citation publication-type="webpage"><collab>WHO</collab>. <source>World Malaria Report 2014</source>. <comment>Available at: <ext-link xlink:href="http://www.who.int/malaria/publications/world_malaria_report_2014/report/en/" ext-link-type="uri">http://www.who.int/malaria/publications/world_malaria_report_2014/report/en/</ext-link></comment> [<comment>accessed</comment>
<date-in-citation>6.10.15</date-in-citation>].</mixed-citation></ref><ref id="R42"><label>[42]</label><mixed-citation publication-type="journal"><name><surname>Guerra</surname><given-names>CA</given-names></name>, <name><surname>Gikandi</surname><given-names>PW</given-names></name>, <name><surname>Tatem</surname><given-names>AJ</given-names></name>, <name><surname>Noor</surname><given-names>AM</given-names></name>, <name><surname>Smith</surname><given-names>DL</given-names></name>, <name><surname>Hay</surname><given-names>SI</given-names></name>, <etal/>
<article-title>The limits and intensity of <italic toggle="yes">Plasmodium falciparum</italic> transmission: implications for malaria control and elimination worldwide</article-title>. <source>PLoS Med</source>
<year>2008</year>;<volume>5</volume>(<issue>2</issue>):<fpage>e38</fpage>, <pub-id pub-id-type="doi">10.1371/journal.pmed.0050038</pub-id>.<pub-id pub-id-type="pmid">18303939</pub-id>
</mixed-citation></ref><ref id="R43"><label>[43]</label><mixed-citation publication-type="journal"><name><surname>Agnandji</surname><given-names>ST</given-names></name>, <name><surname>Lell</surname><given-names>B</given-names></name>, <name><surname>Soulanoudjingar</surname><given-names>SS</given-names></name>, <name><surname>Fernandes</surname><given-names>JF</given-names></name>, <name><surname>Abossolo</surname><given-names>BP</given-names></name>, <name><surname>Conzelmann</surname><given-names>C</given-names></name>, <etal/>
<article-title>First results of phase 3 trial of RTS,S/AS01 malaria vaccine in African children</article-title>. <source>N Engl J Med</source>
<year>2011</year>;<volume>365</volume>:<fpage>1863</fpage>&#x02013;<lpage>75</lpage>, <pub-id pub-id-type="doi">10.1056/NEJMoa1102287</pub-id>.<pub-id pub-id-type="pmid">22007715</pub-id>
</mixed-citation></ref><ref id="R44"><label>[44]</label><mixed-citation publication-type="journal"><collab>The RTS,S Clinical Trials Partnership</collab>
<name><surname>Agnandji</surname><given-names>ST</given-names></name>, <name><surname>Lell</surname><given-names>B</given-names></name>, <name><surname>Fernandes</surname><given-names>JF</given-names></name>, <name><surname>Abossolo</surname><given-names>BP</given-names></name>, <name><surname>Methogo</surname><given-names>BG</given-names></name>, <etal/>
<article-title>A phase 3 trial of RTS,S/AS01 malaria vaccine in African infants</article-title>. <source>N Engl J Med</source>
<year>2012</year>;<volume>367</volume>:<fpage>2284</fpage>&#x02013;<lpage>95</lpage>, <pub-id pub-id-type="doi">10.1056/NEJMoa1208394</pub-id>.<pub-id pub-id-type="pmid">23136909</pub-id>
</mixed-citation></ref><ref id="R45"><label>[45]</label><mixed-citation publication-type="journal"><name><surname>Gessner</surname><given-names>BD</given-names></name>, <name><surname>Feikin</surname><given-names>DR</given-names></name>. <article-title>Vaccine preventable disease incidence as a complement to vaccine efficacy for setting vaccine policy</article-title>. <source>Vaccine</source>
<year>2014</year>;<volume>32</volume>:<fpage>3133</fpage>&#x02013;<lpage>8</lpage>, <pub-id pub-id-type="doi">10.1016/j.vaccine.2014.04.019</pub-id>.<pub-id pub-id-type="pmid">24731817</pub-id>
</mixed-citation></ref><ref id="R46"><label>[46]</label><mixed-citation publication-type="journal"><name><surname>Scott</surname><given-names>JA</given-names></name>, <name><surname>Berkley</surname><given-names>JA</given-names></name>, <name><surname>Mwangi</surname><given-names>I</given-names></name>, <name><surname>Ochola</surname><given-names>L</given-names></name>, <name><surname>Uyoga</surname><given-names>S</given-names></name>, <name><surname>Macharia</surname><given-names>A</given-names></name>, <etal/>
<article-title>Relation between falciparum malaria and bacteraemia in Kenyan children: a population-based, case-control study and a longitudinal study</article-title>. <source>Lancet</source>
<year>2011</year>;<volume>378</volume>(<issue>9799</issue>):<fpage>1316</fpage>&#x02013;<lpage>23</lpage>, <pub-id pub-id-type="doi">10.1016/S0140-6736(11)60888-X</pub-id>.<pub-id pub-id-type="pmid">21903251</pub-id>
</mixed-citation></ref><ref id="R47"><label>[47]</label><mixed-citation publication-type="journal"><name><surname>O&#x02019;Meara</surname><given-names>WP</given-names></name>, <name><surname>Bejon</surname><given-names>P</given-names></name>, <name><surname>Mwangi</surname><given-names>TW</given-names></name>, <name><surname>Okiro</surname><given-names>EA</given-names></name>, <name><surname>Peshu</surname><given-names>N</given-names></name>, <name><surname>Snow</surname><given-names>RW</given-names></name>, <etal/>
<article-title>Effect of a fall in malaria transmission on morbidity and mortality in Kilifi</article-title>, <source>Kenya. Lancet</source>
<year>2008</year>;<volume>372</volume>(<issue>9649</issue>):<fpage>1555</fpage>&#x02013;<lpage>62</lpage>, <pub-id pub-id-type="doi">10.1016/S0140-6736(08)61655-4</pub-id>.<pub-id pub-id-type="pmid">18984188</pub-id>
</mixed-citation></ref><ref id="R48"><label>[48]</label><mixed-citation publication-type="journal"><name><surname>Kleinschmidt</surname><given-names>I</given-names></name>, <name><surname>Schwabe</surname><given-names>C</given-names></name>, <name><surname>Benavente</surname><given-names>L</given-names></name>, <name><surname>Torrez</surname><given-names>M</given-names></name>, <name><surname>Ridl</surname><given-names>FC</given-names></name>, <name><surname>Segura</surname><given-names>JL</given-names></name>, <etal/>
<article-title>Marked increase in child survival after four years of intensive malaria control</article-title>. <source>Am J Trop Med Hyg</source>
<year>2009</year>;<volume>80</volume>(<issue>6</issue>):<fpage>882</fpage>&#x02013;<lpage>8</lpage>.<pub-id pub-id-type="pmid">19478243</pub-id>
</mixed-citation></ref><ref id="R49"><label>[49]</label><mixed-citation publication-type="journal"><name><surname>Gubler</surname><given-names>DJ</given-names></name>. <article-title>Dengue, urbanization and globalization: the unholy trinity of the 21(st) century</article-title>. <source>Trop Med Health</source>
<year>2011</year>;<volume>39</volume>(<issue>4 Suppl.</issue>):<fpage>3</fpage>&#x02013;<lpage>11</lpage>, <pub-id pub-id-type="doi">10.2149/tmh.2011-S05</pub-id>.</mixed-citation></ref><ref id="R50"><label>[50]</label><mixed-citation publication-type="journal"><name><surname>Messina</surname><given-names>JP</given-names></name>, <name><surname>Brady</surname><given-names>OJ</given-names></name>, <name><surname>Pigott</surname><given-names>DM</given-names></name>, <name><surname>Brownstein</surname><given-names>JS</given-names></name>, <name><surname>Hoen</surname><given-names>AG</given-names></name>, <name><surname>Hay</surname><given-names>SI</given-names></name>. <article-title>A global compendium of human dengue virus occurrence</article-title>. <source>Sci Data</source>
<year>2014</year>;<volume>1</volume>:<fpage>140004</fpage>, <pub-id pub-id-type="doi">10.1038/sdata.2014.4</pub-id>.<pub-id pub-id-type="pmid">25977762</pub-id>
</mixed-citation></ref><ref id="R51"><label>[51]</label><mixed-citation publication-type="journal"><name><surname>Horstick</surname><given-names>O</given-names></name>, <name><surname>Tozan</surname><given-names>Y</given-names></name>, <name><surname>Wilder-Smith</surname><given-names>A</given-names></name>. <article-title>Reviewing dengue: still a neglected tropical disease?</article-title>
<source>PLoS Negl Trop Dis</source>
<year>2015</year>;<volume>9</volume>(<issue>4</issue>):<fpage>e0003632</fpage>, <pub-id pub-id-type="doi">10.1371/journal.pntd.0003632</pub-id>.<pub-id pub-id-type="pmid">25928673</pub-id>
</mixed-citation></ref><ref id="R52"><label>[52]</label><mixed-citation publication-type="journal"><name><surname>Bhatt</surname><given-names>S</given-names></name>, <name><surname>Gething</surname><given-names>PW</given-names></name>, <name><surname>Brady</surname><given-names>OJ</given-names></name>, <name><surname>Messina</surname><given-names>JP</given-names></name>, <name><surname>Farlow</surname><given-names>AW</given-names></name>, <name><surname>Moyes</surname><given-names>CL</given-names></name>, <etal/>
<article-title>The global distribution and burden of dengue</article-title>. <source>Nature</source>
<year>2013</year>;<volume>496</volume>(<issue>7446</issue>):<fpage>504</fpage>&#x02013;<lpage>7</lpage>, <pub-id pub-id-type="doi">10.1038/nature12060</pub-id>.<pub-id pub-id-type="pmid">23563266</pub-id>
</mixed-citation></ref><ref id="R53"><label>[53]</label><mixed-citation publication-type="journal"><name><surname>Hadinegoro</surname><given-names>SR</given-names></name>, <name><surname>Arredondo-Garc&#x000ed;a</surname><given-names>JL</given-names></name>, <name><surname>Capeding</surname><given-names>MR</given-names></name>, <name><surname>Deseda</surname><given-names>C</given-names></name>, <name><surname>Chotpitayasunondh</surname><given-names>T</given-names></name>, <name><surname>Dietze</surname><given-names>R</given-names></name>, <etal/>
<article-title>Efficacy and long-term safety of a dengue vaccine in regions of endemic disease</article-title>. <source>N Engl J Med</source>
<year>2015</year>
<comment>[Epub ahead of print].</comment></mixed-citation></ref><ref id="R54"><label>[54]</label><mixed-citation publication-type="journal"><name><surname>Olkowski</surname><given-names>S</given-names></name>, <name><surname>Forshey</surname><given-names>BM</given-names></name>, <name><surname>Morrison</surname><given-names>AC</given-names></name>, <name><surname>Rocha</surname><given-names>C</given-names></name>, <name><surname>Vilcarromero</surname><given-names>S</given-names></name>, <name><surname>Halsey</surname><given-names>ES</given-names></name>, <etal/>
<article-title>Reduced risk of disease during postsecondary dengue virus infections</article-title>. <source>J Infect Dis</source>
<year>2013</year>;<volume>208</volume>(<issue>6</issue>):<fpage>1026</fpage>&#x02013;<lpage>33</lpage>, <pub-id pub-id-type="doi">10.1093/infdis/jit273</pub-id>.<pub-id pub-id-type="pmid">23776195</pub-id>
</mixed-citation></ref><ref id="R55"><label>[55]</label><mixed-citation publication-type="journal"><name><surname>Gibbons</surname><given-names>RV</given-names></name>, <name><surname>Kalanarooj</surname><given-names>S</given-names></name>, <name><surname>Jarman</surname><given-names>RG</given-names></name>, <name><surname>Nisalak</surname><given-names>A</given-names></name>, <name><surname>Vaughn</surname><given-names>DW</given-names></name>, <name><surname>Endy</surname><given-names>TP</given-names></name>, <etal/>
<article-title>Analysis of repeat hospital admissions for dengue to estimate the frequency of third or fourth dengue infections resulting in admissions and dengue hemorrhagic fever, and serotype sequences</article-title>. <source>Am J Trop Med Hyg</source>
<year>2007</year>;<volume>77</volume>(<issue>5</issue>):<fpage>910</fpage>&#x02013;<lpage>3</lpage>.<pub-id pub-id-type="pmid">17984352</pub-id>
</mixed-citation></ref><ref id="R56"><label>[56]</label><mixed-citation publication-type="journal"><name><surname>Gubler</surname><given-names>D</given-names></name>
<article-title>The partnership for dengue control &#x02013; a new global alliance for the prevention and control of dengue</article-title>. <source>Vaccine</source>
<year>2015</year>;<volume>33</volume>(<issue>10</issue>):<fpage>1233</fpage>, <pub-id pub-id-type="doi">10.1016/j.vaccine.2015.01.002</pub-id>.<pub-id pub-id-type="pmid">25597939</pub-id>
</mixed-citation></ref><ref id="R57"><label>[57]</label><mixed-citation publication-type="webpage"><collab>27th Annual EuroMeeting</collab>. <source>Development, innovation access, and patient safety</source>. <comment>Available at: <ext-link xlink:href="https://www.diaglobal.org/tools/content.aspx?type=eopdf&#x00026;file=%2Fproductfiles%2F3175365%2F15101pgm.pdf" ext-link-type="uri">https://www.diaglobal.org/tools/content.aspx?type=eopdf&#x00026;file=%2Fproductfiles%2F3175365%2F15101pgm.pdf</ext-link></comment> [<comment>accessed</comment>
<date-in-citation>6.10.15</date-in-citation>].</mixed-citation></ref></ref-list></back><floats-group><fig position="float" id="F1"><label>Fig. 1.</label><caption><p id="P153">Types of vaccine effectiveness as reported by Halloran [<xref rid="R6" ref-type="bibr">6</xref>], kindly provided by Ira Longini. AR, attack rates of disease; VE, vaccine effectiveness. Presence of unvaccinated individuals in the intervention population is explained by a coverage rate of less than 100%, which is in general never reached.</p></caption><graphic xlink:href="nihms-2017404-f0001" position="float"/></fig><table-wrap position="float" id="T1" orientation="landscape"><label>Table 1</label><caption><p id="P154">Efficacy of pneumococcal conjugate vaccines (PCV) in reducing radiologically confirmed pneumonia in children.</p></caption><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/><col align="left" valign="middle" span="1"/></colgroup><thead><tr><th align="left" valign="middle" rowspan="1" colspan="1">Author (year)</th><th align="left" valign="middle" rowspan="1" colspan="1">Country</th><th align="left" valign="middle" rowspan="1" colspan="1">PCV vaccine</th><th align="left" valign="middle" rowspan="1" colspan="1">Study design (area)</th><th align="right" valign="middle" rowspan="1" colspan="1">PCV-VE (95%CI)</th></tr></thead><tbody><tr><td align="left" valign="middle" rowspan="1" colspan="1">Black (2001)</td><td align="left" valign="middle" rowspan="1" colspan="1">Northern California, USA</td><td align="left" valign="middle" rowspan="1" colspan="1">PCV-7</td><td align="left" valign="middle" rowspan="1" colspan="1">DBRCT (urban)</td><td align="right" valign="middle" rowspan="1" colspan="1">30% (11&#x02013;46)</td></tr><tr><td align="left" valign="middle" rowspan="1" colspan="1">Hansen (2006)</td><td align="left" valign="middle" rowspan="1" colspan="1">USA</td><td align="left" valign="middle" rowspan="1" colspan="1">PCV-7</td><td align="left" valign="middle" rowspan="1" colspan="1">Cluster randomized (rural)</td><td align="right" valign="middle" rowspan="1" colspan="1">&#x02212;2%</td></tr><tr><td align="left" valign="middle" rowspan="1" colspan="1">Klugman (2003)</td><td align="left" valign="middle" rowspan="1" colspan="1">South Africa</td><td align="left" valign="middle" rowspan="1" colspan="1">PCV-9</td><td align="left" valign="middle" rowspan="1" colspan="1">DBRCT (urban)</td><td align="right" valign="middle" rowspan="1" colspan="1">25% (4&#x02013;40)</td></tr><tr><td align="left" valign="middle" rowspan="1" colspan="1">Cutts (2005)</td><td align="left" valign="middle" rowspan="1" colspan="1">The Gambia</td><td align="left" valign="middle" rowspan="1" colspan="1">PCV-9</td><td align="left" valign="middle" rowspan="1" colspan="1">DBRCT (rural)</td><td align="right" valign="middle" rowspan="1" colspan="1">37% (25&#x02013;48)</td></tr><tr><td align="left" valign="middle" rowspan="1" colspan="1">Lucero (2009)</td><td align="left" valign="middle" rowspan="1" colspan="1">Philippines</td><td align="left" valign="middle" rowspan="1" colspan="1">PCV-11</td><td align="left" valign="middle" rowspan="1" colspan="1">DBRT (rural)</td><td align="right" valign="middle" rowspan="1" colspan="1">22.9 (&#x02212;1 to 41)</td></tr></tbody></table><table-wrap-foot><fn id="TFN1"><p id="P155">DBRCT, double blind randomized controlled trial.</p></fn></table-wrap-foot></table-wrap><boxed-text id="BX1" position="float"><caption><title>Key messages</title></caption><list list-type="bullet" id="L6"><list-item><p id="P156">We are in an era of a new paradigm in vaccine evaluation where all aspects of public health value of vaccines beyond efficacy should be evaluated and incorporated into public health decision making.</p></list-item><list-item><p id="P157">Public health priorities are focused on reducing disease burden in a population, and various factors beyond efficacy determine public health impact. Therefore, preventable disease burden should be integrated into regulatory and policy discussions pre- and post-licensure. This could be achieved by:
<list list-type="bullet" id="L8"><list-item><p id="P158">Additional measures: VPDI, overall vaccine efficacy, indirect effects</p></list-item><list-item><p id="P159">Additional outcomes: all cause under age five years mortality or non-etiologically confirmed clinical syndromes (e.g. fever for typhoid, dengue, malaria or radiologically confirmed pneumonia for PCV)</p></list-item><list-item><p id="P160">Additional methods: dynamic mathematical models, probe studies</p></list-item></list></p></list-item></list></boxed-text></floats-group></article>