A search strategy was developed and used in MEDLINE. It included both keyword and free text (title and abstract) terms relating to ‘vaccines’ or ‘immunization’ AND ‘policy’ or ‘policy making’ AND ‘introduction’ or ‘adoption’ from March 2010 to June 2017 with an additional update in August 2020 (Appendix 1). This strategy was then adapted for use in the additional five databases: EMBASE, Global Health, Cochrane Library, CINAHL, and Scopus. The searches did not include any language limitations although they were conducted in English only. Abstracts and full text articles were screened, and data were abstracted in English.

Two reviewers for the March 2010 to June 2017 literature search and one reviewer for the June 2017 to August 2020 literature search independently screened titles and abstracts of the references and assessed the full-text articles for eligibility, using the following exclusion criteria: 1) Did not focus on human vaccination; 2) Did not focus on policy decision-making around vaccine adoption (i.e., considered decision-making at the clinical, individual level, or focused on implementation issues only); 3) Focused on hypothetical vaccines (e.g., HIV vaccine) and did not present a framework for decision-making (i.e., a hypothetical study would only be included if it presented a decision-making framework); 4) Did not consider factors that directly affected decisions (e.g. assessed which macro-level factors, such as the Gross Domestic Product (GDP) were associated with decisions to adopt, rather than focusing on the actual criteria considered within decision-making processes); 5) Focused on the economic evaluation of introducing a new vaccine and did not present any other factors that affected decisions; 6) Was published in a language other than English, Spanish, or French.

We classified the articles identified into the following groups based on the type of article: 1) Articles that present a framework of decision-making for vaccine adoption (article type 1); 2) Studies that collect or analyze empirical data on decision-making for vaccine adoption and relevant examples of such decision-making (article type 2); and 3) Theoretical and empirical articles that provide insights into the process of vaccine policymaking (article type 3) (Appendix 2). Burchett et al (2012) found the published literature describing the process of decision-making for vaccine adoption to be limited. Thus, we specifically added article type 3 to our systematic review. Any disagreement between reviewers in the processes of screening and classification was discussed and resolved through consensus.

We abstracted data based on the type of article, the country income level and Gavi, the Vaccine Alliance (Gavi) eligibility status, the type of vaccine, and the categories and criteria included in the decision-making frameworks or mentioned in the studies. Country income levels were defined, using the 2018 World Bank definitions, based on country Gross National Income (GNI) per capita and the World Bank Atlas method³ [6]. Gavi-eligible countries were defined as those that were eligible for Gavi funding in Phase 3, commencing in 2010, which includes 74 countries globally [7]. The eligibility threshold was defined as country GNI per capita ≤ US$ 1000, based on 1998 World Bank data. Data abstraction for article type 1 and 2 was conducted by the same reviewer. Another reviewer abstracted article type 3.

We performed a quality appraisal of the studies that collected or analyzed empirical data on decision-making for vaccine adoption (Article type 2), using the Mixed Methods Appraisal Tool (MMAT) (Appendix 3). This tool was chosen because many different study designs were used across the studies [8]. Answers to the two initial screening questions determined whether the article could be appraised using the MMAT. The MMAT score ranges from 0 to 100% and is based on meeting certain methodological quality criteria, depending on the type of study (i.e., quantitative, qualitative, or mixed methods). Articles that passed the two screening questions with a quality score of ≥ 25% were included in the analysis.

We used the nine broad categories of criteria that Burchett et al. 2012 identified in their analysis [4]. The categories were: the importance of the health problem, vaccine characteristics, programmatic considerations, acceptability, accessibility, equity and ethics, financial/economic issues, the impact of vaccination, consideration of alternative interventions, and the decision-making process. When analyzing the studies conducted on decision-making and examples of vaccine adoption decision-making (Article type 2), we focused on obtaining a more robust database and compared the criteria identified in these articles to those described in the frameworks (Article type 1). To analyze Type 3 articles, one reviewer abstracted the main themes or factors that the authors mentioned as being important in facilitating successful vaccine policymaking in a specific setting (i.e., country).

Of the 27 articles that were categorized as frameworks of decision-making for vaccine introduction, 20 provided a country perspective, two provided a regional perspective, and five gave a global perspective [9–35]. Most articles focused on a HIC (n = 9) or a UMIC (n = 8); many (n = 18) focused exclusively on countries that are not eligible for Gavi support (Table 2). Eleven articles did not consider a specific vaccine. Of those that did, most focused on one vaccine, and others focused on multiple vaccines. The categories of criteria for decision-making within the frameworks are described below.

The vaccine characteristics were reported in all 27 frameworks and were most frequently captured as vaccine efficacy or effectiveness (26/27; 96%) followed by vaccine safety (Table 3). Delivery issues and other vaccine characteristics, including vaccine wastage, vaccine presentation, and the infrastructure required to implement the new intervention were inconsistently considered throughout the frameworks (Table 3).

The importance of the health problem was reported in 26 of the 27 frameworks. The criteria described under the importance of the health problem in this type of article varied: the burden of the disease was most commonly reported, followed by the cost or economic burden of the disease and the public perception of the disease (Table 3). Political priority was considered to a lesser extent (8/26; 31%).

Financial and economic issues were reported in 26 of the 27 frameworks (Table 2), mostly via the inclusion of an economic evaluation (25/26; 96%) or consideration of the vaccine price (16/26; 62%). The incremental costs induced by the introduction of the new strategy, its financial sustainability, its affordability, and the funding sources were considered to a lesser extent (Table 3). The program implementation costs (i.e., the cost of vaccination, the demand forecast) were the additional economic issues considered.

Alternative interventions were reported in 25 of the 27 frameworks. In all but one of those, the influence of alternative interventions was captured via cost-effectiveness analysis, rather than via the effectiveness of the interventions alone (Table 3).

The decision-making process was reported in 24 of the 27 frameworks. The procedures and enabling factors for decision-making were not considered often.

The impact of vaccination was reported in 21 of the 27 frameworks. In most articles, the impact of vaccination was captured by the impact of the intervention on health outcomes, followed by the impact on non-health outcomes (14/21; 67%). The risks of serotype replacement and the effects of co-administration with other vaccines were reported to a lesser extent (Table 3).

Immunization program considerations and more specifically, the programmatic aspects of introducing a new vaccine were reported to a lesser extent (15/27 frameworks) than the previous categories.

Finally, the acceptability of the vaccine was considered in 11 frameworks while the accessibility, equity, and ethics related to vaccination were considered in 13 of the 27 frameworks.

In summary, the criteria most commonly reported in the frameworks were the burden of disease, vaccine efficacy/ effectiveness, vaccine safety, economic evaluation, impact on health and non-health outcomes, cost-effectiveness of alternatives, and the evidence sources. The public perception of the disease, acceptability of the vaccine, its accessibility, as well as equity and ethical issues, delivery issues, feasibility aspects, and the procedures involved were considered to a lesser extent.

Of the 45 studies categorized as Article type 2, 14 were examples of decision-making for vaccine introduction [36–49], and 31 collected empirical data on decision-making [50–80]. Of the 31 empirical studies, 21 used qualitative methods, three used mixed methods (quantitative and qualitative), three were quantitative, and four were literature reviews.

When we used the MMAT tool, the quality of the 27 empirical studies (excluding the literature reviews) varied, with scores ranging between 25 and 100% and a median of 75% (Appendix 5). Of the 27 empirical studies, four studies scored 100%, 13 studies scored 75%, six studies scored 50%, and four studies scored 25 percent. Considering that all studies passed the two screening questions and had a quality rating of ≥ 25%, we decided that none of the studies would be excluded. Therefore, the analysis is based on all of the empirical studies and examples of decision-making. We did not find any difference in the categories of criteria reported after stratification of empirical studies by MMAT score.

Almost a third of these studies focused on HICs (14/45; 31%), four focused on UMICs, nine focused on LMICs, and only two focused on a LIC; sixteen studies reported on multiple countries’ income-levels. Eleven studies focused on a Gavi-eligible country, while 13 compiled data from a mix of countries (i.e., Gavi-eligible and not eligible). Twenty-one studies focused exclusively on countries not eligible for Gavi support (Appendix 4). Of the 32 studies that focused on a specific vaccines(s), most focused on one vaccine, and five focused on multiple vaccines.

The topics of the studies varied. Some used qualitative or quantitative data collection methods to assess the decision-making process; others presented the criteria valued by the national immunization stakeholders for introducing new vaccines or reviewing immunization policy.

The vaccine characteristics were reported in 34 of the 45 studies and were most frequently stated via the inclusion of vaccine efficacy and/or effectiveness (41/44; 93%), followed by vaccine safety (35/44; 80%) (Table 3). We observed the same pattern in the frameworks. However, the delivery issues were more frequently explored and reported in the studies (29/44; 66%) than in the frameworks (11/27; 41%). Other vaccine characteristics, such as the vaccine wastage; the vaccine presentation and formulation; the vaccine viability under adverse conditions, and the cold chain capacity were considered inconsistently as was the case with the frameworks (Table 3).

The importance of the health problem was reported in 42 of the 45 studies included (Table 3). Similar to the frameworks, the burden of the disease was most commonly mentioned (40/42; 95%). However, the political priority for implementing such vaccine intervention was more frequently explored in the studies (26/42; 62%) than in the frameworks (8/26; 31%) (Table 3).

Financial and economic issues were reported in 41 of the 45 studies (Table 3), which were mostly stated via the inclusion of an economic evaluation (34/41; 83%), and this was similar to what we found in the framework articles. However, the vaccine price (21/41; 51%) and incremental costs (8/41; 20%) were reported to a lesser extent in the studies than in the framework articles: (16/26; 62%) and (7/26; 27%), respectively. However, the funding source was more likely to be considered in the studies (19/41; 46%) compared to the framework articles (3/26; 12%) (Table 3). Other economic issues considered were program implementation costs, programmatic strength, and budget impact analysis.

Alternative interventions were reported in 35 of the 45 studies (Table 3). As found in the framework articles, those that provided information on alternative interventions, the focus was mostly on the cost-effectiveness of the alternatives and less frequently on the effectiveness (Table 3).

The decision-making process was reported in 38 of the 45 studies (Table 3). The sources of evidence and/or the quality of the evidence were reflected similarly in the studies (20/27; 74%) and the frameworks (13/17; 76%). However, the stakeholders involved (34/38; 89%) and the procedures required for decision-making (25/38; 66%) were more likely to be considered in the studies than in the frameworks: (16/24; 67%) and (10/24; 42%) respectively.

The impact of vaccination was reported in 30 of the 45 studies (Table 3). As for the frameworks, it was most frequently captured through the impact of the intervention on health outcomes (28/30; 93%), followed by the impact on non-health outcomes (20/30; 67%). However, the effects of co-administration with other vaccines were questioned to a larger extent in the studies (7/30; 23%) than in the frameworks (1/21; 5%) (Table 3).

Immunization program considerations, and more specifically, the programmatic aspects of introducing a new vaccine were represented to a lesser extent than the previous categories in 26 of the 45 studies (Table 3). The programmatic aspects were captured to a larger extent by vaccine supply in the studies (18/26; 69%) than in the frameworks (7/15; 47%), whereas it was the other way around with the feasibility issue: (13/26; 50%) in the studies and (11/15; 73%) in the frameworks (Table 3).

The acceptability of the vaccine was considered in 20 studies while the accessibility, equity, and ethics related to vaccination were considered in 15 of the 45 studies (Table 3).

In summary, the most common criteria of interest in the studies were the burden of disease, vaccine efficacy/effectiveness, vaccine safety, economic evaluation, the cost-effectiveness of alternatives, and the stakeholders involved. Public perception of the disease, accessibility, equity, and ethical issues regarding the new vaccine, the effects of vaccine co-administration, and risks of serotype replacement and feasibility aspects were considered to a lesser extent.

A total of 61 references were identified as relevant to document decision-making processes for vaccine introduction: seventeen articles already included in either type 1 or type 2 articles [9,11,25,34,35,45–49,59,70,75–78,80] and 44 additional articles [81–124]. Findings are summarized in Table 4. Enabling factors for vaccine introduction include recommendations made from global committees such as the Strategic Advisory Group of Experts on Immunization (SAGE)/ WHO or other guidance developed by UNICEF, Gavi, the World Bank, or the Bill and Melinda Gates Foundation (BMGF). Recommendations made by the Regional Technical Advisory Group on Immunization and Vaccine-Preventable Diseases (TAG) and national bodies, such as the NITAG and the National Regulatory Agency (NRA) have facilitated national policymaking as reported in most references. In many LMICs, Gavi, has partnered with governments to strengthen decision-making and has played a leading role in vaccine introductions. Most references (70%) mentioned country ownership of the immunization programs as a facilitator to vaccine introduction policymaking. It was described as a strong level of governance, political will, and allocation of financial resources. Other key features of decision-making for vaccine adoption include robust policy dialogue among the different national stakeholders and institutions (57%), strong partnerships between the public and the private sectors (57%), primarily to address issues related to purchasing and reimbursement mechanisms for the vaccine. Additionally, most references (52%) mentioned strong health systems as facilitators to vaccine adoption decision-making, especially in the presence of adequate operational capacity to add another vaccine into the national schedule. Finally, lessons learned and experiences from other countries, especially neighboring countries benefited vaccine policy decisions (43%). Having a robust institutionalized process for vaccine introduction was cited as an enabling factor in most references (56%). Relying on the enactment of immunization laws for vaccine introduction was mentioned in less than a quarter of the references. Several references (18%) indicated that the collaboration between the NITAG and the NRA helped with fast-tracking the vaccine introduction and address issues, such as the off-label use of vaccines in a timely manner.

There were several limitations to this review. Our search strategy did not include grey literature as a source of information, which limits the comprehensiveness of our findings. The incomplete retrieval of identified research and publication bias may have affected the robustness of our findings. The screening of the references that came from the June 2017 to August 2020 literature search, the data abstraction, and the analysis of the references were done by one researcher. Although extra checks were performed to improve the analysis and all results were discussed by the research team, some bias may have resulted. Twenty-three percent of the articles included in our review reported on a HIC, while only 3% reported on a LIC; therefore, our findings are biased towards the frameworks used and studies performed in these settings. Additionally, we structured our review on the nine broad categories of criteria that Burchett et al. 2012 developed; we used those for evaluating the frameworks and the studies on decision-making [4]. This approach enabled the comparison of results and assessment of trends in this area of research although it may have driven our analysis away from a different taxonomy of frameworks.