<!DOCTYPE article
PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD with MathML3 v1.3 20210610//EN" "JATS-archivearticle1-3-mathml3.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" dtd-version="1.3" xml:lang="en" article-type="research-article"><?properties manuscript?><processing-meta base-tagset="archiving" mathml-version="3.0" table-model="xhtml" tagset-family="jats"><restricted-by>pmc</restricted-by></processing-meta><front><journal-meta><journal-id journal-id-type="nlm-journal-id">8406899</journal-id><journal-id journal-id-type="pubmed-jr-id">7945</journal-id><journal-id journal-id-type="nlm-ta">Vaccine</journal-id><journal-id journal-id-type="iso-abbrev">Vaccine</journal-id><journal-title-group><journal-title>Vaccine</journal-title></journal-title-group><issn pub-type="ppub">0264-410X</issn><issn pub-type="epub">1873-2518</issn></journal-meta><article-meta><article-id pub-id-type="pmid">32563604</article-id><article-id pub-id-type="pmc">10205045</article-id><article-id pub-id-type="doi">10.1016/j.vaccine.2020.05.077</article-id><article-id pub-id-type="manuscript">HHSPA1898298</article-id><article-categories><subj-group subj-group-type="heading"><subject>Article</subject></subj-group></article-categories><title-group><article-title>Building immunization decision-making capacity within the World Health Organization European Region</article-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Mosina</surname><given-names>Liudmila</given-names></name><xref rid="A1" ref-type="aff">a</xref><xref rid="CR1" ref-type="corresp">*</xref></contrib><contrib contrib-type="author"><name><surname>Datta</surname><given-names>Siddhartha Sankar</given-names></name><xref rid="A1" ref-type="aff">a</xref></contrib><contrib contrib-type="author"><name><surname>Shefer</surname><given-names>Abigail</given-names></name><xref rid="A2" ref-type="aff">b</xref></contrib><contrib contrib-type="author"><name><surname>Cavallaro</surname><given-names>Kathleen F.</given-names></name><xref rid="A1" ref-type="aff">a</xref></contrib><contrib contrib-type="author"><name><surname>Henaff</surname><given-names>Louise</given-names></name><xref rid="A3" ref-type="aff">c</xref></contrib><contrib contrib-type="author"><name><surname>Steffen</surname><given-names>Christoph A.</given-names></name><xref rid="A3" ref-type="aff">c</xref></contrib><contrib contrib-type="author"><name><surname>Jacques-Carroll</surname><given-names>Lisa</given-names></name><xref rid="A1" ref-type="aff">a</xref></contrib></contrib-group><aff id="A1"><label>a</label>World Health Organization Regional Office for Europe, UN City, Marmorvej 51, 2100 Copenhagen, Denmark</aff><aff id="A2"><label>b</label>USA Centers for Disease Control and Prevention, 1600 Clifton Rd, MS H24-2, Atlanta, GA 30329, USA</aff><aff id="A3"><label>c</label>World Health Organization Headquarters, Avenue Appia 20, 1211 Geneva, Switzerland</aff><author-notes><corresp id="CR1"><label>*</label>Corresponding author. <email>mosinal@who.int</email>, (L. Mosina).</corresp></author-notes><pub-date pub-type="nihms-submitted"><day>12</day><month>5</month><year>2023</year></pub-date><pub-date pub-type="ppub"><day>14</day><month>7</month><year>2020</year></pub-date><pub-date pub-type="epub"><day>17</day><month>6</month><year>2020</year></pub-date><pub-date pub-type="pmc-release"><day>23</day><month>5</month><year>2023</year></pub-date><volume>38</volume><issue>33</issue><fpage>5109</fpage><lpage>5113</lpage><abstract id="ABS1"><p id="P1">A National Immunization Technical Advisory Group (NITAG) is a multi-disciplinary body of national experts that provides evidence-based recommendations to policy-makers, assisting them in making sound immunization policy and programme decisions. The World Health Organization (WHO) Regional Office for Europe is working to strengthen the capacity of newly-established NITAGs and has targeted efforts on low- and middle-income countries. The Regional Office, in collaboration with WHO Headquarters and USA Centers for Disease Control and Prevention (CDC), developed a new training strategy and held training workshops to improve NITAGs&#x02019; functioning and ability to make evidence-based recommendations. Feedback from countries that participated in trainings indicated that the updated training materials and interactive approach with follow-up technical support enabled them to align their NITAG charters and processes with WHO recommendations. To ensure continued progress, global and regional partners such as WHO and CDC should continue providing technical support to recently established NITAGs.</p></abstract><kwd-group><kwd>NITAG</kwd><kwd>Capacity building</kwd><kwd>Evidence-based decision</kwd><kwd>Immunization</kwd></kwd-group></article-meta></front><body><sec id="S1"><label>1.</label><title>Introduction</title><p id="P2">Strengthening in-country evidence-based decision-making capacity for immunization has been a priority of the World Health Organization (WHO) Regional Office for Europe (Regional Office) and was adopted as a strategic goal in the 2015&#x02013;2020 European Vaccine Action Plan (EVAP) [<xref rid="R1" ref-type="bibr">1</xref>].</p><p id="P3">A National Immunization Technical Advisory Group (NITAG) is a multi-disciplinary body of national experts that provides evidence-based recommendations to policy-makers and immunization programme managers, assisting them in making sound immunization policy and programme decisions [<xref rid="R2" ref-type="bibr">2</xref>]. NITAGs are valued for their roles in strengthening country ownership and public confidence [<xref rid="R3" ref-type="bibr">3</xref>] in the national immunization programme, and ensuring a nationally owned, independent evidence-based decision-making process [<xref rid="R4" ref-type="bibr">4</xref>]. The indicators WHO uses to assess NITAG functioning are: (1) the provision of a legislative basis for the NITAG, (2) the availability of written terms of reference, (3) representation of at least five disciplines<sup><xref rid="FN2" ref-type="fn">1</xref></sup> within NITAG members, (4) conducting annual NITAG meetings, (5) advance sharing of the meeting agenda and documents, and (6) declarations of interest by NITAG members [<xref rid="R5" ref-type="bibr">5</xref>].</p><p id="P4">The Regional Office focuses on extending the benefits of vaccination to all communities through a process of evidence-based decision-making in the countries, and advocates for all countries to establish and strengthen NITAGs. The advocacy from the Regional Office has resulted in raising awareness of a NITAG&#x02019;s value and has generated political will to establish these committees. The Regional Office has targeted its efforts to strengthen the evidence-based decision-making capacity of newly-established NITAGs in low- and middle-income countries (LMICs) &#x02014; countries with the most limited human and financial resources. From 2011 to 2014, the Regional Office held five training workshops for NITAG chairs, members, and secretariats (referred to as representatives hereafter) from 25 countries (approximately four participants per country) to improve NITAGs&#x02019; functioning and ability to make evidence-based recommendations. The Regional Office facilitated networking among long-functioning and recently established NITAGs for sharing best practices. During 2014&#x02013;2018, NITAG representatives from Albania, Armenia, Belarus, and Georgia visited the Joint Committee on Vaccination and Immunisation, United Kingdom, and the Health Council of the Netherlands.</p><sec id="S2"><label>1.1.</label><title>NITAGs in countries of WHO European Region and globally</title><p id="P5">During the implementation period of the 2015&#x02013;2020 EVAP, notable progress has been made in establishing NITAGs. According to the annual WHO/UNICEF Joint Reporting Form on Immunization (JRF), since 2014, eleven additional countries reported the existence of a NITAG. By 2019, 50 out of 53 countries, including 19 out of 20 LMICs in the WHO European Region reported a NITAG present (<xref rid="F1" ref-type="fig">Fig. 1</xref>).</p><p id="P6">Though progress has been made, the performance of NITAGs in the WHO European Region varies considerably. In 2018, only 71% of all NITAGs and 58% of NITAGs from LMICs, reported meeting all six process indicators for NITAG functioning (<xref rid="F2" ref-type="fig">Fig. 2</xref>), mainly because of challenges in introducing declarations of interest by NITAG members.</p><p id="P7">At the global level, in 2018, 114 of 194 countries reported having a NITAG that met process criteria for NITAG functioning, serving 85% of the world&#x02019;s population [<xref rid="R6" ref-type="bibr">6</xref>].</p><p id="P8">The surveys and evaluations of NITAGs in LMICs conducted by the Regional Office and by other partners from 2016 to 2018 revealed that despite participation in trainings, NITAGs continued facing challenges in formulating recommendations for the Ministries of Health (MoH) [<xref rid="R7" ref-type="bibr">7</xref>,<xref rid="R8" ref-type="bibr">8</xref>]. Many NITAGs based their recommendations on their own expert opinion rather than on systematic collection and evaluation of available evidence. In addition, NITAGs faced challenges in establishing effective coordination with decision-makers, diminishing the impact of NITAG recommendations on national immunization policies.</p><p id="P9">To address these challenges, starting in 2017, the Regional Office, in collaboration with WHO Headquarters (WHO HQ) and the USA Centers for Disease Control and Prevention (CDC), developed a new training strategy to align NITAG composition and functioning with WHO recommendations [<xref rid="R2" ref-type="bibr">2</xref>] and to introduce a systematic approach for making evidence-based recommendations. This report describes the NITAG training strategy, highlights the components, and discusses future plans to provide support to NITAGs in the region.</p></sec></sec><sec id="S3"><label>2.</label><title>Description of NITAG training strategy</title><p id="P10">The Regional Office, WHO HQ, and CDC used the training materials available through the NITAG Resource Center [<xref rid="R9" ref-type="bibr">9</xref>] as well as best practices from well-functioning NITAGs and experience of partners to develop the updated set of training materials tailored to the specific needs based on the level of maturity of NITAGs in LMICs. In addition, whenever possible, methods and processes from WHO&#x02019;s Strategic Advisory Group of Experts (SAGE) on Immunization were used and adapted to the national setting. It was piloted with NITAG representatives from countries eligible for the Vaccine Alliance (Gavi) support<sup><xref rid="FN3" ref-type="fn">2</xref></sup> (approximately four participants from each country) at a workshop in May 2018 in Denmark. The materials were revised and finalized based on the feedback received from the participants and facilitators.</p><p id="P11">The final set of training materials for the structured four-day training includes presentations, group exercises, hands-on activities, peer-to-peer exchange of information, and relevant vaccine-specific examples.</p><p id="P12">During session one of the workshop, NITAG representatives present their work and share their achievements and challenges. This helps to integrate the existing experience of the participants, increase understanding of the training&#x02019;s relevance to their needs, and encourages active involvement during the training. Furthermore, NITAG representative presentations help facilitators understand the differences in various NITAGs&#x02019; functioning, which allows better facilitation of group exercises and discussions, keeping in mind each country context.</p><p id="P13">The second session of the training covers NITAG composition and helps participants understand the procedural requirements for ensuring proper NITAG functioning. The third and fourth sessions address the methods and tools to develop an evidence-based recommendation from a policy question. The final session of the training outlines effective strategies for NITAGs to communicate evidence-based recommendations to policy- and decision-makers.</p><p id="P14">An important training element is participation of representatives of well-functioning NITAGs who share best practices for each training topic, facilitate working groups and serve as a resource. The updated materials were used to conduct a four-day training in Montenegro in April 2019. The topics covered during the 2019 workshop in Montenegro are listed in <xref rid="T1" ref-type="table">Table 1</xref>.</p><sec id="S4"><label>2.1.</label><title>Training materials highlights</title><p id="P15">The training was adapted from existing materials in several ways. In particular, the workshop includes interactive group exercises for each topic that prompt participants to question existing practices of their NITAG, and share and discuss among various participants, with the goal of gaining insight into the changes needed for their NITAG. For example, the first group exercise directs participants to critically review the composition and functioning of their NITAG and propose amendments to Charters and Terms of Reference. Exercises three through seven guide participants in practicing step-by-step the process of developing evidence-based recommendations. Participants develop skills to formulate PICO questions from broad policy questions using real-life examples. They adopt and rank generic criteria for a PICO question from a simulation scenario. NITAG representatives access the SAGE website to find available evidence, including SAGE tables on GRADE quality assessment, and review and interpret the SAGE Evidence to Recommendation Framework. Group exercise eight teaches participants to establish a work group to assist in developing NITAG recommendations.</p><p id="P16">Session five of the workshop focuses on improving skills in communicating recommendations to decision-makers, and participants are provided with background documents and meeting minutes of well-functioning NITAGs, as examples to develop their materials.</p><p id="P17">The training does not provide detailed instruction on how to conduct a systematic literature review because experience has shown that it is challenging within a four&#x02013;day workshop to train participants who may lack background knowledge on this. Instead, the training describes the methodology of conducting a systematic review during the decision-making process and instructs participants how to collect available evidence from existing systematic literature reviews conducted by SAGE, other NITAGs, Cochrane reviews, and other trusted sources. To better use systematic reviews conducted by other groups, participants are taught how to read and critically interpret evidence tables. Additionally, NITAG representatives are encouraged to define the criteria of decision-making that their country wants to use and focus on collecting and evaluating local or country-specific information<sup><xref rid="FN4" ref-type="fn">3</xref></sup> for decision making.</p><p id="P18">To standardize the process of summarizing evidence and translating it into recommendations, the participants are taught to use the SAGE Evidence to Recommendation tables [<xref rid="R10" ref-type="bibr">10</xref>] based on the DECIDE Evidence to Decision framework [<xref rid="R11" ref-type="bibr">11</xref>], adapted to their country context.</p></sec><sec id="S5"><label>2.2.</label><title>Impact of training and follow-up technical support</title><p id="P19">During the four&#x02013;day training conducted with updated training materials in Montenegro in April 2019, a total of 28 NITAG representatives (5 chairs, 13 members and 10 secretariats) from MICs that do not receive Gavi support: Albania, Belarus, Bosnia and Herzegovina, Georgia, Montenegro, North Macedonia, Romania, Serbia, and Ukraine, attended. In the training evaluation, nearly all participants felt that the training improved their skills related to NITAG work [<xref rid="R12" ref-type="bibr">12</xref>] (<xref rid="F3" ref-type="fig">Fig. 3</xref>). Participants reported significant increases in their level of understanding and confidence in performing essential NITAG functions, such as the role of workgroups, translating a policy question into a PICO question, selecting criteria and key sources of evidence for developing recommendations, understanding the WHO approach to grading of scientific evidence, and understanding best practices for communicating NITAG recommendations to the MoH. Additionally, the participants provided suggestions to improve the training materials for future workshops.</p><p id="P20">To ensure that the knowledge and skills gained during training are transferred in routine NITAGs practice, the Regional Office provided follow-up technical assistance to the countries that participated in the 2018 training. In 2019, The Regional Office facilitated the revision of Kyrgyzstan&#x02019;s NITAG Charter to reach WHO indicators of NITAG functioning and improve the process for communicating NITAG recommendations to the MoH. The Regional Office advocated for establishing NITAG working groups in Kyrgyzstan and Tajikistan and provided on-the-job training on collecting and summarizing evidence, and translating evidence into recommendations. Working groups prepared background documents on pneumococcal and HPV vaccines and, for the first time, these NITAGs had the opportunity to base their recommendations on thorough deliberation of evidence. On-the-job training also helped disseminate knowledge and skills in developing evidence-based recommendations to members of NITAGs and Secretariats who did not participate in the regional training.</p></sec></sec><sec id="S6"><label>3.</label><title>Discussion and future plans</title><p id="P21">At the global level, WHO has conducted a systematic assessment of strengths and shortcomings of existing NITAG training materials and methods, including the materials developed in the WHO European Region. The aim is to provide a new standard set of training materials to all WHO regions, addressing the training needs in most country situations. This work includes identification of key additional materials to be developed with particular attention to instructional design and complementary teaching methods, such as videos, role playing, and a broad array of case studies. Key materials will be translated into the main WHO languages. It is anticipated that the training material package will be finalized with the support of an ad hoc Global NITAG Network (GNN) [<xref rid="R13" ref-type="bibr">13</xref>] working group and published on the NITAG Resource Center website [<xref rid="R14" ref-type="bibr">14</xref>] for wide use in the NITAG community.</p><p id="P22">However, more needs to be done to ensure training impact. The Regional Office used follow-up technical support and on-the-job-training to encourage NITAGs to use a robust evidence to recommendation framework to make recommendations. Other approaches such as conducting additional in-country trainings, may also be used. The effect of training significantly depends upon the existence of a strong Secretariat capable of providing adequate support to a NITAG in introducing this systematic approach in developing recommendations.</p><p id="P23">To ensure continued progress, global and regional partners such as WHO and CDC should continue providing technical support to recently established NITAGs from LMICs and advocate to MoHs for sustained support to NITAGs and their Secretariats, including provision of necessary human and financial resources. Building sustainable in-country expertise in decision-making on vaccine related matters requires long term commitments from countries and investments in ongoing technical assistance [<xref rid="R4" ref-type="bibr">4</xref>,<xref rid="R15" ref-type="bibr">15</xref>].</p><p id="P24">The portfolio of vaccines and related issues (e.g., product delivery techniques, formulations, schedules) continues to expand, and countries face complex problems, including shifting health priorities, increasing resource constraints and issues with demand and acceptance of vaccines. In this challenging context, as we move into the next decade, the development of functional and trusted NITAGs will become even more important to add credibility to the policy-making process and for successful implementation of national immunization programmes.</p></sec><sec id="S7"><label>4.</label><title>Disclaimer</title><p id="P25">The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.</p></sec></body><back><ack id="S8"><title>Acknowledgments</title><p id="P26">The authors are grateful to Erin D Kennedy from USA Centers for Disease Control and Prevention for the support in revising training materials and delivering 2019 NITAG training in Montenegro.</p></ack><fn-group><fn id="FN1"><p id="P27">Declaration of Competing Interest</p><p id="P28">The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.</p></fn><fn id="FN2"><label>1</label><p id="P29">Paediatrics, public health, infectious diseases, epidemiology, immunology or other health-care professionals.</p></fn><fn id="FN3"><label>2</label><p id="P30">Azerbaijan, Kazakhstan, Kyrgyzstan, Tajikistan, and Uzbekistan.</p></fn><fn id="FN4"><label>3</label><p id="P31">The burden and epidemiology of the targeted disease, programmatic aspects of the introduction of a new vaccine, factors determining the acceptance of a new vaccine by the public and medical community, and economic considerations for introducing new vaccines.</p></fn></fn-group><glossary><title>Abbreviations:</title><def-list><def-item><term>NITAG</term><def><p id="P32">National Immunization Technical Advisory Group</p></def></def-item><def-item><term>WHO</term><def><p id="P33">World Health Organization</p></def></def-item><def-item><term>CDC</term><def><p id="P34">Centers for Disease Control and Prevention</p></def></def-item><def-item><term>Regional Office</term><def><p id="P35">WHO Regional Office for Europe</p></def></def-item><def-item><term>EVAP</term><def><p id="P36">2015&#x02013;2020 European Vaccine Action Plan</p></def></def-item><def-item><term>LMICs</term><def><p id="P37">low- and middle-income countries</p></def></def-item><def-item><term>JRF</term><def><p id="P38">WHO/UNICEF Joint Reporting Form on Immunization</p></def></def-item><def-item><term>WHO HQ</term><def><p id="P39">WHO Headquarters</p></def></def-item><def-item><term>SAGE</term><def><p id="P40">Strategic Advisory Group of Experts on Immunization</p></def></def-item><def-item><term>MOH</term><def><p id="P41">Ministry of Health</p></def></def-item><def-item><term>PICO</term><def><p id="P42">Patient/Problem, Intervention, Comparison, Outcome</p></def></def-item><def-item><term>HPV</term><def><p id="P43">human papillomavirus</p></def></def-item><def-item><term>GNN</term><def><p id="P44">Global NITAG Network</p></def></def-item></def-list></glossary><ref-list><title>References</title><p id="P45">All URLs accessed 25 March 2020</p><ref id="R1"><label>[1]</label><mixed-citation publication-type="book"><source>European Vaccine Action Plan 2015&#x02013;2020</source>. <publisher-name>World Health Organization Regional Office for Europe</publisher-name>; <year>2014</year>. <comment><ext-link xlink:href="http://www.euro.who.int/_data/assets/pdf_file/0004/257575/64wd15e_EVAP_Rev1_140459.pdf" ext-link-type="uri">http://www.euro.who.int/_data/assets/pdf_file/0004/257575/64wd15e_EVAP_Rev1_140459.pdf</ext-link></comment>].</mixed-citation></ref><ref id="R2"><label>[2]</label><mixed-citation publication-type="journal"><name><surname>Duclos</surname><given-names>P</given-names></name>. <article-title>National Immunization Technical Advisory Groups (NITAGs): Guidance for their establishment and strengthening</article-title>. <source>Vaccine</source>
<year>2010</year>;<volume>28</volume>(<issue>Suppl. 1</issue>):<fpage>A18</fpage>&#x02013;<lpage>25</lpage>.<pub-id pub-id-type="pmid">20412991</pub-id></mixed-citation></ref><ref id="R3"><label>[3]</label><mixed-citation publication-type="journal"><name><surname>Bell</surname><given-names>S</given-names></name>, <name><surname>Blanchard</surname><given-names>L</given-names></name>, <name><surname>Walls</surname><given-names>H</given-names></name>, <name><surname>Mournier-Jack</surname><given-names>S</given-names></name>, <name><surname>Howard</surname><given-names>N</given-names></name>. <article-title>Value and Effectiveness of National Immunization Technical Advisor Groups in low- and middle-income countries: a qualitative study of global and national perspectives</article-title>. <source>Health Policy Plan</source>
<year>2019</year>;<volume>34</volume>:<fpage>271</fpage>&#x02013;<lpage>81</lpage>.<pub-id pub-id-type="pmid">31074778</pub-id></mixed-citation></ref><ref id="R4"><label>[4]</label><mixed-citation publication-type="journal"><name><surname>Howard</surname><given-names>N</given-names></name>, <name><surname>Bell</surname><given-names>S</given-names></name>, <name><surname>Walls</surname><given-names>H</given-names></name>, <name><surname>Blanchard</surname><given-names>L</given-names></name>, <name><surname>Brenzel</surname><given-names>L</given-names></name>, <name><surname>Jit</surname><given-names>M</given-names></name>, <etal/>
<article-title>The need for sustainability and alignment of future support for National Immunization Technical Advisory Groups (NITAGs) in low and middle-income countries</article-title>. <source>Human Vacc Immunother</source>
<year>2018</year>;<volume>14</volume>(<issue>6</issue>):<fpage>1539</fpage>&#x02013;<lpage>41</lpage>.</mixed-citation></ref><ref id="R5"><label>[5]</label><mixed-citation publication-type="journal"><name><surname>Duclos</surname><given-names>P</given-names></name>. <article-title>Progress in the establishment and strengthening of national immunization technical advisory groups: Analysis from the 2013 WHO/UNICEF joint reporting form, data for 2012</article-title>. <source>Vaccine</source>
<year>2013</year>;<volume>31</volume>:<fpage>5314</fpage>&#x02013;<lpage>20</lpage>.<pub-id pub-id-type="pmid">24055304</pub-id></mixed-citation></ref><ref id="R6"><label>[6]</label><mixed-citation publication-type="book"><part-title>The Global Vaccine Action Plan 2011&#x02013;2020</part-title>. <source>Strategic Advisory Group of Experts on Immunization: Review and Lessons Learned</source>. <publisher-name>WHO</publisher-name>; <year>2019</year> [<comment><ext-link xlink:href="http://apps.who.int/iris/bitstream/handle/10665/329097/WHO-IVB-19.07-eng.pdf" ext-link-type="uri">http://apps.who.int/iris/bitstream/handle/10665/329097/WHO-IVB-19.07-eng.pdf</ext-link></comment>].</mixed-citation></ref><ref id="R7"><label>[7]</label><mixed-citation publication-type="confproc"><name><surname>Durupt</surname><given-names>A</given-names></name>. <source>NITAG Evaluation - Experience in Armenia [PowerPoint presentation]</source>. <conf-name>First meeting of the international network of NITAGs</conf-name>. <conf-loc>Veyrier du Lac</conf-loc>; <conf-date>12 May 2016</conf-date> [<comment><ext-link xlink:href="http://www.nitag-resource.org/media-center/document/3424-armenian-nitag-evaluation" ext-link-type="uri">http://www.nitag-resource.org/media-center/document/3424-armenian-nitag-evaluation</ext-link></comment>].</mixed-citation></ref><ref id="R8"><label>[8]</label><mixed-citation publication-type="book"><collab>Third Global NITAG Network meeting</collab>. <source>Record of discussion</source>. <publisher-loc>Ottawa, Canada</publisher-loc>, <day>6&#x02013;7</day>
<month>December</month>
<year>2018</year>. [<comment><ext-link xlink:href="http://www.nitag-resource.org/sites/default/files/8790c9cc0d447705ac4df723a4bb9db418b95d48_1.pdf" ext-link-type="uri">http://www.nitag-resource.org/sites/default/files/8790c9cc0d447705ac4df723a4bb9db418b95d48_1.pdf</ext-link></comment>].</mixed-citation></ref><ref id="R9"><label>[9]</label><mixed-citation publication-type="journal"><name><surname>Adjabga</surname><given-names>A</given-names></name>, <name><surname>Henaff</surname><given-names>L</given-names></name>, <name><surname>Duclos</surname><given-names>P</given-names></name>. <article-title>The NITAG Resource Centre (NRC): One-stop shop towards a collaborative platform</article-title>. <source>Vaccine</source>
<year>2015</year>;<volume>33</volume>:<fpage>4365</fpage>&#x02013;<lpage>7</lpage>.<pub-id pub-id-type="pmid">26165917</pub-id></mixed-citation></ref><ref id="R10"><label>[10]</label><mixed-citation publication-type="book"><source>Guidance for the development of evidence-based vaccination related recommendations</source>. <publisher-name>WHO</publisher-name>; <year>2019</year> [<comment><ext-link xlink:href="https://www.who.int/immunization/sage/Guidelines_development_recommendations.pdf" ext-link-type="uri">https://www.who.int/immunization/sage/Guidelines_development_recommendations.pdf</ext-link></comment>].</mixed-citation></ref><ref id="R11"><label>[11]</label><mixed-citation publication-type="webpage"><source>DECIDE Evidence to Decision (EtD) framework</source>. [<comment><ext-link xlink:href="https://www.decide-collaboration.eu/evidence-decision-etd-framework" ext-link-type="uri">https://www.decide-collaboration.eu/evidence-decision-etd-framework</ext-link></comment>].</mixed-citation></ref><ref id="R12"><label>[12]</label><mixed-citation publication-type="confproc"><conf-name>WHO Regional Training Workshop on Evidence-Based Decision Making in Immunization, Budva, Montenegro, 8&#x02013;11 April, 2019</conf-name>. <source>World Health Organization Regional Office for Europe</source>; <year>2019</year>. [<comment><ext-link xlink:href="http://www.euro.who.int/__data/assets/pdf_file/0020/406118/final_WHO-Regional-Training-Workshop-on-Strengthening-Evidence-Report-mol.pdf" ext-link-type="uri">http://www.euro.who.int/__data/assets/pdf_file/0020/406118/final_WHO-Regional-Training-Workshop-on-Strengthening-Evidence-Report-mol.pdf</ext-link></comment>].</mixed-citation></ref><ref id="R13"><label>[13]</label><mixed-citation publication-type="webpage"><source>Strategic document of the Global NITAG Network 2017</source>
<comment><ext-link xlink:href="http://www.nitagresource.org/media-center/document/3864" ext-link-type="uri">http://www.nitagresource.org/media-center/document/3864</ext-link></comment>
<year>2017</year>.</mixed-citation></ref><ref id="R14"><label>[14]</label><mixed-citation publication-type="webpage"><source>NITAG Resource Center</source> [<comment><ext-link xlink:href="http://www.nitag-resource.org" ext-link-type="uri">http://www.nitag-resource.org</ext-link></comment>].</mixed-citation></ref><ref id="R15"><label>[15]</label><mixed-citation publication-type="journal"><name><surname>Ba-Nguz</surname><given-names>Antoinette</given-names></name>, <name><surname>Shah</surname><given-names>Adeel</given-names></name>, <name><surname>Bresee</surname><given-names>Joseph S</given-names></name>, <name><surname>Lafond</surname><given-names>Kathryn E</given-names></name>, <name><surname>Cavallaro</surname><given-names>Kathy</given-names></name>, <name><surname>Shefer</surname><given-names>Abigail</given-names></name>, <name><surname>Donadel</surname><given-names>Morgane</given-names></name>, <name><surname>Seward</surname><given-names>Jane F</given-names></name>. <article-title>Supporting national immunization technical advisory groups (NITAGs) in resource-constrained settings. New strategies and lessons learned from the Task Force for Global Health&#x02019;s Partnership for influenza vaccine introduction</article-title>. <source>Vaccine</source>
<year>2019</year>;<volume>37</volume> (<issue>28</issue>):<fpage>3646</fpage>&#x02013;<lpage>53</lpage>. <comment><ext-link xlink:href="https://linkinghub.elsevier.com/retrieve/pii/S0264410X1930670X" ext-link-type="uri">https://linkinghub.elsevier.com/retrieve/pii/S0264410X1930670X</ext-link>.</comment>
<pub-id pub-id-type="doi">10.1016/j.vaccine.2019.05.046</pub-id>.<pub-id pub-id-type="pmid">31130258</pub-id></mixed-citation></ref></ref-list></back><floats-group><fig position="float" id="F1"><label>Fig. 1.</label><caption><p id="P46">Status of NITAG<sup>1</sup> present, 2010&#x02013;2019, WHO European Region. <bold>Source</bold>: WHO/UNICEF Joint Reporting Form, 2010&#x02013;2019 data release. <sup>1</sup>National Immunization Technical Advisory Group.</p></caption><graphic xlink:href="nihms-1898298-f0001" position="float"/></fig><fig position="float" id="F2"><label>Fig. 2.</label><caption><p id="P47">NITAG<sup>1</sup> functionality according to six process indicators<sup>2</sup>, 2018, WHO European Region. <bold>Disclaimer</bold>: The boundaries and names shown and the designations used on this map do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement. &#x000a9; WHO 2020. All rights reserved. <bold>Source</bold>: WHO UNICEF Joint Reporting Form, July 2019 data release. <sup>1</sup>NITAG, National Immunization Technical Advisory Group. <sup>2</sup>The indicators WHO uses to assess NITAG functionality are: (1) the provision of a legislative basis for the NITAG, (2) the availability of written terms of reference, (3) representation of at least five disciplines within NITAG members, (4) conducting annual NITAG meetings, (5) advance sharing of the meeting agenda and documents, and (6) declarations of interest by NITAG members.</p></caption><graphic xlink:href="nihms-1898298-f0002" position="float"/></fig><fig position="float" id="F3"><label>Fig. 3.</label><caption><p id="P48">Self-evaluation of participant skills after WHO Regional Training Workshop on Evidence-Based Decision Making in Immunization, Budva, Montenegro, 8&#x02013;11 April 2019. <bold>Source</bold>: WHO Regional Training Workshop on Evidence-Based Decision Making in Immunization, Budva, Montenegro, 8&#x02013;11 April 2019. WHO Regional Office for Europe, 2019 (<ext-link xlink:href="http://www.euro.who.int/_data/assets/pdf_file/0020/406118/final_WHO-Regional-Training-Workshop-on-Strengthening-Evidence-Report-mol.pdf" ext-link-type="uri">http://www.euro.who.int/_data/assets/pdf_file/0020/406118/final_WHO-Regional-Training-Workshop-on-Strengthening-Evidence-Report-mol.pdf</ext-link>).</p></caption><graphic xlink:href="nihms-1898298-f0003" position="float"/></fig><table-wrap position="float" id="T1" orientation="landscape"><label>Table 1</label><caption><p id="P49">Outline of NITAG training curriculum from WHO Regional Training Workshop on Evidence-Based Decision Making in Immunization, Budva, Montenegro, 8&#x02013;11 April 2019.</p></caption><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" valign="middle" span="1"/></colgroup><thead><tr><th align="left" valign="top" rowspan="1" colspan="1">Workshop Topics and Group Exercises</th></tr></thead><tbody><tr><td align="left" valign="top" rowspan="1" colspan="1">Country presentations on NITAG composition and functioning</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">NITAG composition and modes of functioning</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Declaration and management of conflicts of interests for NITAG members</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Composition and functioning of a well-established NITAG</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">&#x02003;<italic toggle="yes">Group exercise:</italic> Review of the NITAG structure</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">NITAG collaboration with the Ministry of Health (MOH)</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">&#x02003;<italic toggle="yes">Group exercise:</italic> Strengthening collaboration between NITAG and MOH</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Collaboration between global, regional, and national technical advisory groups</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Developing a policy question</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">&#x02003;<italic toggle="yes">Group exercise:</italic> Drafting a PICO<sup><xref rid="TFN1" ref-type="table-fn">a</xref></sup> question</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Criteria for decision making</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">&#x02003;<italic toggle="yes">Group exercise:</italic> Adapting generic criteria and ranking outcomes</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Gathering evidence; systematic literature review and evaluation of evidence</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">&#x02003;<italic toggle="yes">Group exercise:</italic> Where to find resources online</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">&#x02003;<italic toggle="yes">Group exercise</italic>: Reading SAGE tables on Grading of Recommendations, Assessment, Development and Evaluations quality assessment</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Evidence-to-recommendation framework</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">&#x02003;<italic toggle="yes">Group exercise:</italic> Reviewing SAGE Evidence to Recommendation</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Experience from a long-functioning NITAG in making recommendations on human papillomavirus (HPV) vaccine</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Evolution of a NITAG&#x02019;s recommendations on HPV vaccine</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Work group and NITAG background documents</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">A well-established NITAG&#x02019;s experience with a work group on HPV vaccine</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">&#x02003;<italic toggle="yes">Group exercise:</italic> Establishing a work group</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">How NITAGs document their work</td></tr><tr><td align="left" valign="top" rowspan="1" colspan="1">Workshop recommendations and country feedback</td></tr></tbody></table><table-wrap-foot><fn id="TFN1"><label>a</label><p id="P50">PICO (Patient/Problem, Intervention, Comparison, Outcome).</p></fn></table-wrap-foot></table-wrap></floats-group></article>