The COVID-19 pandemic led to substantial morbidity and mortality in the African region, with more than 500 million infections and more than 400,000 deaths estimated during 2020–2021. [1] Although highly effective COVID-19 vaccines became authorized and available in early 2021, capacities for low-income countries to equitably procure vaccines was cause for concern, including those in the African region. [2] COVID-19 Vaccines Global Access (COVAX) was established by the World Health Organization (WHO), GAVI, and the Coalition for Epidemic Preparedness Innovation (CEPI) was established to ensure rapid and equitable access to COVID-19 vaccines for all countries regardless of income level. [3] COVAX delivered the first doses of COVID-19 vaccines in West and Central Africa by the beginning of March 2021. [4]

Since the introduction of different COVID-19 vaccines, studies of vaccine effectiveness (VE) have aimed to characterize their real-world performance against a range of clinical endpoints, over time, and against successive SARS-COV-2 variants. Performance of COVID-19 vaccines could plausibly differ by regional setting and among populations that may differ in background comorbidities, immunological profiles, ethnicity, and age structures. Performance may also differ due to changes and adaptations in the virus, by vaccine formulations and vaccine products, or by stability of the vaccine in real-world conditions. [5,6] Multiple regional VE networks were either established or modified to estimate COVID-19 VE, including the European Severe Acute Respiratory Infection Vaccine Effectiveness (Euro-SAVE) Network, the Influenza – Monitoring Vaccine Effectiveness in Europe (I-MOVE) network, and the Network for the Evaluation of Vaccine Effectiveness in Latin America and the Caribbean (REVELAC). [7–9] To support capacity for VE estimates in the African region, the African Region Monitoring Vaccine Effectiveness network (AFRO-MoVE) was established by the WHO Regional Office for Africa (WHO AFRO) in March 2021. [10] The AFRO-MoVE secretariat is composed of staff from WHO AFRO and EpiConcept, a French company contracted to provide domain expertise and services in the field of public health.

In addition to assessing vaccine performance against clinical endpoints, another aim of producing estimates of COVID-19 VE in the AFRO region was to contribute to the shaping and strengthening of COVID-19 vaccine policy. More broadly, building regional capacity to conduct vaccine effectiveness studies also aimed to strengthen the scientific community in Africa with capacity to evaluate new vaccines. The objectives of AFRO-MoVE to support current and future estimates of vaccine effectiveness in the WHO AFRO region are summarized in Box 1. [10]

To assess successes and challenges of AFRO-MoVE in meeting its objectives, an evaluation was initiated in September 2023. Primary goals of the evaluation were to assess how AFRO-MoVE addressed the network’s objectives in supporting regional COVID-19 vaccine effectiveness studies, to describe associated challenges and opportunities in meeting these objectives, and to make recommendations to strengthen future efforts. The specific objectives of the evaluation were to (1) evaluate how the secretariat and network were established and functioned; (2) review country and site-specific activities conducted to generate VE estimates as context for network contributions; (3) summarize the outputs of network activities; and (4) make recommendations for the network on how to improve and build on network activities and the capacity for conducting VE or other related studies in the WHO AFRO region in the near and long-term.

The evaluation of the AFRO-MoVE network was coordinated by WHO AFRO and the AFRO-MoVE secretariat and was conducted by an external team led by U.S. Centers for Disease Control and Prevention (US CDC) alongside scientists from the African Region. Evaluation of AFRO-MoVE combined formative and process evaluation approaches, as described by Berkowitz et al. in 2008, to assess network performance against its stated objectives. [11] The evaluation scope was limited to the contributions of AFRO-MoVE on coordination, financing, facilitation, and capacity building for implementing VE studies across different countries in the region, and did not include assessment of the individual studies nor methods used to generate pooled VE estimates.

The AFRO-MoVE network includes over forty different organizations across eighteen African countries, as well as collaborating international partner organizations (Supplementary Table 1). For evaluation purposes, countries were categorized based on the level of support they received from AFRO-MoVE for conducting VE studies: (Group A) countries receiving both financial and technical assistance to implement VE studies, (Group B) countries implementing VE studies receiving technical assistance but not receiving direct financial support from AFRO-MoVE, and (Group C) all other partners engaging with the network. All countries in Groups A and B are summarized in Table 1.

To fully assess the objectives described in the scope of the evaluation, the evaluation team used multiple sources of information that included both primary and secondary data. A mixed-methods approach was used to synthesize information from the following sources:

Thematic analysis techniques were used to analyse the qualitative data collected during the key informant interviews. This analysis followed a deductive approach, using a set of predefined codes that were informed by the results of the AFRO-MoVE stakeholder survey, conversations with the AFRO-MoVE secretariat, and subject matter expertise. Perceived challenges were summarized together with perceived support by the network in addressing each challenge. Data were collected and exported via REDCap and summarized using Microsoft Excel. Given non-random sampling and small sample sizes from the survey response, key themes, rather than proportions, are presented.

A timeline of study activities was developed and contextualized using the number of monthly COVID-19 cases reported to the World Health Organization for the seven countries funded by AFRO-MoVE — Ghana, Malawi, Mali, Namibia, Senegal, Tanzania, and Uganda. The average across all seven countries was calculated for each month from March 2020 through June 2024 and graphed using Microsoft Excel. Data were obtained from the WHO COVID-19 Dashboard. [12]

The evaluation protocol was reviewed and received ethical approval by WHO AFRO and the AFRO-MoVE Scientific Steering Committee. This activity was reviewed by CDC, deemed not research, and was conducted consistent with applicable federal law and CDC policy.¹ Participants provided oral consent for interviews to be recorded and transcribed.

As a technical network, AFRO-MoVE provided a forum to share expertise, common protocols, scientific strategies, and updates of vaccine effectiveness within the WHO AFRO region, leading to strengthened individual studies. These achievements are particularly notable given the context of an ongoing health emergency caused by the COVID-19 pandemic. As a result of shared expertise, shared protocols, data management and analysis support, and technical expertise, AFRO-MoVE also provided an opportunity to pool estimates between studies within the AFRO region. Although objectives to enable AFRO-MoVE funded studies to publish national or regional estimates were largely unrealized at the time of writing, other African institutions did produce VE studies with high-impact findings. [17–19] AFRO-MoVE’s goals were also broader and longer-term. An increase in regional capacity for vaccine effectiveness was also intended to strengthen an African-led strategy, support regional advocacy, and provide preparedness against future threats.

To increase capacity for VE studies in the region, AFRO-MoVE employed a variety of strategies, including provision of technical expertise, resources, and shared protocols. AFRO-MoVE coordinated information-sharing updates across at least ten different COVID-19 vaccine effectiveness studies taking place in the AFRO region that were funded or technically supported by the network, and provided a venue for updates from additional VE studies that may not have been directly supported by the network. Network meetings and technical assistance webinars were well-received by network stakeholders, and opportunities for ‘experience sharing’ across countries and study sites were frequently highlighted. There was broad agreement that the technical assistance provided by the network was valuable to countries implementing VE studies. More directly, respondents considered the network to be substantially effective in providing financial support, and moderately effective in enabling access to appropriately trained staff.

However, there were also myriad challenges to generating reliable and timely national and regional estimates of COVID-19 vaccine effectiveness, some within the control of AFRO-MoVE or host country governments to address, while other factors were largely outside of stakeholder control. Establishing new studies takes time, but there was agreement that among respondents that further streamlining the approval process for study protocols and improved processes for transfer of funds to study sites may decrease the time required to initiate studies in the future. Other significant challenges such as local uptake of vaccines and test availability may have been outside AFRO-MoVE’s control, but future efforts in the region might look to increase the coordination between VE studies and vaccination campaigns, as low rates of vaccine uptake and limited number of severe cases are significant ongoing challenges to conducting VE studies for COVID-19. It has been documented that low and middle-income countries with existing seasonal influenza vaccination programs were better prepared for the COVID-19 pandemic compared to other countries where the influenza vaccination platform was weak or non-existent. [20] Low uptake and prioritization of influenza vaccine in many African countries may have adversely impacted preparedness for SARS-COV-2 in the region. [21] In addition, respondents reported a substantial challenge in ensuring complete vaccination status information, data which are crucial for generating reliable vaccine effectiveness estimates. [22] Improved linkage between vaccine registry programs and relevant study sites or SARI surveillance systems may reduce this challenge in the future, both for SARS-CoV-2 and other respiratory pathogens like influenza and RSV.

The decline in both reported cases and vaccination uptake in many countries just as studies were established proved challenging both in terms of study operation and relevance of studies being conducted. Once studies were initiated, the decline in reported COVID-19 cases exacerbated difficulties with enrolling cases, which may limit statistical power and increase the risk of selection bias. Since COVID-19 vaccine effectiveness wanes over several months, time-since-vaccination is a key parameter in analysis. There is a relatively short window to include an intervention group with recent receipt of vaccines, and therefore estimate peak effectiveness, a particular challenge for COVID-19 VE studies in a region where low uptake of COVID-19 vaccines was considered to be a substantial challenge. [23] Lastly, a variety of other challenges were also acknowledged, such as difficulty controlling for prior infections and the wide array of potential biases that can distort VE estimates. [6]

Although the objective of generating timely vaccine effectiveness estimates was not met by the time of writing, this should not obscure the network’s underlying contribution to strengthening capacity, or its success in addressing other longer-term objectives. Establishing new studies is challenging, and especially so during a global pandemic. Others have noted that leveraging or adapting pre-positioned platforms was often effective in providing timely data to inform the COVID-19 response, such that maintenance of systems that can more rapidly be modified and strengthened may have longer-term benefits against future threats. [24]

Evaluation of regional capacity, advocacy, and a specific strategy was limited by information provided, and assessment of capacity gains relied on reported information and was not evaluated directly. The response rate to the electronic stakeholder survey was low, and data collected through key informant interviews and an electronic survey may not be representative of the network as a whole. Given the distribution of responses to key informant interviews, our evaluation team was not able to ascertain systematic differences in the challenges faced between those study teams conducting test-negative design and cohort studies, despite previously documented challenges generally observed within each type of observational study. [6] The evaluation did not seek to assess individual studies, but rather the overall performance of the network in supporting the implementation of VE studies in the region. As such, only a small number of study protocols were available and no patient-level data were reviewed, limiting insights into the quality of the studies conducted. This evaluation did not capture information related to AFRO-MoVE’s activities to advocate for additional funding for study sites, national authorities, or other organizations.

This evaluation identified potential opportunities in the near and long-term for AFRO-MoVE. Most immediately, the network’s impact might be strengthened by completing planned analyses and sharing findings through a planned publication. Although direct impact on vaccine policy may have been minimal, this is an important opportunity for the network to highlight capacity gains in the region. Completion of the pooled analysis may also help to demonstrate the viability of this approach in the future. AFRO-MoVE and other regional stakeholders could also develop a clearer strategy for VE studies in the region to support ongoing work and bolster pandemic preparedness for the next novel pathogen. Such a strategy would ideally include a decision regarding VE study aims (e.g. annual estimates), study designs (cohort, test-negative, or other case-control), increased standardization across study protocols to better facilitate pooled analysis, and power considerations (e.g. national estimates vs regional estimates). Furthermore, countries with the requisite capacity for implementing VE studies might aim to include VE studies as part of their national research agendas and advocate to maintain or increase investments in genomic sequencing capacity, enabling timely characterization of circulating variants and subvariants.

There is a need to determine if the network’s desired role is that of a community of practice where technical experts can share updates, knowledge, and resources related to vaccine effectiveness studies or, additionally, a platform for regional VE estimates similar to the models seen in Europe and South America that feature standardized, pooled data from targeted locations with ongoing COVID-19 vaccination efforts. The latter option is highly likely to be the more impactful, yet resource-intensive, opportunity. If neither option is feasible, consideration should be given to phasing out the network given that its specific purpose for supporting COVID-19 VE studies may be less relevant in the region beyond 2025.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.vaccine.2025.126823.