GAVI Supported Evaluations

With support from Gavi, IDRC has conducted evaluations of Uganda’s immunisation programme since 2013. Below is an overview of the evaluations.

1. The GAVI Full Country Evaluation (FCE):

This evaluation was conducted in two phases. Phase one (2013-2016) was implemented in Bangladesh, Mozambique, Uganda, and Zambia, in collaboration with the Institute for Health Metrics and Evaluation (IHME) at the University of Washington (UW) and PATH in the United States. The goal of this phase was to understand and quantify the barriers to, as well as the drivers of, immunisation program improvement, with emphasis on the contribution of Gavi. The methods employed in this phase included;

• Process evaluation methods (i.e. key informant interviews, stakeholder network analysis, process tracking, observation of meetings)
• Resource tracking
• Health facility and household surveys
• A vaccine effectiveness study, including innovative methods such as the use of biomarkers
• Small-area analysis, for example, at the district level of key indicators such as immunisation coverage and child mortality

At country level, the findings and recommendations generated from this phase were used to inform decisions, prepare proposals for new Gavi support such as Health System Strengthening (HSS) support, and improve readiness for and implementation of Gavi support such as New Vaccine Introductions. At the global level, the cross-country findings and recommendations were used as inputs for review of the recently approved Health System and Immunisation Strengthening support (HSIS) framework, for updating Gavi guidelines, and for the design of new programs and initiatives, such as the strategic focus areas of sustainability, strengthening in-country Leadership, Management and Coordination (LMC), and design of the HPV 2.0 program.

Phase two (2017-2018):

The evaluation approach in phase two shifted from a largely descriptive emphasis in phase one (when shedding light on Gavi and country processes was needed) to a more targeted approach, in which the evaluation questions were developed following consultation with country and global level stakeholders. Phase two was implemented in Mozambique, Uganda, and Zambia, in collaboration with PATH in the United States. The goal of this phase was to evaluate the new policies, programs, and processes implemented by the Gavi 2016–2020 strategy. The Uganda-specific evaluation questions of this phase were:

1. What are the drivers of vaccine coverage and equity?
2. What is the effect of an interruption in Gavi HSS funding on routine service delivery, highlighting the Government of Uganda and other partner funding?
3. What are the demand-side reasons for the low coverage of HPV vaccine second dose in Uganda?
4. What are the positive and negative consequences of the new/updated Gavi processes (e.g., program capacity assessments (PCAs) and grant performance frameworks)?
5. What is the composition of the immunization partnership in the country at national and district levels?

Data were collected through document review, attendance and observation at meetings related to immunization planning and implementation, analysis of secondary data and key informant interviews with stakeholders.

2. Evaluation of the drivers of urban immunisation in Uganda, A case study of Kampala city.

Given the notable capacity in prospective evaluation built over six years of implementation of the FCE, IDRC conducted this evaluation as a prime grant recipient. The aim of this evaluation was to evaluate the extent to which the EPI programming in Uganda is designed to address immunisation in an urban context, using Kampala city as a case study. This evaluation was conducted in two phases;

Phase one (June 2019-May 2020):

The focus for this phase was the demand-side drivers of urban immunisation in Kampala city. The evaluation sub-questions of this sub-phase were:

1. To what extent is the EPI model for delivery of immunisation services in Kampala effective?
2. What are the demand side drivers of immunisation coverage in Kampala city?
3. Whether and how is the EPI program demand side approach adapting to the challenges of immunization in an urban context.

This phase employed a parallel convergence mixed methods design. Data were collected through document reviews and partner mapping, Key Informant Interviews (KIIs), Focus Group Discussion (FGDs), observation at meetings related to immunization planning and implementation, In-Depth Interviews (IDIs) and fact checking interviews. A household survey to quantify the drivers of the coverage in Kampala city was conducted among 590 caregivers of children aged 12 – 23 months.

Phase two (June 2020-May 2021):

The focus of this phase was the supply-side drivers of urban immunisation in Kampala city. The evaluation sub-questions of this sub-phase were:

1. What are the supply side drivers of immunization coverage in Kampala city?
2. To what extent is the supply side approach of the EPI program, KCCA and partners adapting to the challenges of immunisation in an urban context?
3. In what ways has COVID-19 and its control measures affected the delivery of immunisation services in Kampala city?

This phase employed a sequential transformative study design. Data were collected through document review, KIIs, IDIs, observation at meetings related to immunization planning and implementation and fact checking interviews. A health facility assessment to identify and measure supply side constraints to vaccination at health facility level was conducted at 90 health facilities in Kampala city. The health facility assessment encompassed three (3) components i.e., 1) Assessment of the vaccine delivery system, 2) Observation of immunisation sessions and 3) Exit interviews with mothers/caregivers of children aged under 5 years who received immunisation services at the health facilities.

The findings and recommendations generated from this evaluation led to the development of the urban immunisation communication plan to guide social mobilization for immunisation in urban settings, and informed the development of the urban immunisation guidelines for Uganda.

GAVI Evaluation Reports:

1. Evaluation of the drivers of Urban Immunisation in Uganda: A case study of Kampala city: August 2021.    Download Report.
2. Gavi Full Country Evaluations May 2020 Annual Dissemination Report.    Download Report.
3. Gavi Full Country Evaluations 2017-2018 Annual Dissemination Report.    Download Report.
4. Gavi Full Country Evaluations 2016 Annual Dissemination Report.    Download Report.
5. Gavi Full Country Evaluations 2015 Annual Dissemination Report.    Download Report.
6. Resource Tracking For Immunization In Uganda 2014/15 & 2015/16.    Download Report.

GAVI Debriefs:

1. Evaluation of the drivers of urban Immunisation in Uganda: a case study of Kampala city August – 2021 .    Download Report.
2. Findings from the 2017 Gavi Full Country Evaluations.    Download Report.
3. Overview of Gavi Full Country Evaluations Findings.    Download Report.
4. Findings from the 2015 Gavi Full Country Evaluations.    Download Report.
5. Findings from the 2014 Gavi Full Country Evaluations.    Download Report.