#1THE WORLD BANK IMPACT EVALUATION Evaluating Impact: Turning Promises into Evidence Community-Directed Interventions and Public- Private Partnerships for Malaria Control in Akwa Ibom and Anambra State, Nigeria Dr. Godfrey Akro, Akwa Ibom Project Manager Dr. Joseph Oranuba, Anambra Project Manager Dr. Benjamin Uzochukwu, Anambra Project M&E Facilitator Mr. Essien Akpan, Akwa Ibom M&E Officer Cape Town, South Africa December 2009#2IMPACT 1. Background □ Very high disease burden EVALUATION ■ Estimated 57.5 million cases and 225,000 deaths (25% of global malaria burden; WHO 2008) ■ 11% of maternal and 29% of under five death is due to malaria Low effective LLIN coverage (use of net) ■ Under Five 3.5% and Pregnant Women 3.2% (though recent mass distribution has probably improved this; Roll Back Malaria 2007) □ Low coverage of ACTS, first-line drug ■ Coverage skewed to urban areas (Oladepo et al. 2007) ■ 2.3% of anti-malarials sold are ACTS (ACTWatch 2008) 2#3IMPACT 2. Operational Context Malaria Control Booster Project Objectives EVALUATION 1. Strengthen capacity of the Federal Government of Nigeria to provide malaria control leadership and coordination 2. Health system strengthening to improve the delivery of the Malaria Plus Package in selected Target States Project Funds 1. Malaria Control Booster Program (MCBP): $180 million 2. Additional Financing (AF): $100 million ୦#42. Operational Context (2) IMPACT EVALUATION □ IE is built into Malaria Control Booster Project and Additional Financing □ Currently, Malaria Booster Project commodities provided through public sector □ But outcomes insufficient □ So we would like to explore alternative channels for service provision Strengthen community systems ■ Increase private sector participation 4#53. Results Chain Inputs Activities Outputs Outcomes IMPACT Longterm Outcomes . Training Materials ⚫LLINS • ACTS • RDTS • SP Incentives • Human Resources (health facility workers, CDDS, PMVS, Project Management) •Technical Assistance • PMV and CDD mapping • Randomization • Training • Distribution of commodities • Demand creation (Behavior Change Communication) • Monitoring and Evaluation • Data collection • Supportive Supervision • Number of health facility workers, CDDS, PMVs trained • Number of commodities distributed (LLINS, ACTS, RDTS, SPs) • Number of social mobilization activities •Timeliness and completeness of reporting (MIS) • % increase LLINS utilization • % Increase in access to diagnosis and treatment • Knowledge, Attitude, Practices • MIS used for decision-making • Reduction in malaria morbidity and mortality •Improved health status • Improved education outcomes • Improved productivity • Informed communities • Cadre of health community- focused health workers 5#64. Primary Research Questions IMPACT EVALUATION □ How can we provide better access to effective malaria prevention and treatment services? □ What is the impact of the Community-Directed Intervention package on target outcomes? □ What is the impact of the Patent Medicine Vendor intervention package on target outcomes? □ What is the impact of both intervention packages when provided at the same time? 6#75. Intermediate Outcome Indicators IMPACT EVALUATION □ % of household members sleeping under an LLIN the previous night (children under 5, pregnant women, other groups) □ % of household members in the past two weeks receiving appropriate treatment with ACT within 24 hours of the onset of symptoms (children under 5, other groups) □ % of household members with fever in the past two weeks who were tested with RDTS and received appropriate treatment □ % of pregnant women receiving appropriate IPT with SP □ % of caretakers with correct knowledge of Home Management of Malaria 7#86. Final Outcome Indicators IMPACT EVALUATION Health □ Parasitemia incidence in children under 15 (under 5 and 5-15) □ Parasitemia incidence in women of child-bearing age (15-49) □ Malaria-related morbidity in children under 15 (under 5 and 5-15) □ Malaria-related morbidity in women of child-bearing age (15-49) □ Malaria-related morbidity in all other household members □ Malaria-related mortality among children and pregnant women □ Pregnancy outcomes. Socioeconomic □ School attendance, performance □ Malaria-related health expenditures and other costs. □ Productivity, income □ Consumption, wealth, well-being 8#96. Identification Strategy/Method Community-Directed Intervention IMPACT EVALUATION □ Randomization of all Primary Health Facilities into 50% treatment and 50% control groups □ CDI intervention package offered to all kindreds in the catchment areas of treatment health facilities Patent Medicine Vendor Intervention □ Randomization of wards into treatment and control groups □ PMV intervention package offered to all PMVs in the catchment areas of treatment health facilities. 9#106. Identification Strategy (2) Step 1: Randomize wards for PMV intervention Step 2: Randomize primary health facilities for CDD intervention CDD-linked HFs Control HFS IMPACT EVALUATION Result (example only!) • 2 clusters CDD only 2 clusters PMV only 2 clusters CDD and PMV ⚫ 3 clusters control • ୦#11IMPACT 7. Study Area Akwa Ibom State □ 25 intervention LGAs □ 264 wards (50% treatment, 50% control) □ 275 primary health. facilities (50% treatment, 50% control) □ Baseline values ■LLIN use by U5s: 8.9% ■ ACT access in 24 hours: 3% Anambra State □ 21 intervention LGAs (entire state) □ 327 wards (50% EVALUATION treatment, 50% control) ☐ 519 primary health facilities (50% treatment, 50% control) □ Baseline values □ LLIN use by U5s: 6.8% □ ACT access in 24 hours: 3% 11#128. Power Calculation PMV ACT ANAMBRA ICC/N 0 IMPACT - PMV ACT EVALUATION 5.0000 10.0000 15.0000 20.0000 25.0000 0.0237 0.0167 0.0137 0.0118 0.0106 0.0500 0.0259 0.0201 0.0178 0.0165 0.0157 0.1000 0.0280 0.0231 0.0212 0.0201 0.0195 0.1500 0.0299 0.0256 0.0241 0.0232 0.0227 0.2000 0.0317 0.0280 0.0266 0.0259 0.0255 0.2500 0.0335 0.0302 0.0290 0.0284 0.0280 0.3000 0.0351 0.0322 0.0312 0.0306 0.0303 0.3500 0.0367 0.0341 0.0332 0.0327 0.0324 PMV ACT AKWA IBOM ICC/N 5.0000 10.0000 15.0000 20.0000 25.0000 0 0.0263 0.0186 0.0152 0.0131 0.0118 0.0500 0.0288 0.0224 0.0198 0.0184 0.017 0.1000 0.0311 0.0256 0.0235 0.1500 0.0333 0.0285 0.2000 0.0353 0.0311 0.2500 0.0372 0.0335 0.3000 0.0390 0.0358 0.3500 0.0407 0.0379 0.0224 0.02' 0.0267 0.0258 0.025 0.0296 0.0288 0.02% 0.0322 0.0315 0.03' 0.0346 0.0340 0.033 12 0.0369 0.0364 0.036#138. Power Calculation CDD Net CDD NET ANAMBRA ICC/N 5.0000 10.0000 15.0000 20.0000 25.0000 0 0.0277 0.0196 0.0160 0.0139 0.0124 0.0500 0.0303 0.0236 0.0209 0.0193 0.0184 0.1000 0.0328 0.0270 0.0248 0.0236 0.0228 0.1500 0.0350 0.0300 0.0282 0.0272 0.0266 0.2000 0.0372 0.0328 0.0312 0.0303 0.0298 0.2500 0.0392 0.0353 0.0339 0.0332 0.0328 0.3000 0.0411 0.0377 0.0365 0.0359 0.0355 0.3500 0.0429 0.0399 0.0388 0.0383 0.0380 CDD NET AKWA IBOM ICC/N 5.0000 10.0000 15.0000 20.0000 25.0000 IMPACT EVALUATION 0 0.0431 0.0305 0.0249 0.0215 0.0193 0.0500 0.0472 0.0367 0.0324 0.0301 0.02% 0.1000 0.0510 0.0420 0.0385 0.0367 0.035 0.1500 0.0545 0.0467 0.0438 0.0423 0.04' 0.2000 0.0578 0.0510 0.0485 0.0472 0.04€ 0.2500 0.0609 0.0549 0.0517 0.05' 0.3000 0.0639 0.0586 0.3500 0.0667 0.0621 0.0528 0.0567 0.0558 0.055 13 0.0604 0.0596 0.059#148. Power Calculation CDD ACT IMPACT CDD ACT ANAMBRA ICC/N 5.0000 10.0000 15.0000 20.0000 25.0000 0 0.0188 0.0133 0.0108 0.0094 0.0084 0.0500 0.0206 0.0160 0.0141 0.0131 0.0125 0.1000 0.0222 0.0183 0.0168 0.0160 0.0155 0.1500 0.0237 0.0203 0.0191 0.0184 0.0180 0.2000 0.0252 0.0222 0.0211 0.0206 0.0202 0.2500 0.0265 0.0239 0.0230 0.0225 0.0222 0.3000 0.0278 0.0255 0.0247 0.0243 0.0240 0.3500 0.0291 0.0270 0.0263 0.0260 0.0257 CDD ACT AKWA IBOM ICC/N 5.0000 10.0000 15.0000 20.0000 25.0000 0 0.0258 0.0182 0.0149 0.0129 0.0115 0.0500 0.0283 0.0220 0.0194 0.0180 0.017 0.1000 0.0305 0.0252 0.0231 0.0220 0.02' 0.1500 0.0326 0.0280 0.0262 0.0253 0.024 0.2000 0.0346 0.0305 0.0290 0.0283 0.027 0.2500 0.0365 0.0329 0.0316 0.0309 0.030 0.3000 0.0383 0.0351 0.0340 0.0334 0.033 0.3500 0.0400 0.0372 0.0362 0.0357 EVALUATION 0.035 14#159. Sample and data IMPACT EVALUATION Household □ From each ward, a set of households will be randomly selected □ Data will be collected on household characteristics, income, assets, consumption, education, other socioeconomic characteristics, malaria-related KAP, health outcomes (including tests for anemia and malaria parasitemia), anthropometric indicators □ Sample size Akwa Ibom: 1,320 households Anambra: 1,635 households CDD Data will be collected on socioeconomic background, education, and malaria-related KAP □ A brief cognitive test will be administered □ Data may be collected during training ㅁ 15#169. Sample and data (2) IMPACT EVALUATION PMV □ Individual-level data will be collected on socioeconomic background, education, and malaria-related KAP (including prescription practices) □ Business-level data will be collected on shop characteristics, quality of drugs available, case load, anti-malarials and their prices, and revenues and profits Primary health facility □ Data will be collected on human resources, infrastructure, service availability, case load (uncomplicated and severe; age and sex distribution), budget 16#1710. Time Frame/Work Plan IMPACT EVALUATION PMV and CDD Mapping Training Baseline Survey Baseline Analysis Demand Creation (Behavior Change Communication) Delivery of Malarial Services to Communities End-line Data Collection Final Impact Evaluation Report Monitoring and Evaluation Supportive Supervision 2010 2011 J F M A M J JASON DJ F M A M J JASOND 17#18IMPACT 10. Sources of Financing □ World Bank Project Funds (Additional Financing) EVALUATION □ Development Impact Evaluation Initiative □ Spanish Impact Evaluation Fund □ Bank-Netherlands Partnership Program 18