If you are interested in this topic, review the webinar presentation, watch the recorded webinar, or read the study report.
The AEN Evidence 2020 is a biennial event hosted by the Africa Evidence Network, which is designed to foster collaboration, facilitate dialogue, and move forward-thinking, practice, and action to advance the use of evidence in decision-making across the continent. This pre-eminent event aims to advance the African evidence ecosystem by building on and moving forward the work of the previous Evidence conferences and one of the key focus areas is impact evaluation.
Impact evaluation provides information about the impacts produced by an intervention or a project. Impact evaluation according to the William and Flora Hewlett Foundation is a type of evaluation design that assesses the changes that can be attributed to a particular intervention. As cited in the report, it is based on models of cause and effect, therefore, it requires a control group or comparison group to control for factors other than the intervention that might account for the observed change. It is documented that impact evaluations assess the degree to which changes in outcomes can be attributed to intervention rather than to other factors which are confounders. It is important that such attribution requires knowing what outcomes would have looked like in the absence of the intervention to be evaluated. Impact evaluation aims to control and or account for confounders that are likely to distort the results, findings, or outcomes of an intervention or programme. According to Dr. Kremer, Professor of Development Studies at Harvard University, impact evaluation is an accountability tool at the end of a project cycle because by using impact evaluation it helps to remove any assumptions about the impact made by an intervention.
The Africa Centre for Evidence (ACE) was commissioned by the William and Flora Hewlett Foundation to conduct a scoping study on impact evaluation capacity in Sub-Saharan Africa and the study was done between July 2018 and May 2019. The purpose of the scoping study was among other objectives to understand the extent and nature of existing impact evaluation capacity in Sub-Saharan Africa, especially East and West Africa, for both the production and use of impact evaluations. It was also to understand how impact evaluation capacity could be developed and what opportunities are there to develop the capacity and by whom the capacity should be developed.
The project was aligned to the William and Flora Hewlett Foundation Evidence-Informed Policy-making strategy under the Global Development and Population Programme which focuses on the long-term goal of supporting the systematic use of good quality evidence by governments to improve social and economic policies over time. The scoping study used a multi-stage and multi-component qualitative design to conduct the study and it included key informant discussions, an online survey and follow-up interviews, a desk review of training resources, and a systematic search to identify authors who have published impact evaluations.
The results show evidence of impact evaluation capacity in sub-Saharan Africa contrary to the general perception suggesting a limited capacity to undertake, document, and publish impact evaluations. The study identified 1520 unique African researchers with African affiliations across 34 different countries which have authored 490 impact evaluation publications between 1990 and 2015. This is evidence of available capacity and skills in Sub-saharan Africa. From the results, only 210 (13%) of these 1520 were first authors from Sub-saharan Africa. From the search by author publications, South Africa had the most impact evaluation researchers who had published, followed by Kenya, Uganda, Tanzania, and Zambia. The results show a concentration of impact evaluation capacity in Southern and Eastern Africa. However, there were also 337 authors across 14 countries in Western Africa involved in impact evaluation indicating the growing capacity in that region.
The study also reviewed that Southern Africa was leading in terms of authors involved in impact evaluation at 591 (39%), followed by East Africa at 555 (37%), then West Africa at 337 (22%), and lastly Central Africa with 37 (2%).
Apart from that, the study also showed that the health, nutrition, and population sector had more impact evaluation authors and articles published at 377 articles which is 77% of the total studies identified. Other sectors shared the remaining 23% of articles with Agriculture and rural development coming second with 49 impact evaluations and the education sector a distant third on publication of impact evaluations 37 studies. Other sectors represented in the publications include finance, social protection, water and sanitation, energy, ICT, public management, and multi-sector. The study also noted that impact evaluation training is taking place through different institutions as accredited courses, short courses, and online courses varying from country to country and region to region.
About the author
Enock Musungwini is a Reference Group Member for AEN. Enock recently completed an MSc Public Health degree at London School of Hygiene and Tropical Medicine, London UK and was awarded the Amanda Jaclyn Berger prize for the Best MSc Research Project under the Public Health programme.
The views expressed in published blog posts, as well as any errors or omissions, are the sole responsibility of the author/s and do not represent the views of the Africa Evidence Network, its secretariat, advisory or reference groups, or its funders; nor does it imply endorsement by the afore-mentioned parties.