By Laurenz Langer
South Africa is a vibrant space for evidence-informed decision-making (EIDM) with an increasingly global reputation. The country’s Department for Planning, Monitoring and Evaluation (DPME) has headed the implementation of a nation-wide evaluation system with evaluation evidence being discussed all the way up to Cabinet meetings. Every government administration also targets its work to a national outcomes framework generating an ongoing demand for a range of evidence to provide insights on the progression towards these national key outcomes. Arguably, South Africa thereby has created one of the most sophisticated institutional frameworks for the use of evidence to inform the design of public policies and programmes on a global scale.
DPME’s work on evidence use has led to a plethora of different types of evidence being produced and positioned to be of relevance to public servants: from Socio-Economic Impact Assessments and impact evaluations, to citizens-based monitoring and institutional performance monitoring, more and more evidence is competing for uptake in the policy-making space. This can be quite confusing and overwhelming for a time-pressed public servant trying to translate political and societal objectives into practical programmes and policies. Beginning in 2015, DPME therefore investigated the possibility to develop a hub of all this policy-relevant evidence and to organise it in a user-friendly manner so that public servants could draw from the whole body of evidence (rather than single studies) to inform their decisions. In this process, led by the research team, DPME experimented with different methods for evidence synthesis, which is a specific type of research that allows decision-makers to systematically engage with bodies of evidence. Following external and internal consultation, the Department decided to conduct an evidence map in the Human Settlements Sector in an effort to inform the design of the Draft White Paper on the transition from Housing to Human Settlements in South Africa.
Evidence synthesis and evidence mapping is a specialised skill and DPME therefore partnered with the University of Johannesburg’s programme to Build Capacity to Use Research Evidence (BCURE). BCURE has extensive experiences in conducting different types of evidence syntheses in different sectors and has been working with South African government departments, including DPME, since 2014. DPME also brought in experienced human settlements researchers as well as decision-makers in a collaborative effort that saw the team accessing over 2,000 pieces of human settlements evidence. From these 2000+ pieces of evidence, the team systematically and transparently filtered the evidence most relevant to South African decision-makers according to agreed criteria. This evidence was then appraised for its trustworthiness and inputted into an interactive online interface. This interface visualises multiple maps of evidence allowing human settlements decision-makers to quickly identify different bodies of evidence and whether they can inform their policy questions. 1-page summaries of all evidence as well as links to full-texts are available too on decision-makers’ demand.
Seven steps to creating an Evidence Map
On 19 September, DPME officially launched the produced evidence to a local and international audience. Having validated the evidence mapping methodology for its usefulness internally consulting with and presenting the evidence map to different government audiences, the international launch aimed to communicate DPME’s evidence mapping programme to a larger group of stakeholders and to celebrate the successful completion of the pilot map. High-profile, local and international, speakers were invited to reflect on DPME’s evidence map including Prof John Lavis (McMaster University, Canada), Nonhlanhla Buthelezi (DHS), Birte Snilstveit (3ie, London), Dr Temba Masilela (HSRC), Ulrike Rwida (National Treasury), Dr Ian Goldman (DPME), and Prof Mark Napier (CSIR). The launch also included the viewing of a short film about the evidence mapping process (see here). Afterwards, attendants broke for dinner and drinks.
Presentation at the international launch of the Evidence Map, 19 September 2016
Discussions and feedback at the launch as well as subsequent presentations of the evidence map at international conferences (Evidence 2016 and the What Works Global Summit) underlined the innovative nature of DPME’s approach to the use of evidence. The process of conducting the evidence map itself as well as the final product introduced a range of new mechanisms to the EIDM community. For example, there was great interest in DPME’s and UJ-BCURE’s approach to co-production and match-making to foster the use of evidence. Decision-makers in policy contexts, both in South Africa and in the UK, praised that the evidence map presented more of a method to inform decision-making in the public sector than a traditional research tool. There is already local and international demand to conduct similar policy-relevant evidence maps and DPME’s model is likely to shape the future of evidence mapping in the EIDM space.
A detailed report and academic paper reflecting on the process of conducting a policy-relevant evidence map is currently being produced. For more information please contact Harsha Dayal or Laurenz Langer.