WIXLIB

FINAL EVALUATION REPORTEXECUTIVE SUMMARYI.The present document is the evaluation report of the Development Account (DA) project“Strengthening statistical and inter-institutional capacities for monitoring the Millennium DevelopmentGoals through interregional cooperation and knowledge-sharing” (ROA 146). This project focuses onincreasing the availability of up-to-date and comparable Millennium Development Goal (MDG) dataat the national, regional and global levels. The project was approved under Development AccountTranche 7 and covered a period of approximately four years (2011-2014). It was implemented bythe five United Nations regional commissions: Economic Commission for Latin America and theCaribbean (ECLAC) as the lead agency, Economic Commission for Africa (ECA), Economic Commissionfor Europe (ECE), Economic and Social Commission for Asia and the Pacific (ESCAP), and Economic andSocial Commission for Western Asia (ESCWA).II.This is an evaluation managed by the Programme Planning and Evaluation Unit (PPEU) of theProgramme Planning and Operations Division (PPOD) of ECLAC and implemented by the externalevaluator Carlos Rodríguez Ariza from March 2015.III.The objective of this evaluation is to assess the efficiency, effectiveness, relevance, and sustainability(evaluation criteria) of project implementation and, in particular, to document the results of the projectin relation to its overall objectives and expected results as defined in the project document.Additionally, the evaluation identified good practices and lessons learned, both of which can be usedwhen designing similar interventions in the future.IV. The evaluation employed a mixed methodology of quantitative and qualitative research tools in anintegrated design to enrich the process, and provided more insightful understanding. The evaluatorapplied the traditional evaluation tools in this order: (a) the desk review (qualitative) consisted of ananalysis of associated project documents to extract information and an overview of the processespromoted before conducting the evaluation fieldwork; (b) semi-structured interviews (qualitative), thatis, individual meetings in which the interviewer applied a framework of themes to explore the issues inan open way, thus allowing the interviewee to bring up new ideas and approaches; (c) focus groupdiscussion guides (qualitative) to lead meetings with groups of people who were involved in this oranalogous project activities and/or have a similar professional profile in an open way so as toprompt debate on relevant issues; and (d) self-administered surveys (quantitative) usingquestionnaires (sets of questions with closed and open answers).V.The evaluation sources included 5 regional commissions, 65 surveys, 52 interviews (28 face-to-faceand 24 carried out remotely by Skype/phone), 2 focus groups, 2 closing workshops (one at each sitevisited), and 2 site visits, involving a visit to Santiago, Chile and the case studies of two regions (firstcase study ECLAC, Chile and Argentina) and second case study (ESCWA, Lebanon and Jordan).VI. A number of potential limitations, assumptions and constraints were identified at the inceptionstage. In most cases, these were addressed or mitigated with the support of the evaluationmanagement and by triangulating information gathered from various sources in order to providestronger evidence-based conclusions.VII. Conclusions: The conclusions are divided into sections related to the evaluation criteria and theevaluation questions around project design, relevance, efficiency, effectiveness, sustainability andcross-cutting issues.(a) Programme design: The project's programme theory was appropriate and relevant ingeneral terms, but it could have been more defined, explicit, clear and better linked after thedesign and inception phases.ix

FINAL EVALUATION REPORT(b) Relevance: Although there are areas for improvement, in general the project was relevantand was aligned with global problems, regional and country needs, the regional commissions,the United Nations and the Development Account. The project could have been more relevant,especially at the interregional level. Its design was vague and it was primarily implementedon the basis of the ongoing subprogrammes. One of the main reasons behind the weakinterregional approach was that a top-down approach was used, more supported by UnitedNations Headquarters than by the regional commissions, and the time frame was too short forit to be properly owned by the regional commissions. There were complementarities andsynergies with other projects and programmes.(c) Efficiency: Project management was good, but project governance left room for improvement, andthis affected knowledge-sharing. Collaboration and coordination mechanisms left room forimprovement. Given the scale of the challenges, activities were implemented in a reasonable timelyand reliable manner, according to the priorities established by the project document.(d) Effectiveness: The project contributed to expected and unexpected results, and in generalwas effective in terms of completing the activities and in terms of contributing to the goals andobjectives outlined in the project document.(e) Sustainability: The project is sustainable even if the challenges remain. At the design stage,some elements that would have contributed to sustainability were omitted while other elementswere devised but not implemented.(f) Cross-cutting approaches: Gender and human rights and knowledge management There was no common cross-cutting gender perspective. Monitoring presented somechallenges related to (i) the need to be less activity- and data-focused and more focusedon analysis; and (ii) the lack of capacity for monitoring and the use and sharing of theresulting knowledge. The progress reports were accurate but failed to capture existing information regardingthe quality of the events, such as workshops. The project did not have a formal and explicitexit strategy or a knowledge-sharing strategy. It did not generate enough opportunitiesfor sharing its achievements and experiences among the regional commissions.VIII. Lessons learnt. The evaluation also reveals some lessons learnt that could be useful for the followingpurposes: (a) to improve the process of design and inception of interregional projects; (b) to developbetter implementation plans; (c) to improve monitoring and reporting; (d) for better consideration ofthe implications of an interregional project in terms of organization, governance, coordination,collaboration and financial management; and (e) to improve the development of exit strategies,knowledge management and knowledge-sharing.IX. Recommendations. There are six recommendations arising from this evaluation that are explained inmore detail in the main report and which highlight the priorities, those in charge and some possibleelements of an action plan for the management response. These recommendations relate to: (a)improving the process of designing interregional projects; (b) developing better implementationplans; (c) improving monitoring and reporting; (d) better consideration of the implications of aninterregional project in terms of organization, governance, coordination, collaboration and financialmanagement; (e) improving exit strategies and knowledge management at the activity and projectlevels to ensure sustainability; and (f) better integration of gender throughout the project cycle.x

FINAL EVALUATION REPORT1. INTRODUCTION1.This document is an end-of-cycle evaluation of an interregional project that focuses on increasing theavailability of up-to-date and comparable Millennium Development Goal (MDG) data at thenational, regional and global level. This is an evaluation managed by the Programme Planning andEvaluation Unit (PPEU) of the Programme Planning and Operations Division (PPOD) of ECLAC andcarried out by the external evaluator Carlos Rodríguez Ariza.2.The unit of analysis or object of study for this evaluation is the project “Strengthening statistical andinter-institutional capacities for monitoring the Millennium Development Goals through interregionalcooperation and knowledge-sharing” (ROA 146). It is understood as the set of components, outcomes,outputs, activities and inputs that were detailed in the project document and the associatedmodifications made during implementation.3.The objective of this evaluation is to assess the efficiency, effectiveness, relevance, and sustainability(evaluation criteria) of project implementation and, in particular, to document the results of the projectin relation to its overall objectives and expected results as defined in the project document.Additionally, the evaluation process identified good practices and lessons learned, both of which canbe used when designing similar interventions in the future.4.The evaluation process comprised a global analysis involving desk review and surveys combined witha visit to Santiago/Chile and case studies of two regions (first case study ECLAC-Chile and Argentinaand second case study ESCWA –Lebanon and Jordan).2. BACKGROUND OF THE PROJECT2.1. BACKGROUND OF THE PROJECT5.The need to monitor progress towards the Millennium Development Goals has provided anopportunity for many developing countries in different regions to develop their statistical systems andproduce better information in support of evidence-based policies for development. Nevertheless, asthe period assigned for the achievement of the Millennium Development Goals was almost over,developing nations faced pervasive statistical challenges in monitoring them. These included persistentdata gaps, insufficient use of the official national data produced by both the national agency incharge of the national Millennium Development Goal reports, as well as by the internationalagencies; and the statistical discrepancies among indicator values produced or compiled by national,regional and international sources. These issues are of great concern in the international community,particularly within the statistical and reporting communities in developing countries. The statisticalcommissions and other regional statistical bodies (such as the Statistical Conference of the Americas)have formally recognized these problems and have urged stakeholders to take immediate action tofind solutions. Key points have included strengthening statistical capacities for data and metadataproduction, and improving statistical information exchange among international agencies anddeveloping countries, as recommended in 2007 by the Statistical Commission (E/CN.3/2007/13).6.The entire United Nations system has prioritized these matters, on the premise that there should alsobe more transparency in the way that international agencies use imputed data. The Inter-Agency andExpert Group on Millennium Development Goal Indicators recommended producing detailedexplanations of the difference between global and national monitoring. These included implicationsfor methods and data presentation, and constructing metadata on population estimates used asdenominators to calculate many Millennium Development Goal indicators. The Inter-Agency and1

FINAL EVALUATION REPORTExpert Group has further recommended involving the regional commissions more extensively inreviewing the discrepancies between national and international sources, in assisting in theorganization of data and metadata exchanges between national statistical systems and internationalagencies, and in improving data-sharing at the international level.7.From the regional perspective, responding to the demands of its member countries, the five regionalcommissions have taken on an increasingly active role in strengthening statistical capacities in relationto the Millennium Development Goals in their respective countries and promoting collaborative workwithin countries and regions in order to share experiences and learn from each other. The statisticaldivisions of the regional commissions have been coordinating efforts to resolve data gaps anddiscrepancies between national and international sources. A joint document was presented fordiscussion at the Inter-agen

PRODOC Project document RC Regional Commission SDG Sustainable Development Goal SDMX Statistical Data and Metadata eXchange SSI Semi-structured interviews TAMs Technical advisory missions ToR Terms of reference UN United Nations UNCTAD United Nations Conference on Trade and Development UNDG United Nations Development Group .