Independent Terminal Evaluation Improve The Health And . - UNIDO

Transcription

Independent Terminal EvaluationIndependent Terminal EvaluationImprove the Health and Environment of Artisanaland Gold Mining Communities in the Philippinesby Reducing Mercury EmissionsUNIDO ID: 120016GEF Project number: 5216

UNIDO INDEPENDENT EVALUATION DIVISIONIndependent Terminal EvaluationImprove the Health and Environment of Artisanaland Gold Mining Communities in the Philippinesby Reducing Mercury EmissionsUNIDO SAP ID: 120016GEF Project number: 5216Vienna, March 2017

Distr. GENERALODG/EVQ/IEV/16/R.13March 2017Original: EnglishThis evaluation was managedby the responsibleUNIDO Project Managerwith quality assurance by theIndependent Evaluation DivisionThe designations employed and the presentation of material in this publication do not imply theexpression of any opinion whatsoever on the part of the Secretariat of the United NationsIndustrial Development Organization concerning the legal status of any country, territory, city orarea or of its authorities, or concerning the delimitation of its frontiers or boundaries.Mention of company names and commercial products does not imply the endorsement ofUNIDO.The views and opinions of the team do not necessarily reflect the views of the involvedGovernments and of UNIDO.This document has not been formally edited.

Table of ContentsList of acronyms and abbreviations . 4Glossary of evaluation-related terms . 5Executive SummaryAIntroduction BEvaluation findings and conclusions CRecommendations DLessons learned Part II.1I.2I.3I.47789Evaluation BackgroundInformation on the evaluation Scope and objectives of the evaluation .Information sources and availability of information Methodological remarks, limitations encountered, andvalidity of the findings 10101011Part IIII.1II.2II.3II.4Country and Project BackgroundProject background .Project summary Project implementation arrangement . .Positioning of UNIDO 1212121416Part IIIABCDEFGH161617192426272828IJKLProject AssessmentDesign .Relevance .Effectiveness .Efficiency Sustainability of project outcomes .Assessment of monitoring and evaluation Monitoring of long-term changes Assessment of processes affecting achievement of projectresultsProject coordination and management Gender mainstreaming Procurement Overall assessment Part IVIV.1IV.2IV.3Conclusions, Recommendations, and Lessons LearnedConclusions Recommendations Lessons learned 36363738AnnexesAnnex 1 Terms of reference Annex 2 List of documents consulted Annex 3 List of persons key informants interviewed .31133333434397678

LIST OF ACRONYMS AND NPMCCUP-PGH-NPCWHOArtisanal and Small-Scale Gold MiningBureau of Products and StandardsBureau of CustomsBangko Sentral ng PilipinasBan ToxicsChemical Control OrderDepartment of Environment and Natural ResourcesDepartment of HealthDepartment of Labor and EmploymentDepartment of Trade and IndustryEnvironmental Management BureauForeign-Assisted & Special Projects ServicesFood and Drug AdministrationGlobal Environment FacilityGlobal Mercury ProjectLocal Government UnitMines and Geosciences BureauMercury Initial AssessmentNational Action PlanNational Association of Small-Scale MinersNon-Governmental OrganizationNational Commission on Indigenous PeoplesNational Implementation PlanNational Project CoordinatorNational Project ManagerNational Strategic PlanOperational Focal PointOccupational Safety and Health CentreProject Implementation ReviewProject ManagerPersistent Organic PollutantsProject Steering CommitteeStrategic Approach to International Chemicals ManagementTerminal EvaluationTerms of ReferenceTechnical Working GroupUnited NationsUnited Nations Environment ProgrammeUnited Nations Industrial Development OrganizationUnited Nations Institute for Training and ResearchUniversity of Philippines – National Poison Management and Control CenterUniversity of Philippines – Philippines General Hospital – National PoisonCentreWorld Health Organization

GLOSSARY OF EVALUATION-RELATED TERMSConclusionsConclusions point out the factors of success and failure of the evaluatedintervention, with special attention paid to the intended and unintendedresults and impacts, and more generally to any other strength or weakness. Aconclusion draws on data collection and analyses undertaken, through atransparent chain of arguments.LogframeManagement tool used to improve and facilitate the planning, design,implementation and monitoring of interventions, most often at the projectlevel, also in literature referred to as LFA – Logical Framework Approach. Itinvolves identifying strategic elements of the project (inputs, activities,expected deliverables (outputs), specific objective (outcome) and overallobjective (goal)) and their causal relationships (“results chain”), indicators,and the assumptions or risks that may influence success and failure. Relatedterm is Results-based Management (RBM).OutcomeThe likely or achieved short-term and medium-term effects (including policyand institutional changes) of an intervention’s outputs. Will materialise afterthe intervention outputs have been delivered and is clearly outside the controlof the project management.(Expected)Results/OutputsThe products, capital goods and services which result from the activities of adevelopment intervention (the “deliverables”); may also include changesresulting from the intervention which are relevant to the achievement ofoutcomes. The outputs are fully within the responsibility and control of theproject management.EffectivenessThe extent to which the development intervention’s results/ outputs andobjectives were achieved, or are expected to be achieved, as compared tothe work plans and budgets, taking into account their relative importance.EfficiencyA measure of how economically resources/inputs (funds, expertise, time, etc.)are converted to results/outputs. It is thus a measure of productivity.ImpactsPositive and negative, primary and secondary long-termconsequences/effects/ results produced by a development intervention,directly or indirectly, intended or unintended, foreseen or not foreseen. Sucheffects could be economic, political, social, technical or environmental, bothon local and national level, primary and secondary. (A related term is“outcome”, but this is normally used directly related to the planned effect ofthe project outputs).RelevanceThe extent to which the objectives of a development intervention areconsistent with beneficiaries’ requirements, country needs, global prioritiesand partners’ and donors’ policies.Note: Retrospectively, the question of relevance often becomes a question asto whether the objectives of an intervention or its design are still appropriategiven changed circumstances and framework conditions.IndicatorQuantitative or qualitative factor or variable that provides a simple andreliable means to measure achievement, to reflect the changes connected toan intervention, or to help assess the performance of a development actor.Indicators should preferably be measured in quantitative terms, but alsoqualitative indicators are used. Indicators should be SMART (specific,5

measurable, achievable, realistic and time-bound)LessonslearnedGeneralizations based on evaluation experiences with projects, programs, orpolicies that abstract from the specific circumstances to broader situations.Frequently, lessons highlight strengths or weaknesses in preparation, design,and implementation that affect performance, outcome and impact.RecommendationsProposals aimed at enhancing the effectiveness, quality, or efficiency of adevelopment intervention; at redesigning the objectives; and/or at thereallocation of resources. Recommendations should be linked to conclusions.ResultsThe outputs, outcomes or impact (intended or unintended, positive and/ornegative) of a development intervention at various levels and points in time(the Results Chain refers).SustainabilityThe likelihood of continuation of benefits (and prospects of the expansionand/or replication thereof) from a development intervention after majordevelopment assistance has been completed, and the external funding hasended (the probability of continued long-term benefits/impact). The resilienceto risk of the net benefit flows over time.

Executive SummaryA.Introduction1.The Global Environment Facility (GEF) medium size project (MSP)“Improve the Health and Environment of Artisanal and Gold MiningCommunities in the Philippines by Reducing Mercury Emissions” wasimplemented from March 2013 to June 2016 by the United Nations IndustrialDevelopment Organization (UNIDO). The main national partners of the projectwere the Department of Environment and Natural Resources (DENR),Department of Health (DOH), and Ban Toxics (BT), with the following financingsources: GEF: 550,000; co-financing (cash and in kind): 1,631,070; Total: 1,631,070.2.The overall objective of the project was to improve the health andenvironment of artisanal gold mining communities in the Philippines byreducing mercury emissions. In particular, the project aimed to introducemercury-free technology in 2 small-scale mining areas, and to supplement thiseffort by providing health training to rural health care workers in the properdiagnosis of mercury poisoning.B.Evaluation of findings and conclusions3.The main purpose of this terminal evaluation was to assess theperformance of the project (in terms of relevance, effectiveness, andefficiency), to determine its impacts (actual and potential) including theirsustainability, and to propose a set of recommendations in view of ongoingactivities and replication.4.This project is highly relevant as the Philippines has signed theMinamata Convention. With the assistance of UNEP and UNITAR, it isundertaking activities for an early ratification of the Convention. Furthermore,recognizing the need for sound management of mercury and mercurycontaining wastes, it has developed a NAP on mercury to reduce releasesincluding in the ASGM sector.5.The project is consistent with the GEF 5 Focal Area Strategy forChemicals. In particular, in strengthening local and national capacity toeffectively reduce mercury use, emissions, and exposure in artisanal goldmining communities, the project is very relevant with Outcome 3.2 of GEFChemicals focal area.7

6.Effectiveness of the project is considered satisfactory. Most of the statedobjectives have been successfully achieved. A significant number of miners,including both males and females, have been trained to use mercury-freemethod to extract gold, and a number of them have already shifted to thismercury-free technique. As a result, the use of mercury for gold mining hasbeen considerably reduced in the project areas, and the communities are fullyaware of the dangers of mercury on human health and environment. Themobilization of significant co-funding increased efficiency of the project.7.The approach originally agreed upon by stakeholders was adopted inproject implementation. The overall project management and supervision wassatisfactorily done by a UNIDO PM who was adequately assisted by supportstaff. At the national level, the project was satisfactorily executed by DENR,DOH, and Ban Toxics. However, delay due to changing of project sitedecreased efficiency to some extent.8.Some financial and socio-political risks that could jeopardizesustainability of project outcomes have been identified. However, as theserisks can be easily mitigated by appropriate measures, chances for continuoussustained impact of the project are considered high.C.Recommendations9.The project has successfully been completed, achieving most of thestated objectives. For continued relevance and sustainability of projectoutcomes, this terminal evaluation proffers the following recommendations:i.The project has been successful and has produced tangible results.In particular, the project has been quite successful in training theminers to shift to mercury-free method, thus reducing the use ofmercury at the project sites and lessening exposure to mercury (asobserved in miners during the health assessments). It isrecommended (for BT and DENR) that these positive outcomes besummarized and disseminated to other ASGM communities in thePhilippines.ii.Some miners have indicated that the initial investment cost to shift tomercury-free method might constitute a barrier. It is recommended(for LGUs, BT, and miners’ association) that a mechanism forfinancial assistance be set in order to facilitate shift to mercury-freegold mining in the ASGM sector.iii.For continued relevance and impact of the project, when the CCO onmercury is adopted, the relevant authorities (DENR) should ensurethat it is strictly enforced to make availability of mercury difficult.

iv.C.The results of the health assessment have not yet been disclosed tominers. DOH should proceed rapidly to inform the miners about theoutcome of these assessments.Lessons Learned10.Valuable lessons, which emerged during the implementation of thisproject, include:i.The project site had to be changed as the LGU of one of the projectsites was no longer supporting the project, thereby causing a delayof about one year in the implementation process. Hence, securingthe commitment of partners ahead of time through signed agreementcan avoid delays in project execution.ii.At the beginning, when the project, aiming at reducing the use ofmercury in gold mining, was introduced to miners, they werereluctant to participate for various reasons including: loss of income ifnot working; mercury method is quick to extract gold. However, whenthe miners’ communities were informed about the health problemsthat they might suffer from if exposed to mercury, they graduallystarted to change their mind. In particular, the women, knowing thatthey will be particularly affected if exposed, convinced the miners toparticipate in the project. Engaging proper awareness-raisingactivities and building their confidence are the basis to secure thecommitment of the communities.

Part I Evaluation BackgroundI.1 Information on the evaluation11.This terminal evaluation (TE) was undertaken in compliance with GEF 1and the UNIDO 2 evaluation policies in order to promote accountability for theachievement of the project objectives through the assessment of results,effectiveness, processes, and performance of stakeholders involved duringproject implementation.12.The evaluation was undertaken from September 2016 – November 2016by a team of independent consultants constituted by Dr. Nee Sun CHOONGKWET YIVE, international consultant, and Allan VILLANUEVA, nationalconsultant.I.2 Scope and objectives of the evaluation13.The TE covered the whole duration of the project from its starting date inMarch 2013 to the completion date in June 2016. It was conducted inaccordance with the UNIDO Evaluation Policy 3 and the UNIDO Guidelines forthe Technical Cooperation Programme and Project Cycle. 4 In particular, itfollowed the GEF Guidelines for GEF Agencies in Conducting TerminalEvaluations 5 and the GEF Monitoring and Evaluation Policy, 6 and assessedthe project with emphasis on those components for which GEF funds wererequired.14.The TE should provide an analysis of the attainment of the projectobjective(s) and the corresponding technical components or outputs. Throughits assessments, the exercise should enable the government, the national GEFOperational Focal Point (OFP), counterparts, GEF, UNIDO, and otherstakeholders and donors to verify prospects for development impact andpromoting sustainability, providing an analysis of the attainment of globalenvironmental objectives, project objectives, delivery and completion of projectoutputs/activities, and outcomes/impacts based on indicators, andmanagement of risks.15.To enhance project relevance, effectiveness, efficiency, and sustainabilityby proposing a set of recommendations with a view to ongoing and futureactivities, the TE will additionally make recommendations for UNIDO and theGEF that may help for improving the selection, and for enhancing the design andimplementation of similar future projects and activities in the country and on aglobal scale upon project .org/files/documents/TE guidelines7-31.project on/evaluation-policy.html3UNIDO (2015). Director General’s Bulletin: Evaluation Policy (UNIDO/DGB/(M).98/Rev.1).4UNIDO (2006). Director-General’s Administrative Instruction No. 17/Rev.1: Guidelines for the TechnicalCooperation Programme and Project Cycle (DGAI.17/Rev.1, 24 August 2006).5GEF (2008). Guidelines for GEF Agencies in Conducting Terminal Evaluations (Evaluation Office,Evaluation Document No. 3, 2008).6GEF (2010). The GEF Monitoring and Evaluation Policy (Evaluation Office, November 2010).2

16.The TE will finally draw lessons of wider applicability from experiencegained in this pilot project on ASGM for replication across the Philippines and incountries in other regions.I.3 Information sources and availability of information17.The availability of information for evaluation purposes was satisfactory.Soft copies of documents produced during project implementation weresubmitted to the evaluation upon request. These documents include: theproject document, progress reports of national counterparts, ProjectImplementation Review (PIR) reports, consultants’ reports, reports ofcontracted activities, minutes of technical working group (TWG) meetings,reports of workshops (e.g., inception, awareness raising, training, etc.), andreports of other activities such as assessment of miners’ health undertaken byDOH. These documents were submitted by the UNIDO Project Manager (PM),and by national stakeholders (DENR, DOH, and Ban Toxics) during the countrymission undertaken on 26 – 30 September 2016. A list of documents submittedto the evaluation is given in annex 2.I.4 Methodological remarks, limitations encountered, and validity of thefindings18.Gathering of missing information as well as verification of informationthrough interviews were done during the country mission which wasundertaken on 26 – 30 September 2016. The major stakeholders of the projectwere interviewed at different locations. The National Project Coordinator(NPC), the Department of Environment and Natural Resources Head Quarters(DENR HQ), DOH HQ, Ban Toxics (BT), and UNIDO country office wereinterviewed in Manila, while the regional offices of DENR, DOH, and Mines andGeosciences Bureau (MGB) were interviewed in Davao City, located about1,000 kilometers south of Manila in Mindanao island. The representatives oflocal government unit (LGU), medical health workers involved in the project,and project beneficiaries (miners’ communities, women, and youth groups)were interviewed at Barangay Mt. Diwata (aka “Diwalwal”), Municipality ofMonkayo, which is located 159 kilometers northeast of Davao City.19.During the mission, it was not possible to interview the head of theDepartment of Health (DOH) in Manila and the Ban Toxics National ProjectManager (NPM), who were both on travel. The latter was interviewed via Skypeon her return from her mission. On the other hand, the head of the DOH, whenback in office, informed the evaluation that she was not aware of the projectsince she was nominated at this position only recently. She neverthelessreferred the evaluation team to a DOH officer who was directly involved in theproject, who was eventually interviewed through a questionnaire that wasemailed to him. He was also contacted by telephone for further information andclarification. The UNIDO project manager (PM) was interviewed in Vienna on 3October 2016. A list of persons interviewed is given in Annex 3.

20.The preliminary findings based on the field mission in the Philippinesand interviews in Vienna were presented to national stakeholders and UNIDOcountry office on 30 September 2016 in Manila, and to UNIDO HQ on 4October 2016 in Vienna, respectively. The feedback and comments receivedduring these presentations have been considered in this report.II. Country and Project BackgroundII.1 Project background21.Artisanal and small-scale gold mining (ASGM) is one of the mostsignificant sources of mercury release into the environment in the developingworld, and accounts for about 15% of the world’s annual gold production.Mercury is often used in ASGM to help separate gold from sediments usingrudimentary processing methods. Workers combine mercury with gold-ladensilt to form an amalgam, which is heated, often in or near homes, to evaporatethe mercury and leave gold.22.In the Philippines, ASGM occurs in more than 40 provinces, andprovides important subsistence-level income for about 300,000 miners andtheir families. For the past five years, ASGM activities have been producing atleast 80% of the Philippines’ yearly gold supply. With that comes the annualrelease of an estimated 70 to 140 metric tonnes of mercury, which isapproximately 3.6-7.2% of the current estimated total anthropogenic mercuryemissions worldwide at 1,921 metric tonnes (Mercury Watch Database). AUnited Nations (UN) study in 2006 found significant mercury contamination atlevels up to 50 times World Health Organization (WHO) standards amongsurveyed gold miners. Gold rushes are occurring in various places, includingDiwalwal, where UNIDO implemented a project aiming at assessing the healthimpact of mercury on mining and downstream population in 1998-2000. Theproject underlined the need for more awareness raising as it appeared thatboth study groups were heavily affected.23.In 2011, Ban Toxics, along with Dialogos (a Danish non-governmentalorganization [NGO]), the International Committee of Environmental,Occupational and Public Health, the Geological Survey of Denmark andGreenland, and the University of Copenhagen, initiated a 3-year projectentitled: Reducing Mercury Use in Artisanal and Small-scale Gold Mining in thePhilippines (2011-2014). The project under evaluation is a follow-up of theefforts initiated by Ban Toxics to reduce mercury emissions.II.2 Project SummaryOverall Objective24.The overall objective of the project was to improve the health andenvironment of artisanal gold mining communities in the Philippines byreducing mercury emissions. In particular, the project aimed to introducemercury-free technology in 2 small-scale mining areas, and to supplement this

effort by providing health training to rural health care workers in the properdiagnosis of mercury poisoning. Specifically, it sought to assist the governmentto develop, implement, and facilitate the demonstration and replication ofmercury reduction/elimination projects, enable local and national stakeholdersto receive health, techniques and technology trainings, and promote policyreforms based on the lessons learned to reduce mercury use, emission, andexposure in ASGM activities.25.To achieve these objectives, the following components were developed:Component 1: Strengthen national capacity to effectively manage mercury byestablishing a formal national institution and training of key stakeholders.Component 2: Develop and deliver health education, techniques andtechnology training programs, including early recognition and identification ofmercury poisoning at the community level, to reduce mercury in ASGM.26.The expected results were the following:a) A national ASGM institution was to be established to facilitate theprocess of mercury reduction/elimination. This was to be built on themomentum of the Philippine Strategic Approach to Danishgovernment/Dialogos/Ban Toxics non-mercury technique project,and the Ban Toxics/US Department of State mercury storage project.b) Health education, technique and technology programs, and mercurypoisoning surveillance program that could be later replicatednationwide was to be developed, and capacity increased throughdelivery of training programs.c) National and local stakeholders in the Philippines were to besensitized and able to replicate technical successes at other ASGMsites aiming to reduce overall mercury use, emissions, and exposurein country; and, important lessons learned were to contribute andpromote sound national management policies on mercury in thefuture.Project duration and costs27.Table 1 gives all relevant information on the project, namely: projectcosts and co-financing, donors, duration, implementing and executingagencies.

Table 1: Information on ProjectProject title:UNIDO Project number:GEFSEC project ID:Project site:Implementing agency:Government coordinating agency:Planned project duration:Start dateActual start datePlanned implementation endActual implementation endGEF grant :ProjectProject:costs ( )PPG:Support costs:Sub-totalCo-funding:UNIDO (cash):Government (cash & in-kind):Sub-totalTotalImprove the health and environment ofartisanal gold mining communities in thePhilippines by reducing mercury emission1200165216PhilippinesUNIDORepublic of Philippines, Department ofEnvironment and Natural Resources (DENR),Department of Health (DOH), and Ban Toxics(BT)24 monthsDecember 2012March 2013December 2014June 2016550,000550,000 (excluding support costs)50,0001,031,0701,081,0701,631,070 (excluding support costs and PPG)II.3 Project implementation arrangement and implementation modalities28.For this project, UNIDO was the GEF agency and Ban Toxics was themain executing partner organization. In addition, DENR was responsible for theorganization and management of the national technical working groupmeetings, inception workshop, while DOH was involved in the health educationtraining workshops.29.UNIDO was responsible for overall project implementation, monitoring,and reporting. A Project Manager (PM) was nominated within UNIDO HeadQuarters (UNIDO HQ) to manage the project. He was supervised by a seniorprofessional staff of the Department of Environment and was assisted in histasks by support staff. Due to staff movement, the UNIDO PM changedtowards the end of the project in August 2015. The new PM stated that thehandover was done adequately, and the transition phase was not difficult as hewas already managing similar projects. UNIDO coordinated closely with itsoffice in Manila, Philippines, for project monitoring and supervision to ensurequality implementation by the main executing partner, Ban Toxics. UNIDOmade these services available as part of its in-kind contribution to the project.

30.UNIDO, as the implementer, and Ban Toxics, as the main executor ofthe project, were jointly responsible to deliver project results, coordination ofstakeholders, and management of pilot projects. A National Project Managerwas nominated within Ban Toxics to manage project activities. In coordinationwith DENR and DOH, UNIDO and Ban Toxics provided technical expertise andguidance. All project components were implemented in collaboration with avariety of local and national stakeholders, including the local governments,local communities, miners’ communities, community health workers, localassociations and groups, and others.Stakeholders / Institutions involved31.The implementation of the project required the involvement of a numberof national stakeholders. During the project development, discussions wereundertaken mainly with representatives of DENR, which was the coordinatingagency, and a National Project Coordinator (NPC) was nominated from thisDepartment. The Department of Health was one of the executing partners ofthe project. It was involved in the health education training workshops and themonitoring of the health of miners at the pilot site. The University of Philippines– Philippines General Hospital – National Poison Centre (UP-PGH-NPC) wasalso involved in these monitoring activities.32.Ban Toxics, an NGO registered in 2008, was the main executor of theproject. It was responsible for project management and delivery of results atthe pilot sites. In particular, it was in direct contact with the LGUs, localcommunities, and the miners.33.The National Commission on Indigenous Peoples (NCIP), an agencyresponsible for protecting the rights of indigenous peoples in the Philippines,was also involved in the project, and it was a member of the technical workinggroup. The Mines and Geosciences Bureau (MGB), a primary governmentagency under the DENR, assisted in the implementation of activities at pilotsites. It also participated in regional workshops during which it provided usefulinformation.34.The LGUs of Compostela Valley Province and of Barangay Mt. Diwata(Diwalwal) were very much involved in the project and facilitated theimplementation of activities at the pilot sites. The miners from Benguet miningcommunity were recruited to act as master trainers at the project sites.

II.4 Positioning of UNIDO35.UNIDO has been working in the ASGM for more than 20 years and in 17countries. The Global Mercury Project (GMP), a UNIDO initiative, waslaunched in 2002 with financial support from the Global Environment Facility(GEF) under the international water focal area, and was co-financed by partnercountries and civil society. Several countries participated in this pilot program,including Brazil, Indonesia, Lao People’s Democratic Republic, Sudan,Tanzania, and Zimbabwe. The GMP worked with governments, NGOs,industry, and community stakeholders to remove existing barriers that preventthe introduction of cleaner artisanal gold mining extractive technologies.

miners to shift to mercury-free method, thus reducing the use of mercury at the project sites and exposure to mercury lessening (as observed in miners during the health assessments). It is recommended (for BT and DENR) that these positive outcomes be summarized and disseminated to other ASGM communities in the Philippines. ii.