Transcription
Project/programmemonitoring and evaluation(M&E) guidewww.ifrc.orgSaving lives, changing minds.
Strategy 2020 voices the collective determination ofthe International Federation of Red Cross and RedCrescent Societies (IFRC) to move forward in tacklingthe major challenges that confront humanity in thenext decade. Informed by the needs and vulnerabilitiesof the diverse communities with whom we work, aswell as the basic rights and freedoms to which all areentitled, this strategy seeks to benefit all who lookto Red Cross Red Crescent to help to build a morehumane, dignified and peaceful world.Over the next ten years, the collective focus of theIFRC will be on achieving the following strategic aims:1.2.3.Save lives, protect livelihoods, and strengthenrecovery from disasters and crisesEnable healthy and safe livingPromote social inclusion and a cultureof non-violence and peaceAcknowledgementsThis guide was developed by the Planning and Evaluation Department(PED) of the IFRC Secretariat. It would not have been possible withoutthe invaluable review and feedback from National Societies. Inparticular, we want to express our thanks to the British Red Cross, theDanish Red Cross, the Norwegian Red Cross, the Swedish Red Cross,the Finnish Red Cross, the American Red Cross, the Australian RedCross, and the Canadian Red Cross. Also, special thanks to Julie Smithfor her creative cartoons and M&E sense of humour. International Federation of Red Crossand Red Crescent Societies, Geneva, 2011Copies of all or part of this guide may be made fornoncommercial use, providing the source is acknowledgedThe IFRC would appreciate receiving details of its use.Requests for commercial reproduction should be directed tothe IFRC at secretariat@ifrc.orgThe designations and maps used do not imply the expressionof any opinion on the part of the International Federation orNational Societies concerning the legal status of a territory orof its authorities.All photos used in this guide are copyright of the IFRC unlessotherwise indicated. Cover photo, from left to right, clockwise:Benoit Matsha-Carpentier/IFRC, Arzu Ozsoy/IFRC, AlexWynter/IFRC.P.O. Box 372CH-1211 Geneva 19SwitzerlandTelephone: 41 22 730 4222Telefax: 41 22 733 0395E-mail: secretariat@ifrc.orgWeb site: www.ifrc.orgProject/programme monitoring and evaluation (M&E) guide1000400 E 3,000 08/2011
Table of ContentsAcknowledgementsAbbreviations and Acronymsinside cover4Introduction5PART 1: M&E concepts and based management (RBM)M&E and the project/programme cycleWhat is monitoring?What is evaluation?Baseline and endline studiesComparing monitoring, evaluation, reviews and auditsM&E standards and ethicsAttention to gender and vulnerable groupsMinimize bias and errorPART 2: Six key steps for project/programme M&E2.1 STEP 1 – Identify the purpose and scope of the M&E system2.1.12.1.22.1.32.1.4Review the project/programme’s operational design (logframe)Identify key stakeholder informational needs and expectationsIdentify any M&E requirementsScope of major M&E events and functions2.2 STEP 2 – Plan for data collection and 2.2.92.2.102.2.112.2.12Develop an M&E plan tableAssess the availability of secondary dataDetermine the balance of quantitative and qualitative dataTriangulate data collection sources and methodsDetermine sampling requirementsPrepare for any surveysPrepare specific data collection methods/toolsEstablish stakeholder complaints and feedback mechanismsEstablish project/programme staff/volunteers review mechanismsPlan for data managementUse an indicator tracking table (ITT)Use a risk log 3840424345472.3 STEP 3 – Plan for data analysis482.3.1 Develop a data analysis plan2.3.2 Follow the key data analysis stages49502.4 STEP 4 – Plan for information reporting and utilization2.4.1 Anticipate and plan for reporting2.4.2 Plan for information utilization2.5 STEP 5 – Plan for M&E human resources and capacity building575866691
International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guide2.5.12.5.22.5.32.5.42.5.52.5.6Assess the project/programme’s human resources capacity for M&EDetermine the extent of local participationDetermine the extent of outside expertiseDefine the roles and responsibilities for M&EPlan to manage project/programme team’s M&E activitiesIdentify M&E capacity-building requirements and opportunities2.6 STEP 6 – Prepare the M&E budget2.6.12.6.22.6.32.6.4Itemize M&E budget needsIncorporate M&E costs in the project/programme budgetReview any donor budget requirements and contributionsPlan for cost contingency6969727273737474747575ANNEXES77Annex 1:Annex 2:Annex 3:Annex 4:Annex 5:Annex 6:Annex 7:Annex 8:Annex 9:Annex 10:Annex 11:Annex 12:Annex 13:Annex 14:Annex 15:Annex 16:Annex 17:Annex 18:Annex 19:Annex 20:Annex 21:Annex 13114115122123127Glossary of key terms for M&EM&E resourcesFactors affecting the quality of M&E informationChecklist for the six key M&E stepsIFRC’s logframe – definition of termsExample M&E stakeholder assessment tableExample M&E activity planning tableM&E plan table template and instructionsM&E plan exampleM&E plan purpose and complianceM&E plan instructionsClosed-ended questions examplesKey data collection methods and toolsProject/programme feedback form templateComplaints logStaff/volunteer performance management templateIndividual time resourcing sheetProject/programme team time resourcing sheetIndicator tracking table (ITT) examples and instructionsExample risk logReporting scheduleIFRC’s project/programme management report – template and instructionsExample tables (logs) for action planning and management responseExample M&E job descriptionM&E training scheduleList of tables, boxes and diagrams2Table 1: Common types of monitoring12Table 2: Summary of major evaluation types15Table 3: The IFRC’s framework for evaluation – criteria and standards17Table 4: Comparing key features of monitoring/review, evaluation and audit20Table 5: Example of indicator tracking table – for one quarter only46Table 6: Comparing data analysis terms: findings, conclusions, recommendations and actions56
International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guideBox 1: Principle Nine of the Conduct for International Red Cross and Red CrescentMovement and NGOs in Disaster Relief6Box 2: Monitoring best practices13Box 3: The challenge of measuring impact18Box 4: Principle Five of the Code of Conduct for International Red Cross and Red CrescentMovement and NGOs in Disaster Relief21Box 5: M&E in emergency settings27Box 6: Types of industry (standard) indicators28Box 7: Examples of IFRC’s key stakeholders and informational needs29Box 8: Specific evaluation requirements for the IFRC’s secretariat-funded projects/programmes30Box 9: Examples of key M&E activities31Box 10: Is an M&E plan worth all the time and effort?33Box 11: Comparing quantitative versus qualitative data35Box 12: Minimizing data collection costs40Box 13: The IFRC’s guide for stakeholder feedback42Box 14: Formats can reinforce critical analysis and use44Box 15: The importance of target setting47Box 16: Benefits of involving multiple stakeholders in data analysis50Box 17: Data analysis questions to help describe the data52Box 18: Using traffic lights to highlight data55Box 19: Criteria of good reporting58Box 20: Internal versus external reporting60Box 21: Example reporting formats62Box 22: Report writing tips63Box 23: IFRC’s project/programme management report outline (refer to Annex 19 for full template) 64Box 24: Reporting roadblocks and solutions65Box 25: Key categories of information use66Box 26: Key mediums of information dissemination66Box 27: Principle Seven of the Conduct for International Red Cross and Red CrescentMovement and NGOs in Disaster Relief70Box 28: Considering participatory M&E71Box 29: Adhering to human resources codes and standards – People in Aid73Box 30: How much money should be allocated for M&E?75Diagram 1: Key M&E activities in the project/programme cycle10Diagram 2: Monitoring questions and the logframe11Diagram 3: Evaluation questions and the logframe14Diagram 4: An example of information flows in project/programme reporting61Diagram 5: The participatory continuum703
International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guideAbbreviations and AcronymsDACFWRSHNSHRDevelopment Assistance CommitteeFederation-Wide Reporting SystemHost National Societyhuman resourcesICRCInternational Committee of the Red CrossIFRCInternational Federation of Red Cross andRed Crescent SocietiesITinformation technologyITTindicator tracking tableM&Emonitoring and evaluationMoUMemorandum of UnderstandingNGOnon-governmental organizationOECDOrganization for Economic Co-operation DevelopmentONSOperational National SocietyPEDplanning and evaluation departmentPMERplanning, monitoring, evaluation and reportingPNSParticipating National SocietyRBMresults-based managementRTEreal-time evaluationSMARTspecific, measurable, achievable, relevant,time-boundSWOTToRVCA4strengths, weaknesses, opportunities and threatsterms of referencevulnerability and capacity assessment
International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guideIntroductionWhat is this guide?The purpose of this guide is to promote a common understanding and reliable practiceof monitoring and evaluation (M&E) for IFRC project/programmes. It is meant to bea desktop reference that supplements the more concise and field-friendly IFRCPMER Pocket Guide. Therefore, this guide is not intended to be read from cover tocover; the reader can refer to specific topics for more detail when needed.This guide does not provide detailed guidance on conducting evaluations; this is provided in separate IFRC resources.1 Instead, emphasis is placed on establishingand implementing a project/programme monitoring and related reportingsystem. However, as evaluation is integrally linked to monitoring, an overviewof evaluation is included for planning evaluation events within the overall M&Esystem.Who is the intended audience?This guide is intended for people managing projects/programmes in National RedCross and Red Crescent Societies and the secretariat. However, it has been designed to be understood by multiple other users as well, including IFRC staffand volunteers, donors and partners. Although it has been designed for use atthe country level, the basic principles can be applied to projects/programmesat other levels.1A guide for managingevaluations will be availablefrom the IFRC’s planning andeducation department (PED).5
International Federation of Red Cross and Red Crescent SocietiesProject/programme monitoring and evaluation guideWhy is M&E important?A well-functioning M&E system is a critical part of good project/programmemanagement and accountability. Timely and reliable M&E provides information to:ÔÔSupport project/programme implementation with accurate, evidencebased reporting that informs management and decision-making to guide andimprove project/programme performance.ÔÔ Contribute to organizational learning and knowledge sharing by reflectingupon and sharing experiences and lessons so that we can gain the full benefitfrom what we do and how we do it.ÔÔ Uphold accountability and compliance by demonstrating whether or notour work has been carried out as agreed and in compliance with establishedstandards (e.g. the Red Cross and Red Crescent Fundamental Principles andCode of Conduct – see Box 1) and with any other donor requirements.2ÔÔ Provide opportunities for stakeholder feedback, especially beneficiaries, toprovide input into and perceptions of our work, modelling openness to criticism, and willingness to learn from experiences and to adapt to changing needs.2IFRC adopts the OECD/DACdefinition of accountability,(see the Glossary of Key Termsin Annex 1). In addition to itsown Fundamental Principlesand Code of Conduct, it alsoendorses other internationallyrecognized standards, suchas the Sphere Standards toenhance accountability ofhumanitarian assistance topeople affected by disasters,and the Good Enough Guidefor impact measurement andaccountability in emergencies(both developed by a coalitionof leading internationalhumanitarian organizations andare listed in Annex 2,M&E Resources).3The use of M&E for resourcemobilization should not beperceived as a pure marketingtactic because assessments ofour performance and resultshelp demonstrate the returnswe get from the investment ofresources, lending credibilityto our achievements.Advice for the readerRefer to the additionalresources in Annex 2,which includes bothIFRC resources for PMERby project/programmeand focus area, aswell as other usefulresources from the international community.6ÔÔ Promote and celebrate our work by highlighting our accomplishments andachievements, building morale and contributing to resource mobilization.3Box 1: Principle Nine of the Conduct for International Red Cross andRed Crescent Movement and NGOs in Disaster ReliefWe hold ourselves accountable to both those we seek to assist and thosefrom whom we accept resources. We often act as an institutional link inthe partnership between those who wish to assist and those who need assistance during disasters. We therefore hold ourselves accountable to bothconstituencies. All our dealings with donors and beneficiaries shall reflectan attitude of openness and transparency. We recognize the need to reporton our activities, both from a financial perspective and the perspective ofeffectiveness. We recognize the obligation to ensure appropriate monitoringof aid distributions and to carry out regular assessments of the impact ofdisaster assistance. We will also seek to
Annex 19: IFRC’s project/programme management report – template and instructions 115 Annex 20: Example tables (logs) for action planning and management response 122 Annex 21: Example M&E job description 123 Annex 22: M&E training schedule 127 List of tables, boxes and diagrams Table 1: Common types of monitoring 12 Table 2: Summary of major evaluation types 15 Table 3: The IFRC’s .
