WIXLIB

AcknowledgmentsFunding for development of this guide has come from several sources, including the FAO-FinlandProgramme, the UN-REDD Programme, PROFOR, and the European Union-FAO Forest LawEnforcement, Governance, and Trade Programme (EU FAO FLEGT Programme).The impetus for the guide came from FAO (Eva Muller, Ewald Rametsteiner, Emelyne Cheney, andRobert Simpson) and PROFOR (Nalin Kishor).A 2012 workshop at FAO’s headquarters in Rome confirmed the need for the guide and producedits rough outline. The workshop participants included Crystal Davis, Alice Thuault, Cecile Njebet,Giorgio Budi Indrarto, Sam Lawson, David Young , Samuel Nguiffo, Phil Franks, Saraswati Rodriguez,Tina Sølvberg, Abdul Wahib Situmorang, Joachim Nahem, Evgeny Kuzmichev, Vladislava Nemova,Marina Smetanina, Stephano Kingazi, Nguyen Phu Hung, Doan Diem, Beatrice Lukama , OrleansMfune, Sam Nketiah, Chris Beeko, Jo Van Brusselen, An Bollen, Herman Savenije, Tapani Oksanen,Tek Narayan Maraseni, Laura Secco, Eva Muller, Bob Simpson, Marjo Maidell, Ewald Rametsteiner,Emelyne Cheney, and Nalin Kishor. Kenneth Rosenbaum facilitated.The 2012 workshop prompted creation of a core group of experts to oversee creation of the guide.This group included Robert Simpson, Ewald Rametsteiner, Nalin Kishor, Jo van Brusselen, Tina Sølvberg,Emelyne Cheney, Crystal Davis, Saskia Ozinga, Filippo del Gatto, Boris Romaniuk, Steve Nsita, NguyenQuang Tan, and Ragna John. Kenneth Rosenbaum and Guido Broekhoven facilitated the groupmeetings. The European Forest Institute (EFI) hosted the first meeting of the group in November 2012;the World Resources Institute (WRI) hosted the second in June 2013. Also participating in parts of themeetings were Rudi Kohnert, Tuukka Castrén, Dan Miller, Lauren Goers Williams, Florence Daviet, Florede Préneuf, and Phil Cowling.As some members of the core group left to take on new assignments, Melissa Othman and Free deKoning filled in to give feedback on drafts. Additional feedback came from outside reviewers NinaRinnerberger, Judith Walcott, and Lera Miles.Many people provided logistical and operational support, including Tania Abdirizzak, Michela Mancurti,Anne Ricchiuti-Romanelli, Elena Bernardini, Janice Saich, Helene Godeaux, Sujatha Venkat Ganeshan,and Veronica Jarrin.People offering general advice, information, and assistance included Benjamin Cashore, Tini Gumartini,Daniel Meireles Tristao, Rosalind Reeve, James Mayers, Francesca Felicani Robles, and Stephanie Ratte.Besides the core group of experts, people who provided information about case studies includedLouise Riley, Christine Holding, Nuru Chamuya, Søren Dalsgaard, Abdul Wahib Situmorang, MireyaVillacís, and Sigrid Vásconez.Phil Cowling and Kenneth Rosenbaum wrote the body of the guide with the valuable assistance ofKristin DeValue. Kristin DeValue led the assembly of the case study, tools, and glossary annexes.The guide was edited by Daria Steigman and designed by Studio Grafik.James Cantrell provided editorial and publication support.10ASSESSING FOREST GOVERNANCE: A PRACTICAL GUIDE TO DATA COLLECTION, ANALYSIS, AND USE

ACRONYMS AND APROFORREDDREDD SCAPESSDISPSSSWOTUKUNUNDPUNFCCCUSUSAIDVPAWRICivil Society-Independent Forest MonitorsDepartment for International Development (UK)European Forest InstituteEuropean Neighbourhood and Partnership InstrumentEuropean UnionFood and Agriculture OrganizationFundación para el Avance de las Reformas y las Oportunidades(Foundation for the Advance of Reforms and Opportunities)Forest Law Enforcement and GovernanceForest Law Enforcement, Governance, and TradeGesellschaft für Internationale ZusammenarbeitGovernance of Forests InitiativeInternational Association for Public ParticipationInstitutional and Context AnalysisInformation and Communication TechnologyInternational Fund for Agricultural DevelopmentInternational Tropical Timber OrganizationInternational Union for Conservation of NatureMemorandum of UnderstandingNational Forest Resources Monitoring and AssessmentNational Forest ProgrammeNongovernmental OrganizationOverseas Development InstitutePolitical Economy AnalysisParticipatory Governance AssessmentProgram on ForestsReducing Emissions from Deforestation and Forest DegradationREDD plus Fostering Conservation, Sustainable Management ofForests and Enhancement of Forest Carbon StocksSustainable Conservation Approaches in Priority EcosystemsSustainable Development InstituteStatistical Package for the Social SciencesStrengths, Weaknesses, Opportunities, and ThreatsUnited KingdomUnited NationsUN Development ProgrammeUN Framework Convention on Climate ChangeUnited StatesU.S. Agency for International DevelopmentVoluntary Partnership AgreementWorld Resources InstituteAcronyms and Abbreviations11

12ASSESSING FOREST GOVERNANCE: A PRACTICAL GUIDE TO DATA COLLECTION, ANALYSIS, AND USE

INTRODUCTIONThis is a guide to measuring or assessing forest governance. Forest governance comprises allthe social and economic systems that affect howpeople interact with forests, including bureaucracies, laws, policies, traditional norms and culture,patterns of land tenure, and markets.1People assess forest governance in many ways. TheUganda case study gathered most of its data in atwo-day stakeholder workshop. The Tanzania casesurveyed thousands of households. The Indonesiacase study involved many weeks of interviews, library research, consultations, and surveys.People assess forest governance for many reasons. Assessments tied to reducing emissionsfrom deforestation and forest degradation, plusfostering conservation, sustainable managementof forests, and enhancement of forest carbonstocks (REDD ) have aimed to fulfill internationalobligations, diagnose problems, and establish abaseline for future monitoring. Assessments under the World Bank Forest Investment Programhave helped set the agendas for donor funding.Assessments by nongovernmental organizations(NGOs) have held officials more accountableand have been the basis for advocacy for reformor better implementation of forest laws.In every assessment, people take on variedroles. Some plan the assessment. Some manageits implementation. Some carry out the data collection, analyze the data or communicate the results. Some participate as information resources,constructive critics, or advisors.People assess forest governance on many scales.The assessment in Liberia highlighted in AnnexI focused on a few concessions; the cases fromTanzania, Indonesia and Uganda covered wholenations; and the one on Ecuador was part of alarger study intended to compare performancein several countries. The Ecuador assessmentlooked specifically at transparency, while thethree national assessments attempted to coverforest governance more broadly.1. For a good, detailed explanation of what makes up forest governance,see the framework presented in PROFOR & FAO (2011). That publicationbreaks forest governance down into pillars, components, andsubcomponents. Davis et al. (2013), World Bank (2009), Situmorang et al.(2013), IIED (2005b), and USAID (2013) offer alternative frameworks.IntroductionThis guide aims to be useful to everyone involvedin a forest governance assessment. Some readerswill want to go through the whole guide to havea full picture of the process. Others will find whatthey want in particular chapters on planning, datacollection, analysis, and use of data.Readers will find that this guide bases its approach on a few key foundations. These are thata good assessment: Requires good planning. For that reason, thisguide goes into detail on planning.Is transparent and includes stakeholder involvement and outside review. For that reason,this guide stresses participatory approaches.Uses data collection methods that aretechnically sound. For that reason, the datacollection chapters of this guide introducetechnical topics.Does not stop at data collection and analysis; rather, it disseminates results in waysthat encourage use of the assessment.For that reason, this guide talks about13

dissemination strategies and ways to buildupon assessments.Is open to learning. It evaluates itself and seeksto improve. For that reason, this guide talksabout piloting, adaptive changes in planning,and self-evaluation at the end of the process.LANGUAGE CHECKThe guide presents approaches to assessmentconsistent with these premises. If you are conducting a large, detailed, and unique assessment, you will be interested in following mostof the steps in the guide. If your assessment issmaller and less complex, you may decide thatyou do not need to follow every step. For example, you may not need to write a work planor a data collection manual if the assessmentis built around a single day’s workshop, or todesign a new method of data collection if theproject wants to use a method designed for aprevious monitoring process.This guide does not set standards. There is nosingle best way to conduct an assessment. Thisguide builds upon what others have done in thisrapidly developing field and points to some useful practices and resources.The potential number of steps can seem daunting,but doing a governance assessment is not necessarily harder than doing other kinds of inventoriesor monitoring. The case studies in Annex I showhow assessments of different sizes and complexity have approached the task and succeeded.To decide what parts of this guide will be usefulto you, read the overview of chapters presentedbelow and consult Table 1. You may discover thatyou want to use the whole guide, or you mayend up using the guide selectively to improveyour planning and to learn new ways to collectdata or increase the impact of your findings.BOX 1: SOME KEY TERMSDifferent sources use terms like “assessment” and “evaluation” differently. This guide gives these termsbroad definitions:Assessment means “appraisal based on careful analytical evaluation” (PROFOR & FAO 2011, p.31).Data collection means the systematic gathering of information.Evaluation means study or measurement, often with an aim to compare the current situation with a pastsituation or a desired goal.Measuring means finding the size, amount, extent, status, or degree of something. As used in this guide,it can apply to both quantitative and qualitative data collection.Monitoring means “systematic tracking or scrutiny for the purpose of collecting specified data orinformation” (PROFOR & FAO 2011, p.31).Refer to Annex VI for definitions of other useful terms.14ASSESSING FOREST GOVERNANCE: A PRACTICAL GUIDE TO DATA COLLECTION, ANALYSIS, AND USE

OVERVIEW OF THE GUIDEThis guide comprises three sections. The first section is about planning, the second is about datacollection and analysis, and the third is about using your assessment.Section I: Planning Your AssessmentChapter 1 of the guide deals with setting objectives. It includes: Identifying why you are doing an assessment.Your answer to “why” forms the foundationof your remaining work.Assessing the context in which you are working. Context can affect when you decide to doan assessment and what approach you take.Setting out your objectives in considerationof what is practical to achieve.This chapter will be most useful to people initiating, funding, or overseeing assessments.Chapter 2 of the guide gets into the details ofplanning. The steps in this chapter lead up towriting an assessment work plan. Not every assessment needs a formal written work plan, butevery assessment needs to make some basicplanning decisions. These include: Setting the technical scope (the specific elements of forest governance you are interested in), geographical scope (e.g., whetherlocal, national, or international), and socialscope (the specific social groups or institutions you want to gain information on).Choosing the general methods you willuse to gather data. For example, will youuse household surveys? Expert opinion?Stakeholder workshops? Document reviews?Overview of the GuideDeciding who will conduct the assessment.Who will fund it, who will do the field work,and who will provide institutional support?Making a budget.Drawing a timeline or setting a time schedule.This chapter will be most useful to people carrying out the high-level planning, but it will also beof interest to funders, stakeholders, and otherswho may have a say in the planning decisions.Chapter 3 deals with planning for data collection. This planning requires some technicalknowledge of data collection methods, andthe chapter offers an introduction to these withpointers to further information. Some assessments will come to this point with definite ideasof how to collect data. For example, they may bepart of an established monitoring process andbe bound to use more or less the same methods as the previous round of monitoring. Otherassessments will be creating new methods andwill need to carefully go through all the steps inthe chapter. These steps are: Deciding what aspects of governance tomeasure. This step builds on the scopesetting in Chapter 2, but takes it to a newlevel of detail. The step may entail creationof indicator sets.15

Identifying data sources. This means understanding where to find relevant informationand who to involve in the search.Selecting data collection methods. Again, thisbuilds on decisions made in earlier planning.However, the general plans for methodsneed to become more concrete guidance fordata collection.Developing data collection tools. These mayinclude interview protocols, surveys, workshop agendas, and so forth. Writing a data collection plan. Complex projects involving many data collectors will alsowant to write a field manual.This chapter will be of greatest interest to themanagers responsible for implementing the assessment as the task of designing data collectionusually falls to them.Section II: Implementing Your AssessmentChapter 4 covers the basics of collecting datafor the assessment. The details of the steps willvary with the methods that the assessment uses.You will need more effort and staff to gather datathrough household surveys than you will by collecting the opinions of a few experts.Chapter 5 covers interpretation and analysis: The basic steps covered in this chapter are common to most data gathering efforts: Recruiting and training staff.Collecting the data.Assuring the quality of the data collected.The information in this chapter will be of use tohigh-level managers, data collection managers,data collection staff, and people outside the assessment who want to understand and critiquedata collection practices.16 Processing the data, which may include entering it into digital form, summarizing it, orproducing visual representations of it.Analyzing the data, which means interpretingthe data in terms of the local context. Thismay include scoring indicators, identifyingpatterns in the data and shedding light ontheir causes and effects, or explaining thedata in terms of social or economic theory.Making recommendations, which typicallytake the form of suggested priorities or actions.As with Chapter 4, this chapter will be of interestto people doing the work of analysis as well asto people who want to understand and critiquethe analysis of others.ASSESSING FOREST GOVERNANCE: A PRACTICAL GUIDE TO DATA COLLECTION, ANALYSIS, AND USE

Section III: Using Your AssessmentChapter 6 covers dissemination of results. Toomany assessments simply publish a reportthat gets read a few times and then filed away.Chapter 6 discusses:Chapter 7 discusses learning and improvement to make the first and future assessmentsbetter. It covers: How to develop a dissemination strategythat will bring your findings and recommendations to the attention of decision makersand stakeholders who can put the information to good use.How to implement the strategy.How to institutionalize the assessment process, or at least make it more likely that thenext assessment will have your records andmethods available to build upon.This chapter should be of interest to people whoare interested in seeing the investment of timeand energy in assessments lead to real change:assessment initiators and funders, managers,data collectors, analysts, and stakeholders. Ongoing self-evaluation of the assessmentprocess during implementation.Evaluation of the process after the assessment is complete.Capturing and sharing lessons learned.Assessment is an evolving practice, and wecan all learn from each other’s experiences.Finding ways to keep gathering feedback after the assessment is over and the staff havemoved on to other projects. Some lessonswill emerge only after events play out overmonths or years.This chapter should be of interest to all whohope to improve the quality of assessments.PostscriptThe postscript briefly notes the rapid evolution offorest governance assessment and encouragespractitioners to contribute to the growth of the field.Overview of the Guide17

AnnexesThis guide includes several annexes that shouldprovide as much practical guidance as the text itself.Each case is outlined using the steps presentedin the main text of the guide.Annex I includes five case studies:Annex II presents a set of references and toolslinked to the chapters of the guide. For example,if you are interested in learning more aboutpolitical or economic assessments (discussedin Chapter 1), creation of survey instruments(Chapter 3), or data visualization (Chapter 5),you will find links to resources on those topicsin this annex. A broad, indicator-based, countrywide assessment in Indonesia that uses severalmethods to gather data (with regional andlocal components).A national survey-based assessment inTanzania that was part of a larger effort to collect biophysical and social data about forests.A national assessment in Ecuador focusingon evaluating transparency and designed tobe part of an international survey of severaldeveloping countries.An assessment in Liberia of the governanceof seven forest concessions.An assessment in Uganda using a nationalstakeholder workshop for rapid scoring of

4 ASSESSING FOREST GOVERNANCE: A PRACTICAL GUIDE TO DATA COLLECTION, ANALYSIS, AND USE . FINALIZE YOUR WORK PLAN AND DEVELOP A DATA COLLECTION MANUAL 85 POINTS ON PROCESS: VETTING THE METHODS 86 . plantation, or agro-forestry project will succeed if the resources are poorly governed. The concept of "forest governance" is often difficult .