Transcription
Results Based ManagementConcepts and Methodology
I. INTRODUCTIONThe present note provides guidance to operatingunits1 on results-based management (RBM) inUNDP. The note offers an explanation of theconceptual and methodological building blocksas well as the instruments that represent UNDP’sapplication of the RBM philosophy. It is intendedto help establish organisation-wide standardswith regard to key aspects of resultsmethodology and terminology.Managing for results is not completely new toUNDP but what makes the current approachdifferent is the determination to make RBM thedriving force behind the organisation’sinstitutional culture and practice – and to developand apply a corporate methodology for thispurpose.II. WHAT IS RBM?The objective of RBM is to “provide a coherentframework for strategic planning andmanagement based on learning andaccountability in a decentralised environment.”2Introducing a results-based approach aims toimprove management effectiveness andaccountability by “defining realistic expectedresults, monitoring progress toward theachievement of expected results, integratinglessons learned into management decisions andreporting on performance”.31 The term “operating units” is used to cover all country offices, funds andheadquarters units that have individual budgets and reportingresponsibilities.2 Note on RBM, Operations Evaluation Department, World Bank, 1997.3 “Results-based Management in Canadian International DevelopmentAgency”, CIDA, January 1999.RBM in UNDP is based on four main pillars:the definition of strategic goals which providea focus for action;the specification of expected results whichcontribute to these goals and alignprogrammes, processes and resourcesbehind them;on-going monitoring and assessment ofperformance, integrating lessons learnt intofuture eedbacktoimproveperformance.III. MANAGING FOR RESULTSTypes of Development ResultsUNDP’s approach to results-based managementis centred on two types of development result:outputs and outcomes. In RBM, inputs and theactivities which transform them into outputsreflect the process of implementingprojects/programmes4 rather than desirableend results in themselves. From a resultsperspective, the implementation process issignificant only in terms of what it leads to – orwhat follows from the process of planning,managing and implementing.Outputs are the specific products and serviceswhich emerge from processing inputs throughprogramme or non-programme activities.Outputs, therefore, relate to the completion(rather than the conduct) of activities and are the4 The term “programme” is used in this document to denote any project,programme or other programmatic intervention irrespective of its beingformalized as a project document or a programme support document(PSD).Page 2
UNDP Results FrameworkTechnical Note: 2000type of result over which managers have a highdegree of influence.Outcomes are actual or intended changes indevelopment conditions that UNDP interventionsare seeking to support. They describe a changein development conditions between thecompletion of outputs and the achievement ofimpact (see Figure 1).F ig u re 1: W h at are res u lts : O u tp u tsIn p u tsO u tp u tsE xperts;equipm ent;fundsP eopletrained;studiescom pleted andO u tco m esJobscreated;incom esincreasedO u tco m esIm p ac tPage 3outputs and outcomes with a level of ambition tofit the country context and UNDP’s resources isgiven below.An individual UNDP output or even a clusterof outputs will not guarantee the achievementof a related outcome, since the contributionof wider group of partners is usuallyessential. Ideally, outcomes should be agreed toby the partners involved as necessary to have apositive influence on a development issue withina reasonable period of time. Seeking to influencedevelopment outcomes is, therefore, by its verynature dependent on partnerships. Since severalactors are involved, outcomes cannot necessarilybe attributed to any one party (such as UNDP).C onditionsim proved:health/longevityTo assist in distinguishing between outputs andoutcomes, three tests may be applied (see Box1).Box 1: Tests to Distinguish between Outcomes and OutputsOutcomes(1) Does it represent a development change?Eg changes in policies/regulations/laws, access toassets or services by the poor, environmentalconservation or institutional capacity.(2) Does it require the help of at least one (or perhapsmore) partners?Outcomes are developmental changes which by theirvery nature cannot be achieved by UNDP on its own.(3) Can it be achieved by the end of the SRF/CCF period?Results included in the SRF must be achievable withinthe SRF period, or the CCF period (if this is later).Outputs and outcomes of varying degrees ofambition can be chosen. Guidance on choosingOutputs(1) Does it make a significant contribution to theachievement of the related outcome?Although UNDP cannot achieve outcomes on its own,we do want to make an important contribution to theirachievement.(2) Can it be delivered by UNDP?Again, although you could argue that even outputsare achieved with the help of at least one partner,UNDP should feel confident that it has a high degreeof control over the achievement or non-achievementof the output.(3) Can it be achieved by end of the SRF/CCF period?Results incorporated into the SRF, including outputs,must be achievable within the SRF/CCF period.
UNDP Results FrameworkTechnical Note: 2000Deciding the Ambition of Outputs andOutcomesFigure 2 illustrates how outputs and outcomes ofvarying degrees of ambition may be chosen. Theapproach in UNDP is to select only those outputsthat clearly have a significant role to play incontributing towards major outcomes. Similarly,since outcomes occupy a middle ground betweenoutputs (completion of activities) and theachievement of impact, it is possible to defineoutcomes with differing levels of ambition.Where in this range you locate your intendedoutcome depends on country conditions, theimportance of UNDP assistance, UNDP’s trackrecord, and the contributions of partners. Theaim is to define outcomes that UNDP and itspartners will have to stretch themselves toachieve but which, at the same time, can beseen to have a significant and crediblerelationship to outputs that UNDP iscontributing. Box 2 provides examples ofoutputs and outcomes and illustrates the varyinglevels of ambition that are possible.The first output (see (1)) in Figure 2 – localdecentralisation feasibility study completed – is atthe lower end of the ambition scale, and clearlyrepresents a result over which the UNDP CO hashigh degree of influence. This type of outputwould be appropriate in a country where theconcept of decentralising resources anddecision-making authority is new.The second example (see (2)) – draftdecentralisation proposals submitted – is moreambitious, and might follow-on from a feasibilitystudy. Although UNDP managers can influencethe preparation of these proposals, they haveless control over whether the submission will beaccepted by the government. The importance ofPage 4country context is clear. This more ambitiousoutput is appropriate where UNDP had built up asufficiently strong reputation and relationship withits partners in order to feel confident of includingit within the SRF.With respect to outcomes, the first example (see(3)) involves the transformation of a draftproposal on decentralisation into actuallegislation. The degree of influence which UNDPhas over this is less than for the outputs, for theBox 2: Example Outputs and OutcomesThe following example outputs reflect an increasing level ofambition and an increasing reliance on the securing thetrust and cooperation of partners. It is importance to defineoutputs which are likely to make a significant contribution tooutcomes, but which take account of the country contextand are therefore realistic within the timeframe of theCCF/SRF:Outputs:Completion of a study of environment-poverty linkages;Police forces and judiciary trained in understanding ofgender violence;National, participatory forum held to discuss draftnational anti-poverty strategy;National human development report produced;Revised electoral dispute resolution mechanismestablished.The examples below illustrate outcomes with an increasinglevel of ambition. As the level of ambition increases, so theimportance of partnership grows, and the directness orcredibility of linkages to key UNDP outputs diminishes.Outcomes:Improved national capacity to monitor human andincome poverty and inequality;Legislation passed which promotes local adoption ofsustainable energy technologies.Increased access of the poor to finance (formal,informal, micro).Reduction in the level of domestic violence againstwomen.Increased regional and sub-regional trade.Page 4
UNDP Results FrameworkTechnical Note: 2000Page 5Figure 2: Setting the Level of Ambition for Outputs and OutcomesOUTCOMESIncreasing importance of partnerships(3) Draftdecentralizationproposal legislatedby parliamentINPUTS(4) Increase in %of nationalresources raisedand managed atlocal levelIncreasing level of ambition(Becoming closer to outcomes)IMPACTIncreasing level of ambition(Becoming closer to impact)(1) Localdecentralizationfeasibility studycompleted(2) Draftdecentralizationproposalscompleted andsubmittedOUTPUTSreason that it is Parliament not UNDP that canlegislate. Moreover, the successful passage ofthe legislation may well depend on a range ofoutputs beyond the draft proposals such as theholding of a national consultation ondecentralisation.Achievement of the more ambitious outcome(see (4)) – increase in the percentage of nationalresources raised and managed at the local level– is clearly contingent on the actions of a widerange of partners and is likely to be achievedover a longer time span.This level of outcome is likely to be appropriateonly in a country with an established consensusand where UNDP has a proven track record withrespect to decentralisation and thus theconfidence of major partners.This lattercondition is important if UNDP is to be in aposition to contribute substantial outputs thathave a credible link to the outcome. If UNDP isnot in a position to contribute major outputs, thenthe level of ambition of the outcome should belowered accordingly.
UNDP Results FrameworkTechnical Note: 2000Implications of Focusing on ResultsThe following sets out the implications of theresults-based focus in broad terms. Section IV, incontrast, examines in more detail the structureand practical application of the SRF.(a) Capturing the Results of “Soft”AssistanceWith the introduction of RBM in UNDP, addedimpetus has been given to capturing theachievements of UNDP’s “soft” assistancebased on the organisation’s presence at thecountry level.In addition to promoting results through hardoutputs such as the development of managementinformation systems or studies, UNDP makes animportant contribution to promoting developmentchange through “soft” assistance, namely policydialogue and advocacy. While this may becarried out through UNDP programmes andprojects, dialogue and advocacy is a major focusof the work of Resident Representatives andother senior UNDP staff. For this reason, it isessential that the SRF capture the resultsstemming from the total operating unitstrategy of the country office. UNDP countryoffice presence itself can contribute to changethrough direct provision of “soft” assistance, inaddition to “soft” assistance provided throughprojects and programmes. The outputs andoutcomes of “soft” assistance should, however,be clear and tangible (see Figure 3).“Soft” assistance can help trigger positive resultsin the most fundamental dimensions of theenabling environment e.g. when contributing to apolicy change that makes national planning moreresponsive to the needs of the poor. Although“soft” assistance interventions may lead to resultsPage 6only in the long-term, and these results may behard to quantify, every effort should be made todefine concrete intended results. In support ofthis, each thematic category of the SRF beginswith strategic areas of support that focus onchanges in the relevant policy, regulatory or legalframework, usually achieved through “soft”assistance.Figure 3: Soft interventions should have hardresults“SoftIntervention”e.g.: advocacyand policydialogue(throughprojs/progs orregular work ofRR, etc)OutputsOutcomese.g.:new policy,legislation, sprepared;A major nationalconferenceorganised to build aconsensus forchange(b) Forging Strategic PartnershipsDeveloping and systematically following apartnership strategy is an essential requirementfor the successful achievement of outcomes.The very process of defining and agreeing thelimited number of major intended outcomeswhich UNDP intends to support requires dialoguewith national counterparts. The nature of thisdialogue and the type of partnership that UNDPis able to build will shape the major outcomeswhich will be included in the Country CooperationFramework and the Strategic ResultsFramework.Effective partnerships, where there is a clearunderstanding of the contribution of each partnerto agreed outcomes, are central to achievingresults. Exerting a maximum influence on anoutcome demands a thorough understanding ofthe interests of other development actors, theroles they can play, how best to work with them,Page 6
UNDP Results FrameworkTechnical Note: 2000and how to mobilise resources. Many outcomes,such as the expansion of a competitive, marketoriented private sector for instance, clearly canonly be achieved through the collective efforts ofseveral partners. The diversity of partnershipsthat may be required is highlighted in Figure 4below. Partnership and coordination strategiesare, thus, also essential and integral componentsof planning for outcomes. National authorities,beneficiary groups, the private sector, and otherdevelopment organisations are usually thestrategic partners for UNDP.Effective partnerships draw on individualstrengths and maximize synergies. Furthermore,in seeking to influence outcomes, UNDP musthave a clear understanding of its own role – interms of leadership, coordination, resourcemobilisation, and monitoring and evaluationarrangements. The importance of partnerships isrecognised in the SRF, which requires that COsset out a partnership strategy statement for eachoutcome (see Section IV).Page 7enhanced through the systematic monitoring ofindicators through the thematic categories of theSRF and assessment through the ROAR.Evaluation helps clarify the underlying factorswhich explain the results reported through theROAR, and helps ensure that UNDP learns fromits lessons of experience. There is a need todevelop ways of evaluating the achievement ofkey outcomes, through partnership-centredevaluations.Within a results-oriented environment, theemphasis of M&E is on:active application of monitoring andevaluation information to the continuousimprovement of strategies, programmes andother activities;monitoring of substantiveresults instead of justimplementation processes;developmentinputs andmonitoring and evaluation of results as theyemerge instead of as an ex-post activity;conduct of monitoring and evaluation as jointexercises with development partners.Figure 4: Partnerships – at the core ofachieving outcomes(d) Use of IndicatorsNationalauthoritiesPrivatesectorCivil societyorganisationsUNDPUN agenciesOUTCOMESOtherdonors(c) Monitoring and EvaluationMonitoring and evaluation (M&E) are key toolsfor the effective implementation of results-basedmanagement. Performance assessment isRBM’s emphasis on outcomes achieved jointlywith other partners requires the monitoring ofchange beyond the confines of an individualUNDP-supported intervention. For outcomeswhich UNDP contributes to in partnership withothers, not all monitoring responsibilities have tofall upon UNDP. It is important, however, that thepartners agree on the key indicators to monitor,and who has responsibility for monitoring them.UNDP makes use of three types of indicator:-corporate outcome indicators set centrallyand providing a standardised basis for
UNDP Results FrameworkTechnical Note: 2000monitoring changes UNDP wants to beassociated with globally(See Section IV);-outcome indicators, identified by the countryoffice measuring progress against specifiedoutcomes; and-situational indicators, which provide a broadpicture of whether the developmentalchanges that matter to UNDP are actuallyoccurring (see Annex II of this TechnicalNote for the list of situational indicators).Indicators are observable signals of status orchange that are intended to provide a crediblemeans of verifying results. Effective identificationof indicators is important for two reasons. Firstly,the ability to track progress and learn lessonsrelies on the selection of indicators that isolatethe essential changes sought.Secondly, the process of defining indicators itselfcan help managers in clarifying the outcomesthey seek. If it proves difficult to identify anoutcome indicator, it usually reflects a lack ofclarity in conceiving the outcome, or theexcessively broad or ambitious nature of theoutcome sought. Where possible, indicatorsshould be derived from a dialogue with UNDP’spartners.In defining indicators, it is important to rememberthat they should be used to provide approximateanswers to a few important questions rather thanseek to provide exact answers to many lessimportant questions. Balance is key, in order toprevent the process of defining and monitoringindicators from becoming a major workload.Critical qualities of indicators include:Validity: does the indicator capture the essenceof the desired result?Practicality: are data actually available atreasonable cost and effort?Clarity: do stakeholders agree on exactly what tomeasure?Page 8Clear direction: are we sure whether an increaseis good or bad?Owned: do stakeholders agree that the indicatormakes sense?Balance requires that the definition and useof indicators has to be taken seriously forcredible and effective assessment, learningand accountability. On the other hand, caremust be taken not to over invest in resultsmeasurement and indicators. If measurement isemphasised too much, there is a risk thatmanagers will be motivated to undertake certainactivities simply because measurable results canbe achieved. In the process, they may bediverted from less measurable, but ultimatelymore fruitful, development interventions.Examples of outcome indicators are given in Box3, and fuller guidance on “do’s and don’ts” isgiven in “Selecting Key Results Indicators: inthe context of the UNDP SRF” which can dology/methodology/html or theintranet at http:/intra.undp.org/osg. This covers awide range of topics, including: indicators assignposts of change; management uses; types;targets; baselines; and economy in choice ofindicators.Selecting indicators for the softer dimensionsof the work of UNDP, i.e., policy advice,dialogue and advocacy, is likely to bechallenging. These must capture policy advice,dialogue and advocacy outcomes relating to thetotal operating unit strategy, and not just projectsand programmes. The focus must be onidentifying the hard results that emerge from softinterventions. Critical discussion is useful sincepronouncing a decree or issuing a formalstatement may at times be only a nominal signalof a policy shift, not truly reflecting actual change.The most significant aspect of policy change mayPage 8
UNDP Results FrameworkTechnical Note: 2000not be formal adoption but how the changemanifests itself in actions.Box 3: Example Outcome IndicatorsSample indicators for Enabling Environment Sub-goal 2,“Strengthen the capacity of key governance institutions”,include:Legislative mechanisms for accountability such as aPublic Accounts Committee are in place to ensureoversight function.Average time for passage of legislation in parliamentduring the current year.An independent electoral management body existswith the capacity to conduct free and fair elections.Existence of public confidence in the justice system (asindicated by survey data)For the Gender Sub-goal 1, “Ensure gender equality in thedecision-making processes at all levels”, sample outcomeindicators include:Incorporation of gender-disaggregated socio-economicanalysis and findings into national and local plans.Frequency with which women’s and gender equalityconcerns are raised in the media;National baseline studies and regularly updated dataon gender issues available in the country,disaggregated by sex (including GDI and GEM).Examples of policy change may be parliamentaryapproval of legislative amendments that arerequired to execute new policy directions, e.g.,for the establishment of a new governmentfunction, programme or institution. The actualallocation of budget resources or theimplementation of new programmes or servicescould be other observable signals of materialpolicy change. Capturing these results ofUNDP’s “soft” interventions immeasurablystrengthens the ability to tell UNDP’s story and todemonstrate its role and impact.Capacity development activities have alwaysbeen difficult to measure, so how should this bePage 9done? Again, there are no pre-establishedformulas. However, past experience can behelpful. When dealing with capacity developmentprojects and programmes, the critical question indefining indicators is to ask the question capacityfor what? Determining change depends on acareful analysis of the function that the specificinstitution performs (or seeks to perform). What isthe essential service being provided, and whoare the clients of the institution? The concept ofclient service may be helpful in defining resultsemanating from capacity development efforts.Even if there is no room for undertakingextensive client surveys, the data may exist onother aspects of client service: institutionaloutreach – the number of clients that have beenserviced; the time it takes to process a complaint;the backlog of cases. Even for complexinstitutional issues such as planning, it may stillbe possible to identify a concrete indicator, suchas the timeliness of sectoral reports or the viewsthat external experts have about the quality ofdocuments.UNDP’s Two-Way Results FrameworkA key lesson which has emerged not only fromthe experience of other public sector agenciesbut also from UNDP’s own efforts is the need forresults management systems to be rigorous yetsimple and flexible. They need to be tailored tothe specific country needs of the organisationwhile responding to corporate requirements.The UNDP results system, therefore, builds on atwo-way process (top-down and bottom-up):top-down: the establishment at corporatelevel of an overarching Goal and a broad setof Goals, Sub-Goals, Strategic Areas ofSupport, and corporate outcomes andindicators;
UNDP Results FrameworkTechnical Note: 2000bottom-up: the articulation of actual resultsplans – intended outcomes and outputs – aswell as associated outcome indicators andpartnerships by operating units, primarily atcountry level.The convergence of the top-down and bottom-upapproaches yields a single integrated system forstrategic planning – from the country or operatingunit level to the consolidated corporate level (seeFigure 5).Figure 5: Top-down and Bottom-up ApproachesPage 10improving its credibility in the country; andaligning and assessing the performance of COstaff based on common, transparent andverifiable results.Figure 6: Strategic Results FrameworkTop-down componentsGOAL:Sub-Goal 1:Strategic Areas ofSupportIntended OutcomesOutcome IndicatorsIntended OutputsSAS1SAS2Overarching GoalGoalsSub-GoalsStrategic Areas of tom-up artnershipsPlans andactualresultsThis system is built on a single instrument:the Strategic Results Framework (SRF) (seeFigure 6). The SRF is used for strategic planningand forms the basis for performance assessmentthrough the results-oriented annual report(ROAR).The SRF has a wide range of managementuses, for example, providing a basis for aresults-oriented dialogue between UNDP,Government and other partners; enablingprogrammes and projects to be focused on a fewstrategic results rather than being dispersedacross too many areas; guiding the allocationand management of programme resources, bylooking at the relationship between results andresource use; sharpening UNDP’s identity andThe outcomes and outputs furnished within theSRF by operating units should reflect the keyresults against which managers wish to beassessed. However, because outcomes are notthe result of one single actor’s intervention,enforcing individual and personal accountabilityin respect of the substantive attainment ofoutcomes would be unreasonable. Theattainment of outcomes is subject to sharedaccountability among partners. A fullunderstanding of the factors determining whyoutcomes are achieved (or not) requires in-depthevaluation that situates the results in the countrycontext. While managers can be heldaccountable for ascertaining that outcomes aremonitored, their full accountability can be appliedonly to outputs. In a situation of sharedaccountability, it is important that responsibilitiesand performance expectations be clearly defined.Managers, while not being held accountable forthe achievement of outcomes, are expected toreport on progress against intended outcomes.Page 10
UNDP Results FrameworkTechnical Note: 2000The ROAR provides a vehicle for sharpeningperformance, including as a RBM assessmentand reporting instrument. It is intended to providea basis for feedback and continuous adjustment.The Country Office Management Plan (COMP)is connected to but remains separate from theSRF, focusing on the management actionsrequired for achievement of intendeddevelopment results.Thus the SRFconcentrates on the substantive developmentresults which UNDP wishes to achieve, bothindividually and collectively with partners, whilethe COMP focuses on how the CO expects tomanage itself in order to help achieve thosedevelopment results (see the COMP TechnicalGuidelines for further information).Page 11These six thematic categories reflect the four keyprogrammatic areas that have been defined forUNDP;5 the importance accorded to specialdevelopment situations/crisis countries category,and UNDP’s role as Chairman of the UNDG andfunder/manager of the Resident CoordinatorSystem (RCS).The SRF has a logical, hierarchical structure (seeFigure 7). At the highest level is an over-archingCorporate Goal that represents the unifyingpurpose of UNDP’s work:To contribute to the eradication of extremepoverty and the substantial reduction of overallpoverty (WSSD Commitment 2), UNDP will focuson: sustainable human development withparticular emphasis on the reduction of humanpoverty.Figure 7: SRF StructureIV. THE SRF: STRUCTURE ANDAPPLICATIONThis section explains the structure of the SRFand provides guidance on its application,including examples that highlight specific issues.The Structure of the SRFThe SRF provides a broad frame of reference forresults in six areas that are critical to UNDP:The enabling environment for SHD(governance);Poverty reduction;Environment;Gender;Special development situations (SDS);UNDP support to the UN.OverarchingCorporate GoalGoalsSubGoalsStrategicAreas ofSupportThe SRF is structured into the following “topdown” elements for each of the six thematiccategories:5 Executive Board decisions 94/14, 95/23 and 98/01.
UNDP Results FrameworkTechnical Note: 2000Goals, which represent the ultimate aims ofassistance in each of the thematic categoriesbut are not exclusive to UNDP. They arealmost all geared towards the over-archingcorporate goal of reducing human povertyand, at the same time, build on , which are specific to UNDP,reflect UNDP’s comparative advantages andshow how the organisation contributes to theoverall Goal.Strategic Areas of Support (SAS), which areareas in which UNDP can or already doeswork, as a key player or leader/coordinator,to help achieve the Sub-Goal and, ultimately,the Goal in a particular thematic category.The SAS are now determined exclusively atthe corporate level: operating units will not beable to add any SAS to the corporatelydefined list from 2000 onwards.The purpose of defining the Goals, Sub-Goalsand Strategic Areas of Support is to provide aframe of reference for operating units to definethe specific results that they will strive to achieveat their level. This “bottom-up” part of the SRFhas four major elements:Page 12the SRF, it may be appropriate to chooseoutcomes achievable within the CCFtimeframe. In this case, the outcome will berealised after 2003, and the value of theindicator should reflect the progressexpected by the end of the SRF period.Key outputs, which are specified by COs andother operating units. COs will identify a fewkey outputs with a significant anddemonstrable contribution to the outcome.Outputs will be specified so as to include aclear current year target against whichprogress can be measured.This approach to intended outputs reflect thelessons of experience in 1999 which showedthat, in most cases, output indicators weresimply a reformulation of the output itself. Forthis reason, the structure of the SRF hasbeen simplified by the removal of the outputindicator column.Typology of results: a new typology of resultshas been developed based on the empiricalevidence contained in country ROARs andconsultations within UNDP (see below). Thiswill allow a better comparative analysis ofresults across regions, categories of results,and over time.Intended outcomes, which are the keydevelopment changes to be targeted byUNDP assistance. They are defined byCountry Offices (COs) and other relevantoperating units (see al
UNDP but what makes the current approach different is the determination to make RBM the driving force behind the organisation's institutional culture and practice - and to develop . seen to have a significant and credible relationship to outputs that UNDP is contributing. Box 2 provides examples of outputs and outcomes and illustrates the .
