WIXLIB

C.Australian volunteers gain professionally and personally. Well supported and strategically placedvolunteers that have a good experience and contribute to partner organisations are at the heart of asuccessful volunteering program. The logic states that if the program supports a diverse range ofvolunteers who have the right skills, attitudes and attributes for the assignment, then those volunteerswill complete their assignments well and have a positive experience. Whilst on assignment andafterwards, volunteers will promote greater cultural awareness and build strong connections betweenpartner countries and Australia which will result in the volunteers gaining both personally andprofessionally.The program will support the volunteers to gain professionally and personally by: ensuring the program is inclusive and accessible to a broad range of skilled Australians (in line with theprogram’s Diversity and Inclusion strategy’s focus on: Disability inclusion; Indigenous persons; Diasporaand diverse cultural backgrounds; Gender equality; LGBT ; Remote, rural and regional persons; Youngerand older persons)having strong volunteer recruitment practices (targeted to country needs)supporting volunteers prior to, on-assignment and afterwards with fit-for purpose learning anddevelopment programs and networksBy successfully achieving these three end of program outcomes, the Australian Volunteers Program will havecontributed to longer term development and diplomacy goals: the Australian Government is achieving its development objectivesAustralians become more globally literate and connectedAustralian aid is perceived positively in Australia and overseasThe program logic is available in Annex 1.4.2. Assumptions (risks to delivery of program outcomes)The achievement of program outcomes, as articulated in the program logic, is based on a number of underlyingassumptions, which define conditions assumed to be in place. Reframed, these assumptions are actually risksto the delivery of program outcomes.Tracking progress against the program logic through monitoring and regular reflection, provides theopportunity to determine whether these assumptions have held or not, and whether or how they will affectthe achievement of results. The MELF data collection tools aim to provide insights into these assumptions(risks). As part of the Annual Reflection events (Section 6.1), assumptions in the program logic will be reviewedand tested.The assumptions are: there will be ongoing bi-partisan support for the Australian Volunteers Programvolunteers have a positive volunteering experiencethe successful delivery of volunteer assignments will make a real contribution to partner organisations’ability to achieve their objectivesthe success of volunteer assignments and partner capacity development, will meaningfully contribute toAustralia’s development and public diplomacy objectivespotential, current and returned volunteers have a positive perception of the Australian aid program andAustralia’s development objectives overseas7

most volunteers will want to share their volunteering experiences and articulate how volunteerscontribute to development outcomes with others in Australia and overseas Australian organisations, partner organisations, champions and DFAT see value in and will activelypromote the benefits of the volunteer program with the right support, volunteering overseas can be an attractive and accessible opportunity to skilledAustralians from all walks of life a diverse and relevant range of Australian organisations and in-country partner organisations will beinterested in and have the capacity to partner with the program DFAT post, Canberra and State offices will be interested in strategically engaging with the volunteerprogram and leveraging the program to support the delivery of development outcomesRisks to delivery of the MELF are captured in Section 8.5. Assessing progressThe approach to assessing progress in the Australian Volunteers Program draws on some of the key principlesof developmental evaluation3 to provide, as far as possible, real time data that helps to interpret what isworking and what is not within the program. Whilst the Australian Volunteers Program has a (relatively) linearand seemingly simple program logic (Annex 1), the reality is that the Australian Volunteers Program isoperating in a multitude of complex operating environments. Building capacity of partner organisations, andunderstanding how to influence public diplomacy are situation specific and complex undertakings. Theelements of developmental evaluation being applied include: A heavy emphasis on monitoring using multiple sources and obtaining (as far as possible) real time dataIdentifying patterns in data that may prompt new thinkingInterpretation and meaning being assigned by those involved in delivering the program through a processof asking what does the data tell us? So what does it mean for the way we operate and the partnershipswe have? Now what are we going to do (same or differently)?The focus is on learning and using reflection to test whether we are on the right path to achieving thedesired outcomes5.1. Key evaluation questionsKey evaluation questions are high level questions designed to frame the analysis of progress and performanceof the Australian Volunteers Program. They will be used by the program to help structure regular reflectionevents, and to guide reporting. The key evaluation questions in the design document were reviewed andrefined by the program team.Effectiveness What have been the outcomes4 / results (intended and unintended, positive and negative) of theprogram for volunteers, partner organisations, development in partner countries, and raisingawareness of the value of volunteering?To what extent has the program contributed to these outcomes?To what extent has the program improved the reach and diversity of: partners in Australia and overseas (partner organisation type, size, sector)volunteers including gender, disability, ethnic background (Aboriginal, Torres Strait Islander, ordiaspora community), age, location3Patton, M (2010), Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York, NY: Guilford PressInclusive of outcomes in relation to gender equality, disability inclusion, private sector engagement, climate change, innovation and childprotection.48

avenues and audiences for public diplomacy?Value for money How efficiently is the program being delivered, including achieving its outputs as intended (on time, ofgood quality, on cost)?To what extent has the program been able to learn and improve its approaches to managing avolunteering program, including through trialling new approaches?Relevance How relevant is the program to partner country priorities, Australian aid priorities5, and partnerorganisation priorities?5.2. Program indicatorsThis section provides a summary of the regular indicators of Australian Volunteers Program progress and thetools used to collect this data. Annex 2 provides more detailed information on the indicators and data sources.The program indicators are used to gather regular information on the progress of the Australian VolunteersProgram. They provide “windows” for viewing progress on the program. Regular data collection and analysis(using the key evaluation questions as higher level guides) will help the team to understand areas of progress,and risks to delivery. The interpretation of why progress is fast or slow, is done informally through regularfeedback/conversations with implementing teams and partners, as well as formally through the annualreflection events (section 6.1). Where the Australian Volunteers Program team can’t plausibly explain reasonsfor progress, the MEL team can commission further evaluative research (section 5.4), or the program canrevisit the assumptions in the Program Logic (section 4.2) to ensure these still hold true.Whilst the program logic doesn’t detail key program management indicators, these support effective andefficient program management, are required as part of the contract, and provide the evidence base foranswering the key evaluation questions on value for money. Management indicators are provided in thediagram below for consideration.Most information is available through the online Management Information System. By integrating key datainto the Management Information System the program will provide real time data on a number of keymeasures as outlined in Figure 2 below.Where meaningful, all indicators relating to volunteers will be disaggregated by gender. Such indicators will befurther disaggregated by other diversity characteristics where the sample size allows it and where doing so willnot identify individuals.5Ibid.9

Figure 2Real time data collected through the MELFPartnershipsNumber and types of Australianorganisations, partner organisations,Australian-Partner organisatinpartnerships by:-location/countryorganisation typesectorlength of engagementNumber of volunteers supportedSDGsPublic DiplomacyVolunteersNumber and nature ofcommunications activities (events,social media, print and radio media)where benefits and achievements ofvolunteering promotedNumber of volunteers, returnedbrand mentions, reach, traffic volumecumulative program total) by:and audience engagement on AVPmedia channelsvolunteers, and applicants (onassignment, new this month,cumulative total this year,--Gender and inclusion(disability, regional Australian,ethnicity)assignment typeprofessional background andsectorEffective and Efficient Program ManagementContract ManagementRisk and Financial ManagementProgram Learning and QualityX (target) number of volunteersNumber, nature and severity of# of innovative approaches trialledrecruitedChild Protection issuesand assessed.% key deliverables on time andNumber, nature and severity of% of attendees at regional andto DFAT quality standardssecurity and work, health and safetyannual reflection sessions thatincidents (%age managedreport learning and using evidenceprofessionally and timely)to inform decision making% of volunteers, POs, AOs andPost report positive experienceof transition [first year only]% of financial reports produced on% compliance with operationstime with accurate detailmanual via annual country auditsOther indicators where data will be available but less frequently include:10

Figure 3Additional MEL data collectedPublic DiplomacyPartnerships% of the Australian public whoVolunteers% of POs that demonstrate progresssupport international volunteering% volunteers who are satisfied withon development outcomes & nature% of the Australian public who havethe level of personal and professionalof these outcomes.heard of the Australian Volunteersdevelopment achieved.% POs that demonstrate progress onProgram% volunteers demonstrating progressorganisational capacity development% increase in the Australianagainst key competenciesindicators & nature of capacityVolunteer Program’s share of voicedevelopment.and brand reach% of POs identifying benefits andsatisfaction with the program(reasons why)Nature, duration, satisfaction andbenefits of relationships between POsand volunteers, AOs and POs.% volunteers and POs who rate theexperience of working together as% of POs and Assignments that aligngood/very goodwith country priorities% of volunteers who report feeling% of volunteers who report a safe &supported by the programproductive work environment at thePO5.3. Tools for data collectionThe MELF uses new technologies for providing a more meaningful and robust data to measure the progress ofthe program. Advancement in digital technology has led to the creation of tools, offering a wide range of usefulvisualisations and qualitative and quantitative data capture and analysis.Our approach to data collection and management will be: User-centric which ensures a meaningful involvement of service users from listening, acting upon userfeedback, to consider user voice in planning, implementation and review.Using assessments to inform strategy and performance management by regularly analysing andresponding to data and using it to change and improve. This means acting on assessment findings duringthe life cycle of the intervention rather than just at the end.Collection of data on a regular basis as opposed to a one off activity, and aimed at learning, makingcorrections, and addressing issues — rather than just reporting and accountability.Utilising a variety of data collection tools in line with the above-mentioned approach. These includeconducting micro-surveys, social listening, and feedback from current and returned volunteers throughinteractive sessions and online tools.Proportionate, only collecting data and at a frequency that is necessary for the uses outlined in the MELF.There are 11 key data sources for the Australian Volunteers Program regular monitoring approach as outlinedin Table 1.11

Table 1Data sources for the Australian Volunteers Program MELFData Source/Tool6DescriptionFrequency of collectionData availabilityVolunteer assignmentfeedbackCaptured in MISAt the commencement of the assignment partnerorganisations and volunteers complete anassignment plan.At the mid-point (for assignments 11 months andover) the partner organisations and volunteerreview progress. At the end of the assignment theVolunteer provides feedback. (partnerorganisations feedback captured independently).The end of assignment evaluation issupplemented with anexit interviewMid-point (forassignments 11 monthsand over)End of assignmentContinuous and ongoingPartner organisationend of assignmentevaluationCaptured in MISAt the end of each assignment, the programrequests feedback from the partner organisations,provided independently of the volunteerEnd of assignmentContinuous and ongoingPartnership Plan andreview.Captured in MISAt the commencement of a partnership a threeyear plan is established, identifying capacity needsand options for volunteer support. This generatesvolunteer assignments (including, where relevant,a pipeline for up to 3 years).The partnership plan is reviewed on a 12 to 18month basis6 to track progress, including oncapacity development outcomes and to receivefeedback on volunteer assignments.Progressive roll-out and12 to 18 month reviewProgress against PartnershipPlan updated on a rolling basis.Due to volunteer cycles, data onpartner organisations capacitydevelopment may not beavailable until end Y2 and thencan be updated annually.Case studies / stories ofchange from partnerorganisationsCase studies collected from partner organisationsabout the impact of the program on theirdevelopment outcomes, using an outcomeharvesting approach. Analysed to share learningduring reflection sessions.AnnualAnnually on a rolling basis.ManagementInformation SystemStores and enables analysis of basic informationon volunteers and partner organisationsOngoingContinuousMedia monitoringand media analyticsMonitoring: To capture stories that are profiled inthe media about international volunteering.Analytics: To capture insights about social mediasites including engagement, reach, contentperformance, reactions, audience demogra

5 key evaluation questions to guide all monitoring, evaluation and learning activities indicators for regular monitoring tools to support regular data collection on the indicators, and