WIXLIB

A Guide to Monitoring and Evaluating Adolescent Reproductive Health Programs7 DATA COLLECTION AND THE M&E WORKPLAN 107Preparing for Data Collection 107Types of Data Collection Methods 112Selecting Appropriate Data Collection Methods 118Collecting Data 125Developing a Workplan for Monitoring and Evaluation 1278 ANALYZING M&E DATA131Processing M&E Data 131Analyzing M&E Data 1339 USING AND DISSEMINATING M&E RESULTS149Why Use and Disseminate M&E Results? 149Using M&E Results to Improve and Strengthen Your Program 150Disseminating M&E Results to Others 151Tailoring Dissemination of Results to Different Audiences 152Common Dissemination Formats 15210 TABLES OF ARH INDICATORS155Where Are the Indicators in the Tables From, and How Can I Use Them for My Program? 155What Kinds of Indicators Will I Find in Each of the Four Tables? 155What Other Information Will I Find in the Indicator Tables? 157Indicator Table I:Program Design Indicators 159Indicator Table II:Program Systems Development and Functioning Indicators 169Indicator Table III:Program Implementation Indicators 183Indicator Table IV:Program Intervention Outcome Indicators 193GLOSSARY 219BIBLIOGRAPHY 231APPENDICES1 SAMPLING SCHEMES FOR CORE DATA COLLECTION STRATEGIES 243How to Choose a Systematic Sample of ClustersCluster Sampling for Household SurveysAlternative Methods for Choosing Sample Households, Youth and ParentsCluster Sampling for School-based SurveysHow to Allocate a Proportional Sample of Students to SchoolsCluster Sampling for Health Facility SurveysAlternative Methods for Sampling Service Transactions and Clients for Exit InterviewsSampling for Peer Education Program EvaluationsSampling for Client Follow-up SurveysSampling for Focus Groups and Other ÒSmall GroupÓ Data Collection EffortsSampling for In-Depth Interviews2432472502522552572602612622632632 HOW TO CALCULATE SAMPLE SIZE REQUIREMENTS 2653 REFERENCE SHELF 2694 EVALUATION WEB SITES 271vi

Table of ContentsPART II: INSTRUMENTSINSTRUMENTS AND QUESTIONNAIRES 275Adapting Instruments to Meet Your M&E NeedsDeveloping SurveysDeveloping and Leading Focus Group DiscussionsUsing Mystery Clients2752752782791 CHECKLISTS 2811A1B1C1D1EProgram Design ChecklistChecklist of Stakeholder InvolvementTraining Course Checklist for ARH Program StaffChecklist for ÒYouth-FriendlyÓ Service CharacteristicsChecklist of Selection Criteria for Peer Educators2 TALLY SHEETS2A2B2C2D2E281283285286288289Monthly Tally Sheet for CounselingTally Sheet for Communication ProductsTally Sheet for Stakeholder InvolvementTally Sheet on Number and Characteristics of Youth CounseledInstitutional Infrastructure Tally Sheet2892902912922943 REPORTING FORMS 2973A3BReporting Form for Counseling 297Peer EducatorsÕ Reporting Form 2984 ARH COALITION QUESTIONNAIRE 2995 COMPOSITE INDICES 3035A5BIndex on Quality of Counseling (for Individual Counseling Sessions) 303Policy Environment Score: Adolescents 3046 INVENTORY OF FACILITIES AND SERVICES 309Background CharacteristicsSection 1: Equipment and Commodities InventorySection 2: Conditions of FacilitySection 3: IEC Materials and ActivitiesSection 4: SupervisionSection 5: Protocols and GuidelinesSection 6: Use of Information in Facility ManagementSection 7: Service StatisticsSection 8: StaffingSection 9: Fees for Services3093113173183203213223233243257 OBSERVATION GUIDE FOR COUNSELING AND CLINICAL PROCEDURES 327Counseling ObservationContraceptive MethodsDiscussion of STIs and Other Health IssuesMedical ProceduresInterviewer Impressions of Consultations3283303323333348 INTERVIEW GUIDE FOR STAFF PROVIDING RH SERVICES 335Background CharacteristicsExperience and Training in Reproductive Health ServicesContraceptivesOther Reproductive Health PracticesSocio-Demographic Characteristics335336340343345vii

A Guide to Monitoring and Evaluating Adolescent Reproductive Health Programs9 GUIDE FOR CLIENT EXIT INTERVIEW347Background Characteristics 347Section 1: Basic Features 348Section 2: Information About Services 34910 QUESTIONNAIRE FOR DEBRIEFING MYSTERY CLIENTS 357Background Characteristics 357Questions for Mystery Clients 35811 COMMUNITY QUESTIONNAIRE363Section 1: Community Information 364Section 2: Reproductive Health Services in the Community 366Section 3: Identification of the Facility 36812 COMPREHENSIVE YOUTH SURVEY373Table of ContentsIntroductionModule 1: Background and Related InformationModule 2: Reproductive Health KnowledgeModule 3: STI/HIV/AIDSModule 4: Attitudes, Beliefs and ValuesModule 5: Social InfluencesModule 6: Sexual Activity, Contraception, and PregnancyModule 7: Skills and Self-EfficacyModule 8: Leisure Activities and ConcernsModule 9: Media InfluenceModule 10: Drugs and AlcoholModule 11: Health-Seeking 3443844044313 FOCUS GROUP DISCUSSION GUIDE FOR IN-SCHOOL ADOLESCENTS 44514 ASSESSING COALITION EFFECTIVENESS WORKSHEET 447I. Collaborative Structure and Community Context 447II. Collaboration Staffing and Functioning 44815 PARENTS OF YOUTH QUESTIONNAIREviii451

Part I: The How-To’s of Monitoring and EvaluationIntroductionAround the world, young people are growing up in an environment of dynamicchange. For some, this complexity provides opportunity and choice; for others,it means a struggle for survival. Many young people have stamina and energy,curiosity, a sense of adventure and invulnerability. They are resourceful and resilient evenunder the most difficult conditions.The period of adolescence is, however, a life phase in which young people are particularlyvulnerable to health risks, especially those related to sexuality and reproduction: HIV/AIDS,unwanted pregnancy, unsafe abortion, too-early marriage and childbearing, sexually transmitted infections and poor nutrition.How adolescence is experienced andaffects reproductive health has largely to dowith the timing and sequence of sexual initiation, marriage and childbirth; the degreeto which the timing and sequence of theseevents are socially sanctioned or forbidden;and the number and availability of optionsregarding education, job training andemployment. There is a great deal of variation worldwide, and even within countries,in the social and cultural values that shapethese events. Close relationships betweenyouth and their parents and extended family are particularly important in influencingyouth development. Access to preventiveand curative services, including contraception and treatment for sexually transmittedinfections, are also important in ensuringthe reproductive health of youth.Youth development programs designed tohelp young people reduce their reproductive health risks reflect that variation. Manyof these programs regard young people as acritical resource for the future, and use creative strategies to tackle their complexproblems. But many programs face limitedfunding, community resistance, nonsupportive laws and policies or lack of experience. By knowing more about what worksin youth programs and services, we canbuild strong programs that accomplish whatthey intend.Reproductive health refers to the health andwell-being of women and men in terms ofsexuality, pregnancy, birth and their relatedconditions, diseases and illnesses. Manyprograms reaching youth are trying toachieve reproductive health goals thatrelate to critical sexual and reproductivehealth outcomes, such as: fertility: the number of pregnancies awoman has in her lifetime abortion: as it relates to fertility and tohealth complications for women whohave unsafe or clandestine abortions1

A Guide to Monitoring and Evaluating Adolescent Reproductive Health Programs illness: caused by sexually transmittedinfections, reproductive tractinfections, HIV and/or nutritionalstatus mortality: primarily related topregnancy and childbearing, includinginfant and maternal mortality, and alsoincluding AIDS-related deaths nutritional status: which impacts bothwomenÕs health and that of theirinfantsNoteWhat do we mean by “youth”?Programs reaching young people use different terminologyto refer to youth. “Adolescents” is often used to refer toyoung people ages 10–19, “young adults” generally refers tothose ages 15–24 and “youth” may refer to all young people ages 10–24.This guide encompasses each term and usesthe phrase “adolescent reproductive health” (ARH) tocover each type of program.Why Monitor and Evaluate YouthPrograms?Monitoring and evaluation shows ifand how youth programs are working.Monitoring and evaluation (M&E) can tell usif and how program activities are working.Program managers and donors want to beable to demonstrate results, understand howtheir programs are working and assess howthe programs interact with other events andforces in their communities.M&E can be used to strengthenprograms.Program managers and staff can assess thequality of activities and/or services and theextent to which the program is reaching itsintended audience. With adequate data, youcan compare sites, set priorities for strategicplanning, assess training and supervisoryneeds and obtain feedback from the target2audience or program participants. You canprioritize resource allocation, improve information for fund-raising, provide informationto educate and motivate staff, provide information for advocacy and argue for the effectiveness of your program approach.M&E results can help institutionalizeprograms.M&E results can help stakeholders and thecommunity understand what the programis doing, how well it is meeting its objectivesand whether there are critical needs inhibiting your progress. M&E results can be usedto educate your board of directors, currentand prospective funding agencies, localgovernment officials and key communitymembersÑsuch as local leaders, youthand parentsÑwho can help ensure social,financial and political support for youthprograms. Sharing results can help your program establish or strengthen the network ofindividuals and organizations with similargoals of working with young people. It canalso give public recognition and thanks tostakeholders and volunteers who haveworked to make the program a success,and may attract new volunteers.M&E shapes the decisions of fundingagencies and policymakers.Funding agencies and policymakers areinterested in monitoring and evaluationresults for a variety of reasons. They need tomake strategic choices about how to spendresources and to prove that the expenditureproduces quality results. M&E results alsohelp with decisions about identifying andsupporting the replication or expansion ofparticular program strategies. M&E findingsoften reveal unmet needs or barriers to program success and can be used to lobby forpolicy or legislative changes. M&E resultscan raise awareness of youth programsamong the general public and help buildpositive perceptions about young peopleand youth programs.

IntroductionM&E results contribute to the globalunderstanding of “what works.”The dissemination of M&E resultsÑboththose that show how your program isworking and those that find that somestrategies are not having the intendedimpactÑcontributes to our global understanding of what works and what doesnÕt inimproving young peopleÕs reproductivehealth. This advances the field by building abody of lessons learned and best practicesthat can strengthen ARH programs aroundthe world.M&E mobilizes communities tosupport young people.Monitoring and evaluation results enablecommunities and youth to inform localleaders about youth needs and to advocatefor funding. Results point to ways in whichwe can develop new and better systems ofsupport for young people and identify additional community resources. They canincrease the communityÕs understanding ofthe potential and actual benefits of the program and its accomplishments, develop asense of ownership through participation,improve coordination and mobilize supportfor youth and the array of programs thatfoster their health and development.Who Should Use This Guide?This Guide is designed for program managers who monitor and evaluate adolescentreproductive health programs. Some examples of the people who might find thisguide useful include the following: Community-level programmanagers: A manager of acommunity youth centerÕs peereducation program can use this Guideto set up a system to monitorimplementation of program activities.NoteSeeking outside helpMonitoring and evaluation is an essential aspect of youthreproductive health program development. However, manyprograms do not have the expertise to carry out someaspects of program evaluation, especially when evaluatinglarge, complex programs. After reading this Guide, you maychoose to seek technical assistance from local universitiesand research institutes who have the expertise to help youdesign and conduct an effective and efficient evaluation.Guide to track progress in theprogramÕs implementation. Municipal-level health managers:A manager of a clinicÕs pregnancy andsexually transmitted infection (STI)reduction program can use this Guideto set up an evaluation that will trackchanges in the incidence of pregnancyand STIs among youth in the entiremunicipality. State- or provincial-level healthofficials and managers ofnongovernmental organizations(NGOs): An official at the state level ina health system can use this Guide tocompile data across districts,municipalities or other ge

Monitoring and Evaluating Adolescent Reproductive Health Programs Tool Series 5, June 2000 Susan Adamchak Katherine Bond Laurel MacLaren Robert Magnani Kristin Nelson Judith Seltzer. In July 2011, FHI became FHI 360. FHI 360 is a nonprofit human development organization dedicated to improving lives in lasting ways by