WIXLIB

LearningandGrowingthroughEvaluationState AsthmaProgramEvaluationGuide

Learning and Growing through EvaluationState Asthma Program Evaluation GuideThis is a one and three-quarterinch side bind that can be usedto label the side of your threering binder.uation Guide

Learning andGrowingthroughEvaluationState AsthmaProgramEvaluation Guide

Copies of Learning and Growing through Evaluation: State Asthma Program Evaluation Guidewhich can be viewed or downloaded from the APRHB websitehttp://www.cdc.gov/asthma/program eval/guide.htm.Suggested citation according to format of Uniform Requirements for Manuscripts Submitted toBiomedical Journals*:Centers for Disease Control and Prevention. Learning and Growing through Evaluation: StateAsthma Program Evaluation Guide. Atlanta, GA: Centers for Disease Control and Prevention,National Center for Environmental Health, Division of Environmental Hazards and HealthEffects, Air Pollution and Respiratory Health Branch, April 2010.Centers for Disease Control and Prevention,National Center for Environmental Health,Division of Environmental Hazards and Health Effects,Air Pollution and Respiratory Health BranchWebsite: http://www.cdc.gov/nceh/airpollution/E-mail: -* International Committee of Medical Journal Editors. Uniform requirements for manuscripts submittedto biomedical journals. Ann Intern Med 1988: 108:258–265.

AcknowledgmentsWe gratefully acknowledge the following individuals and organizations who contributed to developing, writing,editing, reviewing, designing, and producing this document.EXECUTIVE DIRECTIONPaul L. Garbe, DVM, MPHMichael A. McGeehin, PhD, MSPHAUTHORSIndependent Evaluation ConsultantLeslie A. Fierro, MPHBattelle Centers for Public Health Research and EvaluationCarlyn E. Orians, MALinda Winges, MAJoanne Abed, PhDShyanika W. Rose, MACONTRIBUTORSCenters for Disease Control and PreventionThomas Chapel, MBAKari H. Cruz, MPHSheri Disler, BAGema Dumitru, MD, MPHSarah Gill, MSElizabeth J. Herman, MD, MPHMichele Mercier, MPHAmanda Savage-Brown, PhDRobin Shrestha-Kuwahara, MPHMaureen Wilce, MSMembers of APRHB-State Evaluation WorkgroupsPartnership Evaluation WorkgroupSurveillance Evaluation WorkgroupInterventions Evaluation WorkgroupREVIEWERSState Asthma Department of Health StaffCarina Blackmore (FL)Kristine Caroppoli (PA)Manzoor Choudry (NC)Vadim Drobin (PA)Rebecca Giles (UT)Marcia Henning (NM)Debra Hodges (AL)Carrie Huisingh (MA)Peng Li (OK)Van Nguyen (IL)Justin Peng (CT)Christine C. Rameker (WI)Air Pollution and Respiratory Health Branch StaffCathy Bailey, MSTheresa Harrington, MD,Jeneita Bell, PhD, MPHMPH&TMTegan K. Boehmer, PhDShahed Iqbal, PhDDan Burrows, MS, HSACarol Johnson, MPHDavid Callahan, MDChristi Jones, BSPaige C. Cannon, BAMichael King, PhD, MSWHuey Chen, PhDZong Law, MSJacquelyn H. Clower, MPHDavid M. Mannino, MDGary Coil, MPHPat McCarty, BAPamela Collins, MPA, MSASarah Merkle, MPHGregory Crawford, MSPHJeanne Moorman, MSScott Damon, CPH, MAIATeresa A. Morrison, MD, MPHMarcia Griffith, MPHTursynbek Nurmagambetov, PhDDESIGN AND PRODUCTION STAFFBattelle Centers for Public Health Research and EvaluationMarcía TreeceKate BlessingDeanna Rossi (CA)Alejandro Amil Rosario (PR)Sarah Schillie (IL)Elizabeth Traore (NH)Elizabeth Wasilevich (MI)Lei Zhang (MS)Emeka Oraka, MPHCara Person, MPHIsabela Ribeirio, PhDShubhayu Saha, PhDJeremy Sarnat, ScDJoannie Shen, MD, PhD, MPHKathryn R. Sunnarborg, MPHLinda Thomas-Houston, BASameera R.Wijayawardana, MSFuyuen Y. Yip, PhD, MPHHatice Zahran, MD, MPH

Asthma Program Evaluation GuideIntroduction and OverviewThe Centers for Disease Control and Prevention’s (CDC) National Asthma ControlProgram and state asthma programs across the country are mobilizing their resources toreduce the burden of asthma in our communities. The sound evaluation practices detailedin Learning and Growing through Evaluation can help ensure that we use those resourceseffectively and efficiently; that we have a means of demonstrating the value of our programs;and that we are developing a body of knowledge that tells us “what works.”Learning and Growing is an evaluation guide intended for use by state and territorial publichealth departments (SHDs) that are receiving CDC funding for state asthma programs. Othergroups that focus on improving asthma management practices, whether or not they receive CDCfunding, may also find elements of the guide useful in designing and implementing their ownprogram evaluation activities.CDC’s approach to public health program evaluation takes into account the great variety amongstate programs, offering a framework that can be tailored to specific programs. As diverse as thestate programs are, all share common aspirations with each other and with CDC’s Air Pollutionand Respiratory Health Branch (APRHB). These include: Reducing morbidity and mortality from asthma Reducing asthma disparities Improving quality of life for asthma patients and their families Sustaining and improving statewide asthma programsAs its title suggests, this guide focuses on learning together how we can reach our goals and ongrowing in our capacity to systematically examine, or evaluate, our efforts.Organization of the GuideThe guide is comprised of two modules that provide an overview of the CDC Framework forEvaluating Public Health Programs (MMWR, 1999) as applied to asthma programs.Fundamental to CDC’s approach is an emphasis on generating information or knowledge thatwill be useful to the many people invested in a program’s success. Whether you are new toprogram evaluation or have years of experience, becoming familiar with this approach will giveus a common vocabulary and support our work together on this cooperative agreement.The first module applies the CDC Framework to evaluation planning. It addresses both thecreation of a strategic evaluation plan, which is designed to prioritize the use of scarce evaluationresources over the life of the cooperative agreement, and also planning sound evaluationstrategies for use in evaluating specific program activities. A second module covers methods forevaluating the three main components of state asthma programs: surveillance, partnerships, andinterventions. The second module is designed to be consulted as needed when consideringevaluation strategies and preparing individual evaluation plans.The three-ring binder format has been chosen to permit the addition of supplemental material asyou use the guide over time. Pages are numbered consecutively within chapters. Tables andPage iIntroduction

Learning and Growing through Evaluationfigures are also numbered consecutively within chapters to facilitate cross-referencing. Werecommend that you print this document in color, if possible, to take full advantage of its designfeatures. The guide is designed for printing one page to a sheet and double-sided.Tools and TemplatesThroughout the guide you will find a number of templates and checklists that should facilitateyour evaluation planning and practice, particularly if you are new to evaluation. They can beeasily adapted to the particular context in which your program operates. The examples given areprovided for illustrative purposes only and are not meant to promote one particular evaluationquestion or method over another. By following the planning process outlined in the guide, youwill arrive at your own conclusions regarding proposed evaluations. Blank templates in MSWord are available from your APRHB Evaluation Technical Advisor.Additionally, we have created vignettes in which we follow a fictional state asthma programcoordinator, Sofia, who is relatively new to evaluation, and her recently hired evaluator,Anthony. The vignettes provide snapshots of how the two work together to plan for evaluation.We follow them as they engage stakeholders, propose evaluation candidates, and set prioritiesfor evaluation. A short review highlighting the evaluation points illustrated in the vignettesfollows each one.To encourage you to reflect on what you are reading and how the information can be applied toyour program, we have inserted blank “Notes” pages at random intervals throughout thedocument. Feel free to use these to jot down ideas as they occur to you.The field of program evaluation has a rich history. We have provided a few select resources inthe text of the guide and in individual appendices, a longer topical resource list in Appendix G,and a comprehensive alphabetical listing of references in Appendix H. If you would like to learnmore about a particular aspect of program evaluation, the APRHB will gladly provideinformation about resources and training opportunities.Finally, as with any specialized field, evaluation has its own technical vocabulary. We haveincluded an appendix with notes from each chapter. Terms and concepts covered in Appendix AChapter Notes are highlighted in blue bold and marked with a leaf icon in the margin. We havealso included a glossary, Appendix B; terms included in the GLOSSARY are highlighted in green,bold, and small caps. Clicking on either the blue or green highlighted terms will take you directlyto the appendices.CDC is committed to supporting states as they discover and share “what works” in their asthmaprograms. By learning and growing together through evaluation, we can contribute to Americabreathing easier.IntroductionPage ii

MODULE 1

Asthma Program Evaluation GuideModule 1Table of ContentsPageChapter 1. Evaluation and Your State Asthma Program. 1-1Preparing for Successful Evaluation . 1-2The Underlying Framework. 1-3A Common Vision . 1-5Brief Introduction to Models and Graphic Representations . 1-9State Asthma Program Impact Model . 1-9Chapter 2. Thinking Strategically: The Strategic Evaluation Plan . 2-1Step A. Establish an Evaluation Planning Team. 2-5Step B. Describe the Program . 2-6Step C. Prioritize Program Activities for Evaluation . 2-7Step D. Consider Evaluation Design Elements. 2-12Step E. Develop a Cross-Evaluation Strategy . 2-19Step F. Promote Use through Communication . 2-23Step G. Write and Revise Your Strategic Evaluation Plan . 2-25What Have We Learned?. 2-25Chapter 3. Planning for an Evaluation: The Individual Evaluation Plan . 3-1Step 1. Engage Stakeholders . 3-2Step 2. Describe What is Being Evaluated . 3-3Step 3. Focus the Evaluation Design. 3-4Step 4. Gather Credible Evidence . 3-4Step 5. Justify Conclusions . 3-5Step 6. Ensure Use of Evaluation Findings and Share Lessons Learned. 3-9Pulling It All Together—How the Evaluation Will Be Managed. 3-10Revisiting the Strategic Evaluation Plan . 3-11What Have We Learned?. 3-11

Learning and Growing through EvaluationList of TablesPageTable 1.1 Six Steps in the CDC Framework for Evaluating Public Health Programs . 1-4Table 1.2 Standards included in the CDC Framework for Evaluating Public Health Programs . 1-4Table 2.1 Program Activity Profile. 2-7Table 2.2 Potential Criteria for Evaluation Prioritization . 2-8Table 2.3 Activities Rank Ordered by Criteria. 2-11Table 2.4 Example Evaluation Question Development Table (partially completed). 2-13Table 2.5 Example Evaluation Design and Data Collection Summary (partially completed) . 2-15Table 2.6 Issues to Consider When Looking Across Proposed Evaluation Strategies . 2-20

Asthma Program Evaluation Guide Introduction and Overview T he Centers for Disease Control and Prevention’s (CDC) National Asthma Control Program and state asthma programs across the country are mobilizing their resources to reduce the burden of asthma in our communities. The sound evaluation practices detailed inFile Size: 1MBPage Count: 149