Transcription
PRIORITIZATIONANDCAPACITY ASSESSMENTFINAL REPORTPublic Health Improvement ProcessBoulder County, ColoradoJune 14, 2011Primetime Research and EvaluationMargret Dugan, Ph.D. & Laurie James‐Hawkins, M.A.www.primetm.net
INTRODUCTION and BACKGROUNDINTRODUCTIONIn 2008, the Colorado Public Health Act (Senate Bill 08‐194) was signed into law. The primarypurpose of the bill is to assure that core public health services are available to every person inColorado with a consistent standard of quality. Under the new law, each county must establish— or be part of — a local public health agency. The state of Colorado has developed acomprehensive public health plan that outlines how quality public health services will beprovided. Each local health department must conduct a community health assessment anddevelop its own corresponding plan as soon as practicable.The Act requires that Boulder County Public Health (BCPH) conduct assessments of communityhealth status and the capacity of the broader public health system to address health needs. Itmust then prepare a county public health plan on the basis of these assessments, includingsetting priorities for providing services. Beyond planning, BCPH must ensure that a core set ofpublic health services are provided equitably. In doing so, BCPH must ensure that initiatives arecollaborative, consistent with state plans, and in alignment with resources. BCPH formallyinitiated this process, known as the Public Health Improvement Process (PHIP), in 2009.Beyond contributing to state level health improvement, the Boulder County PHIP is onecomponent of a larger local public health quality improvement process known as the NationalPublic Health Performance Standards Program (NPHPSP). The NPHPSP supports local healthsystems answering questions such as “What are the components, activities, competencies andcapacities of our public health system?” And, “How well are the Ten Essential Public HealthServices (see description below) being provided in the local public health system?” The TenEssential Public Health Services are the means of addressing any and all priorities identified. In2010, BCPH embraced the opportunity to use NPHPSP methods and tools to assess the localhealth system capacity.In 2011, BCPH contracted with Primetime Research and Evaluation (Primetime) to lead a formalprioritization and capacity assessment based on the NPHPSP process. From March throughJune 2011, Primetime staff developed methods to prioritize BCPH activities and measure itscapacity to provide public health services according to Ten Essential Services developed by theNPHPSP.This report details the capacity assessment designed by the NPHPSP to identify the strengths,limitations, gaps, and needs of the current public health system in Boulder County, Colorado. Inaddition, a summary of the prioritization process and suggested recommendations areincluded. The results detailed in this report are intended to serve as the impetus for thedevelopment of a system‐wide strategic improvement plan.Prioritization and capacity assessment, PHIP, Boulder County 20112
BACKGROUNDIn an effort to strengthen public health practice, systems‐based performance, and public healthinfrastructure, the NPHPSP was formed in 1994. The Centers for Disease Control andPrevention (CDC) spearheaded the national partnership initiative with seven key public healthorganizations as well as input from state and local public health departments to develop publichealth performance standards and evaluation measures and tools. To this end, they developeda framework of the Ten Essential Public Health Services they felt most important to reachoptimal levels of public health performance nationwide. These are:The Ten Essential Public Health Services1. Monitor health status tocommunity health problems.identifyandsolve2. Diagnose and investigate health problems andhealth hazards in the community.3. Inform, educate, and empower people abouthealth issues.4. Mobilize community partnerships to identify and solvehealth problems.5. Develop policies and plans that support individual and community health efforts.6. Enforce laws and regulations that protect health and ensure safety.7. Link people to needed personal health services and assure the provision of health carewhen otherwise unavailable.8. Assure a competent public health and personal health care workforce.9. Evaluate effectiveness, accessibility, and quality of personal and population‐basedhealth services.10. Research for new insights and innovative solutions to health problems.To help local public health departments measure their capacity and performance on the TenEssential Public Health Services, the NPHPSP also developed four standards by which toevaluate their performance on the Ten Essential Public Health Services. They are:1. Ten Essential Public Health Services are the fundamental framework for describingpublic health activities that should be undertaken by all states and communities (seeabove description).Prioritization and capacity assessment, PHIP, Boulder County 20113
2. A focus on the Public Health System means all public, private and voluntary entities thatcontribute to the delivery of essential public health services in the community, county,or jurisdiction. This may include public health agencies, healthcare providers, publicsafety agencies, human service and nonprofit organizations, education and youthdevelopment organizations, recreation and art‐related entities, economic andphilanthropic organizations, and environmental organizations or agencies. (The publichealth system in Boulder County includes – but extends far beyond – BCPH.)3. An Optimal level of performance is required to initiate, build upon and maintainessential public health services. Optimal standards provide every public health systemwith benchmarks by which the system’s effectiveness may be judged. When comparingthe local system’s performance to the optimal or “gold” standard, the public healthsystem is able to pinpoint areas of strength to build upon or those programs which mayneed to be improved or removed.4. Continuous quality improvement is the process that NPHPSP promotes in order toassess strengths and weakness. This standard is intended to support a process ofquality improvement.System partners use the assessment process and theperformance standards results as a guide for learning about public health activitiesthroughout the system and determining how to make improvements.To assist the local public health departments to conduct their capacity and performanceassessments, and to provide a standardized tool by which they could compare the resultsacross the sites, the NPHPSP developed the Ten Essential Public Health Services and fourmeasurement standards into three assessment tools designed to assess the major activities,components, and practice areas of the state and local public health systems: a State Public Health System Performance Assessment Instrumenta Local Public Health System Performance Assessment Instrumenta Local Public Health Governance Performance Assessment InstrumentThese three assessment tools are comprised of a series of questions to be answered at thestate level and the local level. The assessment questions serve as the performance measuresand all questions are preceded by model standards, which represent the optimal levels ofperformance based on a set of indicators that are unique to each essential service. Scoring isbased on a 5‐point scale, and the CDC provides analysis for local sites using an algorithm.Undertaking a public health improvement process, as BCPH has, is a valuable tool in identifyingareas for system improvement, strengthening state and local partnerships, and assuring that astrong system is in place for effective response to day‐to‐day public health issues as well aspublic health emergencies. Local public health capacity assessment instrument users findnumerous benefits to their undertaking an assessment, including:Prioritization and capacity assessment, PHIP, Boulder County 20114
Improvement of organizational and community communication and collaboration, bybringing a wide variety of diverse partners to the same table. Education of participants about public health and the interconnectedness ofactivities, which can lead to a higher appreciation and awareness of the manyactivities related to improving the public’s health. Strengthening of the diverse network of partners within state and local public healthsystems. This can lead to more cohesion among partners, better coordination ofactivities and resources, and less duplication of services. Identifying strengths and weaknesses to address in quality improvement efforts.Responses to the assessment can be tracked over time to identify systemimprovements or changes. Providing a benchmark for public health practice improvements, by providing a “goldstandard” to which public health systems can aspire.This report provides a description of BCPH’s capacity assessment of the health system inBoulder County and a comprehensive review of the quantitative and qualitative results. Theintended audience for this report includes participants involved in the formal assessmentprocess, public health practitioners and stakeholders, and others interested in supporting localpublic health system‐based efforts. It is hoped that BCHP and the wider health system may usethese assessment findings as the basis for identifying a strategic direction for enhancingperformance.The findings detailed in this report are framed in positive language terms. The discussioncaptured during the individual sessions, as well as the final scores, are intended to clarify andemphasize constructive comments that can assist in the development of a strong and strategy‐driven public health system improvement process.Although it is anticipated that BCPH’s leadership will take a lead role in facilitating system‐leveldesign and development of a capacity and performance improvement process in the prioritizedareas identified, the overall assessment and quality improvement efforts require participationand a commitment from all levels and components of the health system in Boulder County.Prioritization and capacity assessment, PHIP, Boulder County 20115
OVERVIEW OF PUBLIC HEALTH PRIORITIZATIONAND CAPACITYASSESSMENT PROCESSPrioritization ProcessTo begin, Primetime staff performed a thorough review of the archival records and relevantliterature regarding the prioritization process. Following this, interviews were conducted withthe BCPH PHIP Core Team, as well as selected BCPH employees and a variety of communitystakeholders. The interviews consisted of a review of a list of 30 health outcomes and werefollowed by a discussion of what the interviewee felt were the most important health outcomesto prioritize over the next five years. The results from this process are presented below inSection One: Prioritization Results Summary.Capacity Assessment ProcessThe formal health system capacity assessment (HSCA) in Boulder County was conductedthrough a series of ten 3‐hour meetings held during the month of May 2011. Each meetingfocused on one of the Ten Essential Public Health Services. These meetings were conducted asfacilitated focus groups consisting of 6‐10 invited participants. These participants’ tasks were 1)to review the evaluation measures developed by the NPHPSP for that session’s essentialservice, 2) participate in the group discussion about the health system’s capacity andperformance of that public health function, 3) come to consensus on a rating score of thecurrent capacity, and 4) to express concerns and provide recommendations for improvement.The group facilitators recorded the sessions on digital recorders; the discussion and votes, aswell as comments about the assessment tools themselves, were captured on a laptopcomputer. The meeting was divided into two sessions. The first reviewed the prioritiesrecommended to the health system as a result of data review and staff/community interviews(n 50). Each priority was presented and each group discussed their perceptions regarding therecommended priorities. The second session consisted of going through each question in theNPHPSP capacity assessment instrument and rating the system on each essential service.Each meeting was structured to complete the following: The participants reviewed the standards for each question posed in the assessment tool. Without discussion, an initial vote was taken to determine how each participant ratesthe public health system in Boulder County is in terms of level of activity in regards tothe question. If there was consensus, no further discussion occurred and the vote was recorded asfinal. If, however, there was not consensus, a facilitated discussion began and continued untilconsensus was reached or the time keeper stopped further discussion. NOTE: Ifconsensus could not be reached within the time limit agreed to, then a majority votewas taken. This did not occur very often. All votes were recorded throughout the capacity assessment and an oral summary waspresented at the end of the session.Prioritization and capacity assessment, PHIP, Boulder County 20116
Key Stakeholder ParticipationParticipants selected for each meeting were chosen based on area of expertise or interest andrelationship to the essential service to be discussed. Invitations were sent to a broad range ofkey partners from the local public health agency, state service agencies, community basedorganizations, academic institutions, hospitals, school systems, foundations, law enforcementagencies, and non‐profit organizations. Additionally, invitations were sent to people in localgovernmental or quasi‐governmental entities including first responders, elected officials, socialservice providers, administrators, diversity advocates, and others. Invitations were also sent topeople in the business community, media, and judicial institutions. Approximately 70 keyparticipants (6‐10 per meeting) responded to the request to participate in the assessmentsessions. Although there were sectors of the public health system in Boulder County whichwere underrepresented during the capacity assessment process (i.e. local hospitals), Primetimeand the participants in the groups felt that the results were representative of the system.Process LimitationsAlthough attempts were made to encourage participation from multiple stakeholders, somerepresentatives were missing from the process, including those from the business community,media, and judicial and insurance institutions. The assessment format (one three hour meeting plustravel time) may have precluded some participants, especially those in high profile or demandingroles, from engaging in the meetings. The time commitment may also have hindered the ability ofsome to participate due to lack of employer support or conflicting priorities. It is also possible thatthe group process deterred introverted individuals who prefer less interactive approaches.Reaching true consensus on each question was deemed to be unattainable in the given timeframeor when an individual(s) was unable to concede to the larger group. As mentioned above, majorityvote was used to capture individual item scores when consensus was not achieved. Minority voteswere not recorded and are not part of the overall scoring using the CDC algorithm.Assessment ToolThe local capacity assessment tool developed by the NPHPSP (version 2) contained over threehundred questions designed to generate discussion and a rating of each of the major activities,components, and practice areas comprising the ten essential services of the local public healthsystem. The assessment questions are designed to serve as the performance measures. Allquestions are preceded by model standards, which represent the optimal levels of performancebased on a set of indicators that are unique to each essential service.Quantitative Scoring, Data Entry, and Data AnalysisSession participants were asked to classify the percent of activity that was met within the localpublic health system for each essential service, using a five‐point classification rating scale: NONE:0% of the activity met MINIMAL: 0%‐25% of the activity met MODERATE: 26%‐50% of the activity met SIGNIFICANT: 51%‐75% of the activity met OPTIMAL: 75%‐100% of the activity metPrioritization and capacity assessment, PHIP, Boulder County 20117
An algorithm, developed by the CDC, was utilized to develop scores for every essential publichealth service. Each question was assigned a point value and given a weight depending on thenumber of questions and tiers. The score range was 0 to 100, with higher scores depictinggreater performance in a given area. Each response was entered into the CDC database foranalysis. A report was generated from this database highlighting the quantitative results.Qualitative Data Collection and AnalysisIn addition to the scores that were collectively assigned by each group after discussion andconsensus building, qualitative data, including recordings of all discussion topics andconclusions as well as comments on the assessment tool, was analyzed into common themesand summarized for this report.Assessment Tool LimitationsThe five‐point rating scale delineated in the tool was awkward. (Please see the “CommentsRegarding the Instrument” section accompanying each meeting discussion detailed below.)Often the question was worded to require a “yes” or “no” response, at the same time thatwording forced participants forced to quantify according to the rating scale as an activity levelpercentage. Participants were frequently reminded, for example, that a response of “no” didnot connote an appropriate response, but rather reflected “no activity,” classified as 0 percent.Activity meanings in each specific question often required the explanation of a unique set ofqualifiers and definitions from an 86‐page glossary of definitions provided with the assessmenttools by the NPHPSP. This then entailed long discussions about the intended meaning of thequestion. At times, the researchers endeavored to interpret a common meaning for a questionno one clearly understood. Even the researchers had to stretch for a meaningful explanation ofambiguous wording. This was frustrating and wasted valuable meeting time.Scoring LimitationsThe scores are subject to the biases and perspectives of those who chose to participate in the groupand who engaged in the group dialogue. Although dissenting and positive statements are recorded,the majority vote may not have adequately reflected the viewpoint of some participants. Everyattempt was made to capture all comments made during the lively discussions, but,understandably, this cannot always be guaranteed.Generalizability of ResultsThe results of this assessment are based on a facilitated group process conducted during a specifictime period that captured the current opinions of the participants assessing the ten essentialservices in which they function. As such, this assessment provides a “snapshot” approach. Changesto the local public health system at all levels occur constantly. The assessment process is subjectivebased on the views of those who agreed to participate.Prioritization and capacity assessment, PHIP, Boulder County 20118
SECTION ONE: PRIORITIZATION RESULTSPreliminary Health Outcomes Identified in Initial Prioritization ProcessThe prioritization process Primetime undertook identified the following health issue areas astop concerns for the individuals in the local public health system who participated in theinterview process: 1) Mental Health, 2) Substance Abuse, 3) Unplanned and Teen Pregnancy,and 4) Obesity.Mental HealthAlmost everyone interviewed identified Mental Health as a top health issue area and indicatedthat they felt current resources were inadequate to meet the need in Boulder County.Discussion of the impact of mental health on physical health, as well as the wide spread impactdepression and other mental health problems have on those around the sufferer were alsomentioned.Substance AbusePeople also agreed that substance abuse was an issue in Boulder County. Many people felt thatthis fell under the heading of mental health; many others identified it as a separate issue.Again, the primary feeling was that there were drastically limited resources compared to theneed they see within the county.Unplanned and Teen PregnancyThis was mentioned by many as a fundamental problem that needs to be addressed in that ithas major consequences for the parents and children involved and also for the county and itsmonetary output in services for these families. In addition, this was frequently cited as anunderlying issue for many of the health problems on the outcomes list and was directlyconnected to both mental health issues and substance abuse.ObesityThis was identified as a major health concern despite its absence from the original list ofoutcomes. The majority of people felt that this was an underlying cause of most of the healthoutcomes on the list presented to them. They also felt that this was an area that wasspecifically a problem in East Boulder County and that it is an important one to address in orderto improve communitywide health.Concerns and Issues about Preliminary Prioritization of Health Issue AreasDuring the capacity assessment small group meetings, Primetime revisited the health issueareas that had been recommended as PHIP focus areas in order to gain more feedback from awider audience before final recommendations were made. There was strong support and nodisagreement with identifying both mental health and substance abuse as top focus areas forthe next five years. There were questions regarding unplanned and teenage pregnancy,primarily due to the general feeling that Boulder County already does a lot of work in this area.Prioritization and capacity assessment, PHIP, Boulder County 20119
Finally, there was also some disagreement as to the inclusion of obesity in the list of outcomesand discussion as to if it really was an outcome. People in the small groups and in the PHIPCore Team meetings mentioned the following items which they felt needed to be assessed forinclusion in the prioritization process: 1) vaccine‐preventable disease, 2) unintentional injury, 3)oral health, 4) cancer, and 5) heart disease. Desire to review and integrate quantitative data tocompliment the qualitative process undertaken by Primetime was also mentioned.RecommendationsPrimetime recommends the following in terms of prioritization:1) Choose no more than three focus areas in order to ensure that resources can beadequately allocated to obtain measurable results.2) Mental Health and Substance Abuse were the only priorities identified with broad basedsupport in both BCPH and within the community. Therefore, Primetime suggests thatthese be adopted as focus areas one and two.3) If Unplanned and Teen Pregnancy is adopted as a focus area, Primetime recommendsreframing it to be “Prenatal Health” which includes the components of both planning forpregnancy and ensuring healthy birth outcomes.4) If Obesity is adopted as a focus area, special focus should be placed on East BoulderCounty and Latino/a populations.5) The following should be given consideration as top causes of death in Boulder County: a)unintentional injury, b) heart disease, and c) cancer.6) Primetime cautions against spreading resources too thin as this will make measurableprogress difficult to achieve and has the potential to discourage partners within thelocal public health system from participating in future public health improvementefforts.7) BCPH must ensure that there is wide public support for the chosen priorities byincreasing their information, education and outreach within the community.8) BCPH should begin a formal evaluation of the PHIP from the outset of the strategicplanning process.Prioritization and capacity assessment, PHIP, Boulder County 201110
SECTION TWO: OVERALL QUANTITATIVE CAPACITY ASSESSMENTRESULTSWide community stakeholder participation in the NPHPHS assessment process allows us toreport on the capacity of the broad public health system in Boulder County (including andsurpassing BCPH) to conduct the ten essential public health services. Overall, half of theessential services were rated at the significant activity level with forty percent being rated atthe moderate activity level. Ten percent achieved the highest activity rating of optimal. Thehighest ratings were achieved in essential services two (diagnosing and investigating) and six(enforcing laws). The lowest ratings were given in essential services one (monitoring anddiagnosing) and three (informing, educating and empowering).Figure 1: Summary of essential public health service performance scores and overall score (withrange) for the public health system in Boulder CountyFigure 1 (above) displays performance scores for each essential service along with an overallscore that indicates the average performance level across all ten essential services. The rangebars show the minimum and maximum values of responses within the essential service and anoverall score.Prioritization and capacity assessment, PHIP, Boulder County 201111
Figure 2: Rank ordered performance scores for each essential service for the public healthsystem in Boulder CountyFigure 2 (above) displays each composite score from low to high, allowing easy identification ofservice domains where performance is relatively strong or weak.Figure 3: Rank ordered performance scores for each essential service, by level of activity, forthe public health system in Boulder CountyFigure 3 (above) provides a composite picture of the previous two graphs. The range lines showthe range of responses within an essential service. The color coded bars make it easier toidentify which of the essential services fall in the five categories of performance activity.Prioritization and capacity assessment, PHIP, Boulder County 201112
Figure 4: Percentage of essential services scored in each level of activity for the public healthsystem in Boulder CountyFigure 4 displays the percentage of the system's essential services scores that fall within the fiveactivity categories for the public health system in Boulder County.Figure 5: Percentage of model standards scored in each level of activity for the public healthsystem in Boulder CountyPrioritization and capacity assessment, PHIP, Boulder County 201113
Figure 5 (above) displays the percentage of the system's model standard scores that fall withinthe five activity categories.Figure 6: Percentage of all questions scored in each level of activity for the public health systemin Boulder CountyFigure 6 (above) displays the percentage of all scored questions that fall within the five activitycategories. This breakdown provides a closer snapshot of the system's performance, showingvariation that may be masked by the scores in Figures 4 and 5.Prioritization and capacity assessment, PHIP, Boulder County 201114
SECTION THREE: DETAILED QUANTITATIVE AND QUALITATIVECAPACITY ASSESSMENT RESULTS BY ESSENTIAL PUBLIC HEALTHSERVICEThe following graphs and narratives provide the detailed ratings information tabulated byNPHPSP as well as summaries of discussions conducted during each of the ten 3‐hour facilitatedfocus group sessions. Each session focused on one of the ten essential public health services.The discussion summaries are not exhaustive. Each summary reflects opportunities forenhancing performance of the local public health system based on specific language andmeasures identified in the model standards and from input from participants.EPHS 1. Monitor health status to identify community health problemsCapacity AssessmentGroup rating of Capacity Percentage: 38% (lowest rating score of the ten essential serviceratings, which ranged from 38% to 81%, with a mean score of 57%)Discussion SummaryTo date, a single Population‐Based Community Health Profile (CHP) has not been conducted inBoulder County. Consequently, there is not a consistent community health assessment orcommunitywide use of community health assessment or CHP data. However, the groupsuggests that developing a community health profile should:1.2.3.4.Identify community priorities and goalsTrack trends and progressBe widely disseminated across the local public health systemBe reviewed and updated regularlyAlthough there is no health profile per se, participants say that several entities at the local levelare collecting data (e.g., police, hospitals, health systems, insurers), but there is no systematicapproach to coordinate the data collection efforts or to compile the information into aCommunity Health Profile. Although some of the data are available upon request, thePrioritization and capacity assessment, PHIP, Boulder County 201115
incentives for partners to provide this information are unclear, and the current capacity of thesystem to routinely develop and maintain a Community Health Profile is uncertain.On a positive note, the group is certain that there is adequate diagnosis and investigation ofhealth problems and health hazards. It is not all it could be, but progress is being made with theresources, time, and talent that are available. It appears adequate resources do not exist fordeveloping and maintaining a Community Health Profile, and this issue is not a priority in publichealth based on the current funding streams. One person said, “Funding sources do not allowus to be creative with programming and statute requirements often stifle creativity;nonetheless, there is a lot more we can do.”While the foregoing may be accurate, the group feels that the public health system in BoulderCounty should consider putting into place (or begin to build) a system to collect data about thecommunity for health assessment purposes and define the priorities, steps, timeframe,partners, and resources needed. It
In 2011, BCPH contracted with Primetime Research and Evaluation (Primetime) to lead a formal prioritization and capacity assessment based on the NPHPSP process. From March through June 2011, Primetime staff developed methods to prioritize BCPH activities and measure its
