Transcription
2022Employment First QualityAssurance ReportEMPLOYMENT.FIRST@DHSOHA.STATE.OR.US
ContentsIntroduction . 2A. Quality of Planning . 2Quality Assurance Activities . 0VR Outcomes for ODDS Individuals. 1VR Application to Eligibility 2017-2021 . 1VR Eligibility to Plan 2017-2021 Graph . 2VR Application to Employment Outcome . 3Hours Worked Per Week at VR Closure . 4ODE Data . 5Quality Improvement Initiatives . 8B. Quality of Providers. 9Quality Assurance Activities . 9Quality Improvement Initiatives . 12C. Quality of Services . 13Quality Assurance Activities . 13VR Transition to ODDS Graph. 16ODDS to VR Transition Tables . 17Quality Improvement Initiatives . 19D. Quality of Training . 20E. Quality of Data . 22Quality Assurance Activities . 22Quality Improvement Initiatives . 23F. Quality of Life . 24Quality Assurance Activities . 24NCI Data Graphs . 26Quality Improvement Initiatives . 27Summary and Conclusion . 271 Page2022 Employment First Quality Assurance Report
IntroductionThis report summarizes the outcomes of the Employment First QualityAssurance Plan. The objectives defined in the 2014 Quality Assurance Planprovide the framework for the metrics reported here. The last Quality AssuranceReport was completed for State Fiscal Year (SFY) 2016. For the purposes ofthis document, all metrics available for SFY 2017 through SFY 2021 will bereported.Employment First in Oregon is committed to successful employment outcomesfor people with intellectual and developmental disabilities. Throughout thisreview period, the Office of Developmental Disabilities Services (ODDS),Vocational Rehabilitation (VR), and the Oregon Department of Education (ODE)have implemented new and revised guidance and protocols to ensure the mostperson-centered outcomes are met, and people who want to work are affordedthe supports needed to reach their employment goals.Throughout this report it is important to note that COVID-19 had a significantimpact on SFY 2020-2021 outcomes.A. Quality of PlanningQuality improvements in service planning documents across ODDS, VR andODE have been broadly implemented, communicated, and trained to betterreflect the needs and desires of individual. Plans across agencies reflect thesame outcome and are coordinated in annual planning meetings and othercommunications.2 Page2022 Employment First Quality Assurance Report
Quality Assurance ActivitiesGoal 1: Continue to MONITOR AND EVALUATE employment service planning across all agencies to ensureinformed choice and compliance with integrated employment service expectationsStrategiesStatusContinue to monitor and evaluate timelinessOngoing work continues and will remain a top priority for Employmentof service planning across all agenciesFirst. ODDS ISPs include Career Development Plans and are completed(ODDS, VR, ODE).annually.VR has federal metrics it is required to meet and report on. Highlights aresummarized in the following graphs. ODE Individual Education Programdocuments are completed on an annual basis.Work with ODDS QA Staff to enhance themonitoring of the employment-relatedaspects of the current ODDS QA FieldReviewsWork with VR QA Staff to develop data tosupport continued analysis and evaluation ofeffectiveness of program activities.This item was accomplished with implementation of the new employmenttab of the Developmental Disabilities Quality Assurance Field Review toolas reported in 2016.Monitor and evaluate data available fromODE related to effectiveness of employmentservice planning.ODE publishes an annual Post School Outcomes Report. Information onSecondary Transition for Students with Disabilities is aspxTable 1The Lane v. Brown Semi-Annual Report compiles findings on ProviderLevel Outcomes, Wages, Hours Worked, Plan and Closure, as well asadditional metrics identified.
VR Outcomes for ODDS IndividualsGraph 1VR Application to Eligibility 2017-2021Graph 21 Page2022 Employment First Quality Assurance Report
VR Eligibility to Plan 2017-2021 GraphGraph 32 Page2022 Employment First Quality Assurance Report
VR Application to Employment OutcomeGraph 43 Page2022 Employment First Quality Assurance Report
Hours Worked Per Week at VR ClosureGraph 54 Page2022 Employment First Quality Assurance Report
ODE DataPSO data reflects what school leavers did in the first 12 months after leavingschool. Due to this data collection period, the results for each year of schoolleavers is reported two years after their exit date.Table 25 Page2022 Employment First Quality Assurance Report
Graph 6Graph 7Graph 8Graph 96 Page2022 Employment First Quality Assurance Report
Goal 2: Develop and implement TARGETED MONITORING strategies to help evaluate the quality of employmentservice planning across all agenciesStrategiesStatusWork with ODDS QA Staff to develop checklist ODDS Regional Employment Staff currently perform Quality Assuranceand procedures for in depth qualitative fileReviews for Employment Services. These regional staff complete qualitativereview by ODDS and EF field staffmonitoring during these onsite visits to CDDPs and Brokerages, as well asduring visits they make to Provider Organizations. There is follow up trainingPursue coursework or additional training onand technical assistance with CDDPs, Brokerages, and Providerqualitative research techniques for QA StaffOrganizations when warranted.who will be conducting qualitative fieldreviews.Develop plan for monitoring and evaluatingeffectiveness of new Career DevelopmentPlan (CDP) form and planning process.Work with VR to analyze effectiveness of VRAll Vocational Rehabilitation Counselors currently have people withstrategy to assign ODDS consumers toIntellectual/Developmental Disabilities (I/DD) on their caseload.specially trained VR Counselors (DDSpecialists).Develop and implement strategy for monitoring All employment services are individualized and planned as documented in anODDS time-limited services and reviewingindividual’s annual ISP. Time limited services must align with a goal fortime in statusintegrated employment, and services utilized must be oriented with a plan toachieve that goal.Employment Path Facility service settings were required to be HCBScompliant by Sept. 1, 2018. Additionally, it was required to be used incombination with Employment Path Community or another community service.The last Sheltered Workshop in Oregon closed September 2020. Future workshould emphasize qualitative analysis in the Employment Quality AssuranceReviews to ensure time utilized in time limited services is relevant andprogressing toward employment goals.Table 37 Page2022 Employment First Quality Assurance Report
Quality Improvement InitiativesProjectWork with ODE to evaluate feasibility ofenhancing the requirements for the Summary ofPerformance to be a more robust “school leaving”document (consider modeling after NationalTransition Documentation Summit).Work with ODE to evaluate feasibility of requiringthat a Summary of Performance be provided to allindividuals in the Target Population underExecutive Order No. 13-04 instead of just thosestudents who graduate with a regular diploma orwho exceed the age of eligibility.StatusCompleted: The Summary of Performance (SOP) is required under thereauthorization of the Individuals with Disabilities Education Act of 2004(IDEA 2004). The SOP must be given to students that graduate with aregular diploma or age out. This means that students that stay in theeducation system until age 21 automatically receive a Summary ofPerformance upon exit, which includes individuals in the targetpopulation.Additionally, ODE guidance is that all students that leave school with amodified diploma, an extended diploma, or an alternative certificateshould also receive an SOP, which includes individuals in the targetpopulation. The SOP is completed during the final year of a student’shigh school education, and the timing of completion of the SOP may varydepending on the student’s post-secondary goals.Table 48 Page2022 Employment First Quality Assurance Report
B. Quality of ProvidersIn early January 2021, Employment First launched a new workforce website for available jobs in the I/DDfield, such as job coaches, job developers and direct support professionals. Impact Oregon is a job boardwebsite where ODDS providers and case management entities can post available jobs, for free. The websitewas launched January 2021 and can be accessed here: https://impactoregon.careers/.Providers continue to embrace the belief that individuals with I/DD can work in integrated settings.Employment First has established consistency in qualifications and credentialing across programs and haveestablished and implemented core competencies for all employment service providers.All providers who operated sheltered workshops have transformed their operations to no longer providefacility-based workshop services. Providers continue to improve community employment outcomes forindividuals with I/DD, as evidenced in the Employment Outcomes System.Quality Assurance ActivitiesGOAL 1: Monitor implementation of CORE COMPETENCIES for employment service providersStrategiesStatusWork with ODDS and DD Licensing toCompleted: All new employment professionals entering the supportedsupport implementation of coreemployment workforce in Oregon on or after January 1, 2015 must complete corecompetency-based qualifications forcompetency training within 90 days of providing the employment service.ODDS independent and agencyODDS created 12 online core competency modules within the DHS Learningemployment service providers.Management System to be able to track training for employment professionals.Additionally, this requirement is reviewed when employment regional specialistscomplete 120 day and annual reviews for Quality Review and Assurance.Table 59 Page2022 Employment First Quality Assurance Report
GOAL 2: Develop and implement TARGETED MONITORING strategies to help evaluate the quality of employmentservices providers and delivery of employment servicesStrategiesStatusWork with DD Licensing Staff to revise licensingCompleted: Beginning July 1, 2016, all provider agencies must beand certification regulatory requirements and field certified under OAR 411-323 and endorsed under the employment rulereview process, as needed, to reflect integrated(OAR 411-345) prior to providing an employment service.employment expectations.In June 2017, ODDS audited credentials for all 31 Provider Agenciesdelivering employment services who had been “grandfathered” bylicensing to not require an employment endorsement as they had alreadybeen licensed. All Providers were brought into compliance.Work with DD Licensing Staff to develop andCompleted: DD Licensing and regional employment staff are currentlyimplemented enhanced monitoring strategiesconducting field reviews of employment providers. Updates to thisspecific to providers of employment servicesprocess include any new requirements set in rule, such as the core(random quality checks, on-site inspections)competency and credentialing requirements.Table 610 P a g e2022 Employment First Quality Assurance Report
GOAL 3: Develop and implement PERFORMANCE-BASED TOOLS to evaluate employment service providers,incorporating customer feedback where possibleStrategiesStatusSupport implementation of VR Job Development VendorCompleted: VR vendor level data is now reported in theReport Card (including feedback from job developer customer semi-annual Employment First report.satisfaction survey, if possible)Develop common benchmark performance-based standardsCompleted: ODDS provider level data is now available atfor ODDS employment service providers and monitorhttps://spdweb.hr.state.or.us/EOS/Data. Under the ‘Provider’performance against those standardsdropdown, select any individual provider you would like to seeinformation for.Monitor performance of school districts through Post-SchoolCompleted: The PSO data is reported annually.Outcomes (PSO) data on school report cards.Table 711 P a g e2022 Employment First Quality Assurance Report
Quality Improvement InitiativesStrategyWork within DHS (ODDS, VR and DD Licensing) to evaluateexisting requirements for provider quality assurance programs todetermine whether changes are needed to standards, monitoring,reporting, and other requirements to support implementation of theEmployment First Initiative.ResultsCompleted: Cross agency staff determined thatchanges are not needed at this time to existingrequirements for provider quality assuranceprograms.Support development of a state template to collect qualityassurance program data from providers.Evaluate feasibility and desirability of supporting development of amentoring program that matches experienced and successfulemployment professionals with newly certified employmentprofessionals.Completed: In 2018 Employment First included in itstraining and technical assistance contract a stipendfor provider mentoring.If a mentoring program is implemented, consider waiveramendments to classify mentoring employment professionals asexperts, which could enable them to receive additional pay as amentor-classified Employment Specialist.Table 812 P a g e2022 Employment First Quality Assurance Report
C. Quality of ServicesEmployment First program outcomes continue to improve, and more individuals are working in integratedjobs in their communities. Employment services are delivered in a person-centered manner and in the mostintegrated setting possible. Individual choice of providers is enhanced with navigating the EmploymentOutcomes System data, and provider outcomes can be measured by the individual and their team. Qualityassurance measures have been implemented to ensure timely and quality service planning.Quality Assurance ActivitiesGOAL 1: Continue to MONITOR AND EVALUATE OUTCOMES across all agencies and in all service settings to ensurecompliance with integrated employment service expectationsStrategiesStatusMonitor established program outcomes and targets to Completed: The Employment First Report, published annually inlook for trends to inform Employment First qualitySeptember, reports on the Executive Order, Lane v. Brown settlement,assurance and quality improvement effortsand Integrated Employment Plan metrics. These metrics are monitoredby the statewide Employment First Stakeholder Policy Group.Monitor and evaluate timeliness of service deliveryacross all agencies (ODDS, VR, ODE).Completed: Data is reported in Lane v. Brown Semi-Annual Report.Develop additional quality assurance measures uponissuance of sub-regulatory guidance by CMS toensure compliance with HCBS community-basedsettings rules in non-residential service settings suchas where employment services are provided.Completed: CMS requires every state’s settings in which it providesHome and Community Based Services (Medicaid HCBS) to becompliant with federal regulations.Sheltered workshops are presumed to not meet HCBS requirements.The last sheltered workshop in Oregon closed in September 2020.Table 913 P a g e2022 Employment First Quality Assurance Report
GOAL 2: Develop and implement TARGETED MONITORING strategies to help evaluate the quality of employmentservices being provided to individuals with I/DDStrategiesStatusWork with ODDS QA Staff toCompleted: Employment First developed a qualitative file review checklist with thedevelop checklist and procedures for input of a national subject matter expert. Updates to this checklist have beenin-depth (qualitative) file review byimplemented and regional employment specialists currently complete field reviews forODDS and EF field staffqualitative analysis.Pursue coursework or additionaltraining on qualitative researchtechniques for QA Staff who will beconducting qualitative field reviews.Develop plan for monitoring andevaluating effectiveness of newDiscovery/Career Explorationservice in improving employmentoutcomesUpdate:In 2017, ODDS convened a stakeholder group that consisted of self-advocates, ODDSemployment providers, VR Counselors (VRCs), Services Coordinators/PersonalAgents (SC/PAs) and education professionals to discuss what we had learned sinceimplementing Discovery and to make program improvements. The group, which metfrom 2017-2019, helped to develop the new Discovery Toolkit, including four newrequired trainings and:1. Pre-Referral Discovery Checklist2. The Discovery Profile3. ODDS Workers Guide: The Experiential Components of Discovery4. ODDS Workers Guide: Discovery Guidelines for Service Coordinators/PersonalAgents & Discovery Providers5. Sample Employment Tour Prompts & Ideas6. Home Visit QuestionsIn SFY 2017, 768 individuals completed Discovery. In SFYs 2018, 2019, 2020, and2021, 668, 575, 315, and 131 individuals, respectively, received a Discovery Profile.Moving forward these numbers should be analyzed for service utilization andevaluated against job data to determine if this service may be a predictor of communityemployment outcomes.DD-AR-21-018: Transmittals/21018.pdf14 P a g e2022 Employment First Quality Assurance Report
Improve monitoring and evaluationas individuals move betweensystems to ensure people maintainemployment and needed supports(e.g., monitor outcomes after VRcloses an individual as rehabilitatedand the individual transitions toODDS long-term employmentsupports and services).Update: In SFY 2017, 123 ODDS individuals utilized ODDS initial job coaching afterVR job placement. Of these individuals, 112 (91%) began their ODDS job coachingbefore their VR closure date, and 5 (4%) within one week of their VR closure.In SFY 2018, 118 ODDS individuals utilized ODDS initial job coaching after VR jobplacement. Of these individuals, 110 (93%) began their ODDS job coaching beforetheir VR closure date, and 4 (3%) within one week of their VR closure.In SFY 2019, 134 ODDS individuals utilized ODDS initial job coaching after VR jobplacement. Of these individuals, 129 (96%) began their ODDS job coaching beforetheir VR closure date, and 2 (1%) within one week of their VR closure.In SFY 2020, 104 ODDS individuals utilized ODDS initial job coaching after VR jobplacement. Of these individuals, 97 (93%) began their ODDS job coaching before theirVR closure date, and 3 (3%) within one week of their VR closure.In SFY 2021, 282 ODDS individuals utilized ODDS initial job coaching after VR jobplacement. Of these individuals, 242 (86%) began their ODDS job coaching beforetheir VR closure date, and 2 (1%) within one week of their VR closure.These measures clearly demonstrate efforts to create a smoother transition betweenservices has been extraordinarily successful.Table 1015 P a g e2022 Employment First Quality Assurance Report
VR Transition to ODDS GraphGraph 1016 P a g e2022 Employment First Quality Assurance Report
ODDS to VR Transition TablesTable 11Table 13Table 12Table 1417 P a g e2022 Employment First Quality Assurance Report
Services Accessed by ODDS Individuals Three Months Prior to VRApplication in SFY 2021Day or Employment ServiceNumber of Individuals per Service*Benefits Counseling1Discovery14DSA - Community65DSA - Facility6Employment Path Services Community94Employment Path Services - Facility3Individual Supported Employment100Small Group Supported Employment22Total Individuals (in Day or EmploymentServices)252*This is a distinct count, meaning the same individual can be counted in multipleservices. That is why the total individuals is smaller than the total distinct counts.Table 15GOAL 3: Monitor success of school districts in achieving integrated employment outcomes for transition studentsand monitor satisfaction of students with transition servicesStrategiesStatusAnalyze Post-School Outcomes data to identify districtsCompleted: ODE currently publishes an annual Post Schoolmeeting and not meeting state engagement targets forOutcomes Report.competitive employmentPerform qualitative analysis of Post-School Outcomes data Not Applicable: ODE tracks and analyzes the answers to thisto review responses to prompt “If you could tell your school question. Due to the identifying information found in theseone thing, what would you tell them to help better prepareanswers, it was not possible to add this item to the ODE/DHStransition students for life after high school?” (Prompt:data sharing agreement. It will therefore not be reported on.What was most helpful or could have been done better?”Table 1618 P a g e2022 Employment First Quality Assurance Report
Quality Improvement InitiativesStrategyDevelop and implement process or system that improvesour ability to track and more uniformly monitor and reporton complaints involving employment services that are filedand resolved at the local/field level across all agencies(ODE/VR/ODDS).StatusCompleted: Employment First currently tracks complaints at thestate levelTable 1719 P a g e2022 Employment First Quality Assurance Report
D. Quality of TrainingThe coordination of training and building the professional skill levels of allpeople who support individuals with I/DD obtaining integrated employmentcontinues. Through annual contributions and representation at the StatewideTransition Conference, to VR In Service coordination, SC/PA Conferencepresentations, and National APSE Conference participation, Oregon’scommitment to collaboratively improving provider training and competence isevident.Transformation Grants:Living Opportunities and Washington Initiative for Supported Employment(WISE) agencies were selected to mentor organizations committed to thetransformation process statewide through the Employment First grants. Allsheltered workshops in Oregon have closed as of September 2020.Clackamas Community College Job Coaching Certificate ProjectODDS has partnered with Clackamas Community College’s CustomizedTraining Department to develop an 80-hour certificate program to train directservice professionals and job coaches. This training is intended to create afunnel for new workers to enter our industry from Workforce and createadditional capacity for training existing job coaches already in our field.WISE Training & Technical AssistanceDHS contracted with WISE to provide technical assistance and training toexecute the Oregon Statewide Employment First Project. Oregon EmploymentLearning Network (OELN) was a series of trainings offered throughout the state,free to providers to attend in person. WISE obtained Association of CommunityRehabilitation Educators (ACRE) certification for the OELN series, adding anadditional layer of credential. DHS also contracted with WISE to develop ondemand training modules that help support people with I/DD become employedand maintain their employment. These trainings launched online in early 2021.20 P a g e2022 Employment First Quality Assurance Report
GOAL 1: Monitor progress in providing foundational Employment First training to staff and other individualsinvolved in providing employment services (to help build culture and expectation for integrated employment)StrategiesStatusCompile and analyze attendance data on trainings being offered to staff Completed: The Core Competency Modules areand individuals involved in employment service planning for individualscontained in the Oregon Department of Humanwith I/DD (“EF 101”—vision, values, history) to determine reach andServices Learning Management System. In ordercoverage of messaging.to have a training profile, users must register foraccess. This has allowed ODDS to be able toReview post-training survey data to determine impact.ensure attendance and successful completion of allonline modules as required.Ensure trainings are conducted in such a way as to provide data fortracking reach (i.e., number of attendees).Table 18GOAL 2: Evaluate effectiveness of core competency training being provided for employment professionals.StrategiesStatusMonitor outcomes achieved by individuals who attended trainingCompleted: ODDS launched the Corecompared to those who did not.Competency and Training Standards forSupported Employment Professionals forMonitor whether individuals attending training were able to demonstratestatewide training and competency reporting.competency post-training in order to attain an Association for Persons in Each module contains an assessment at theSupported Employment (APSE) or other certification.conclusion to demonstrate key concepts of themodules. A score of 90% on the assessment isIn longer term, evaluate feasibility of implementing portfolio method torequired to receive a passing score.demonstrate competency as a performance tool for employmentprofessionals cross agencies.Table 1921 P a g e2022 Employment First Quality Assurance Report
E. Quality of DataData collection, sharing and analysis allows QA Staff to monitor and evaluate activities, informs key policyand strategy decisions and quality improvement efforts, and helps monitor progress in implementing theExecutive Order.Progress toward Executive Order goals of increasing provider capabilities and client employment outcomeshave regularly been tracked in the Employment Outcomes System and reported on in the Lane v. BrownSemi-Annual Report.Quality Assurance ActivitiesGOAL 1: Create integrated, cross-agency data collection and reporting system to improve reporting on employmentoutcomesStrategiesStatusAdditional InformationEnter into data sharing agreements with Completed: DHS has data sharing agreements inShared data is reported inkey agency partners to improveplace with ODE, Oregon Employment Departmentthe Lane v Brown Semireporting on outcomes(OED) allowing the Employment First data analystAnnual Reports:to draw from multiple sources when analyzing thehttps://www.oregon.gov/dhs/outcomes for ODDS clients. ODDS and VR,EMPLOYMENT/EMPLOYMincluding Oregon Commission for the Blind (OCB),ENT-FIRST/Pages/datahave data sharing arrangements that do not require reports.aspxa formal agreement, because both programs arewithin the Oregon Department of Human Services.Revise data sharing agreements asCompleted: Data sharing agreements continue toneeded to continuously improve thebe updated as needed.quality of the available dataTable 2022 P a g e2022 Employment First Quality Assurance Report
GOAL 2: Collaborate with ODDS to Improve reporting and available data on ODDS employment service planning,utilization, and
2022 Employment First Quality Assurance Report Quality Improvement Initiatives Strategy Results Work within DHS (ODDS, VR and DD Licensing) to evaluate existing requirements for provider quality assurance programs to determine whether changes are needed to standards, monitoring, reporting, and other requirements to support implementation of the
