Transcription
Employment FirstQuality Assurance/ QualityImprovement UpdateJuly 2015Erinn Kelley-Siel, DirectorDepartment of HumanServicesKate BrownGovernorPrepared byEmployment First
ContentsExecutive Summary . 3Introduction . 5Purpose . . 5Methodology . 6Employment First Quality Assurance/Quality Improvement Focal Areas . 7Quality of Planning . 7A. Quality Assurance Activities . 81. Goal 1, Strategy 1: Timeliness of Service Planning . 82.VR Application to Eligibility (Graph 1) . 103.VR Eligibility to Plan (Graph 2) . 114.VR Application to Employment Outcome (Graph 3) . 135.Goal 1, Strategy 2: Enhance Employment-Related Aspects of ODDS QA Field Reviews . 146.Goal 1, Strategy 3: VR Program Effectiveness. 157.Hours Worked Per Week at VR Closure (Graph 4) . 168.Goal 1, Strategy 4: ODE Data Related to Employment Service Planning . 171.ODE Job Location (Table 1) . 192.ODE Graphs (Graph 5-8) . 203.Goal 2, Strategies 1 and 2: Checklist/Procedures for Qualitative ODDS File Review . 214.Goal 2, Strategy 3: Monitor and Evaluate Effectiveness of New Career Development Plan . 215.Goal 2, Strategy 4: Analyze Effectiveness of Specially-Trained VR Counselors . 226.Goal 2, Strategy 5: Monitor ODDS Time-Limited Services . 22B. Quality Improvement Initiatives . 221. ODE Summary of Performance (content) . 222.ODE Summary of Performance (utilization) . 23Quality of Providers. 23C. Quality Assurance Activities . 241. Goal 1, Strategy 1: Core Competency-Based Qualifications for ODDS Providers . 242.Goal 2, Strategy 1: Certification and Field Review Process . 253.Goal 2, Strategy 2: Enhanced Monitoring Strategies for Providers . 264.Goal 3, Strategy 1: VR Job Development Vendor Report Card. 265. Goal 3, Strategy 2: Performance-Based Standards for ODDS Employment ServiceProviders. 266.Goal 3, Strategy 3: Performance of School Leavers by School District . 27D. Quality Improvement Initiatives . 271. Provider Quality Assurance Programs; State Template for Data. 271Employment First July 2015 Oregon Department of Human Services
2.Mentoring Program for New Employment Specialists . 27Quality of Services . 28E. Quality Assurance Activities . 281. Goal 1, Strategy 1: Established Program Outcomes and Targets . 282.Goal 1, Strategy 2: Monitor and Evaluate Timeliness of Service Delivery . 303.Goal 1, Strategy 3: Additional Outcome Measures for Service Quality. 304.Goal 1, Strategy 4: QA Measures Related to HCBS Settings Rules . 305.Goal 2, Strategies 1 and 2: Checklist/Procedures for Qualitative ODDS File Review . 316.Goal 2, Strategy 3: Effectiveness of New Discovery/Career Exploration Service . 317.Goal 2, Strategy 4: Transition Between Programs . 338.Goal 3, Strategy 1: School District Level Results . 349.Goal 3, Strategy 2: Qualitative Analysis of Post-School Outcomes Data. 34F. Quality Improvement Initiatives . 351. Complaints Involving Employment Services . 35Quality of Training . 35G. Quality Assurance Activities . 361. Goal 1, Strategies 1, 2 and 3: Foundational Employment First Training . 362.Goal 2, Strategies 1, 2 and 3: Effectiveness of Training; Demonstration of Competency . 37H. Quality Improvement Initiatives . 381. Communities of Practice. 382.Funding for Training . 38Quality of Data . 39I. Quality Assurance Activities . 391. Goal 1, Strategy 1: Data Sharing Agreements with Agency Partners . 392.Goal 1, Strategy 2: Data Quality. 403.Goal 2, Strategies 1 and 2: Data Collection and Reporting from ODDS Systems . 401.Quality Improvement Initiatives . 40Post-School Outcomes Survey (sample size) . 40J.Quality of Life . 41K. Quality Assurance Activities . 411. Goal 1, Strategy 1: Quality of Life Indicators from National Core Indicators Survey . 41L. Quality Improvement Initiatives . 421. NCI Survey Data . 42Conclusion. 43Appendix A- Outcomes by VR Vendor (Table 2). 442Employment First July 2015 Oregon Department of Human Services
Executive SummaryThe purpose of this report is to present an update on implementation of the qualityassurance and quality improvement strategies identified in the Employment FirstQuality Assurance/ Quality Improvement Plan (QA Plan) issued July 2014.These strategies are organized under the focal areas of Quality of Planning, Quality ofProviders, Quality of Services, Quality of Training, Quality of Data, and Quality ofLife. While it was not possible to implement every item, multiple strategies in eachfocal area are accomplished or underway.Many strategies involve measures developed by a cross-agency working team. Thisreport sets the baseline for these measures. Future analysis will demonstrate Oregon’sperformance in each area. The results for State Fiscal Year 2015 include:91% of Oregon Department of Developmental Disability Services (ODDS)individuals at Vocational Rehabilitation (VR) received a VR eligibilitydetermination within 60 days;The average hours worked for ODDS individuals successfully closed throughVR were 16.6 hours per week;The average wage for ODDS individuals successfully closed through VR was 9.55 per hour; and65% of Discovery recipients had an open case in VR at some point in the fiscalyear.This report also includes Post School Outcomes data from the Oregon Department ofEducation. These results for 2013 ODDS school leavers include:Of respondents that worked, 81.6% worked in non-sheltered settings; andOf respondents that worked, 64.5% made at least minimum wage.In order to show performance over time, this report includes results from theEmployment First Data Report. That report has established metrics, which show stronggrowth in VR outcomes for ODDS individuals, including:An annual increase of 7.7% in applications to VR;An annual increase of 61.6% for individuals entering an Individualized Plan forEmployment; and3Employment First July 2015 Oregon Department of Human Services
An annual increase of 40.3% for individuals successfully closing withcompetitive integrated employment.This QA Update shows a great deal of collaborative effort by multiple state agencies tomove Employment First quality assurance and quality improvement efforts forward. Italso shows that outcomes are improving for many who are seeking employmentservices and competitive integrated jobs.4Employment First July 2015 Oregon Department of Human Services
IntroductionPolicy and procedure changes at the state and federal level make Employment First inOregon a rapidly evolving landscape. These changes include new WorkforceInnovation and Opportunity Act (WIOA)1 requirements for Vocational Rehabilitation(VR), the Oregon Department of Education (ODE), and other partners. These changesalso include new regulatory requirements for Home and Community Based Servicesadministered through the Office of Developmental Disability Services (ODDS);2 aswell as new state policies, such as no new entries to sheltered workshops as of July 1,2015.3 While VR, ODDS, and ODE address and implement these large changes, theycontinue to work together towards the goal of Employment First for people withintellectual and developmental disabilities in Oregon. Quality assurance and qualityimprovement play an important role in ensuring the success and sustainability of theseefforts.Purpose .The purpose of this report is to present an update on implementation of the qualityassurance and quality improvement strategies identified in the Employment FirstQuality Assurance/ Quality Improvement Plan (QA Plan) issued in July 2014. Thisreport is also intended to present the collaborative and independent work taking placeacross agencies to ensure the success of Oregon’s Employment First initiative.The 2014 QA plan is ambitious, and it was not possible to implement some portions ofthe plan by the phase 1, 2 and 3 implementation dates in the original plan. VR, ODDS,and ODE prioritized certain strategies based on the changing needs of the system;therefore, not every strategy in the 2014 QA Plan has been set in motion. All theThe Workforce Innovation and Opportunity Act (WIOA) makes important changes to theRehabilitation Act of 1973 and other federal laws. WIOA is designed to help job seekers accessemployment, education, training, and support services to succeed in the labor market. WIOA placesincreased emphasis on serving students and youth with disabilities, with the goal of better preparingthem for competitive integrated employment.2HCBS service settings must be “integrated in and [support] full access of individuals receivingMedicaid HCBS to the greater community, including opportunities to seek employment and work incompetitive integrated settings.” 42 CFR 441.301(c)(4)(i).3In order to further Oregon’s Employment First policy, effective July 1, 2015, ODDS will no longerfund Sheltered Work or services in a Sheltered Workshop setting, for individuals who are newlyeligible for services, or individuals already using services but not in a Sheltered Workshop setting.See the Governor’s Executive Order 15-01 and Oregon Administrative Rule 407-025-0020(2).15Employment First July 2015 Oregon Department of Human Services
strategies are included and reported on in this document, regardless of theirimplementation status. Oregon is committed to and is engaged in developing a robustQA/QI system that reflects how systems and programs are performing together to meetgoals and objectives, and that informs policy and strategic planning going forward. TheQA Plan will change and evolve over time as implementation efforts continue.MethodologyThe Employment First QA/QI analyst worked with the cross agency working team thatdeveloped the 2014 plan, to decide which items to prioritize first in regard todeveloping and reporting data. This team is comprised of representatives from VR,ODDS, ODE and the Oregon Council on Developmental Disabilities. The groupselected goals and strategies that involved VR services, the new Discovery service,4transition between services at ODDS and VR, and time frames for service planning anddelivery. Measures for these areas were then developed in collaboration with theEmployment First and VR data analysts.There have been major changes in the data systems at ODDS that will improve theaccuracy of the data reported. In September 2014, ODDS employment services datamoved to the revised eXPRS billing system, and reflects new service definitions. Thiswill allow for more accurate tracking of specific services provided by ODDS.Furthermore, the Employment Outcomes System (EOS), which provides provider-leveldata, was revised to reflect the new service definitions. The revised EOS was launchedin March 2015.ODDS and VR successfully pulled data for at least one item in each area prioritized bythe cross agency working team for this report. These ‘first pulls’ are setting thebaseline for these measures. This means we can perform analysis in future reports,demonstrating Oregon’s performance in each area. Data for state fiscal year (SFY)2015 in this report is subject to upward revisions in future reports, because all ODDSbilling data and VR data is not yet complete for SFY 2015.Discovery is a new short-term ODDS employment service that provides exploration of employmentthrough work experiences, informal interviews, and other activities designed to explore theindividual’s interests and strengths. More information regarding the Discovery service can be foundin the Discovery scovery%20FAQ.pdf46Employment First July 2015 Oregon Department of Human Services
For purposes of showing performance of the system over time, this report referencesthe annual Employment First Data Report. That report includes established metrics,showing performance over time. Discussion of these results is on page 29.Employment First Quality Assurance/Quality Improvement Focal AreasEmployment First uses the following structural framework to highlight the qualityassurance and quality improvement focal areas for this initiative: Quality of Planning,Quality of Providers, Quality of Services, Quality of Training, Quality of Data, andQuality of Life. Consistent with national trends,5 Oregon’s system focuses on qualityoutcomes with the ultimate goal of improving the employment outcomes forindividuals with intellectual and developmental disabilities.Using the above structural framework, this update will detail the current status of theQuality Assurance Strategies and Quality Improvement Initiatives originally publishedin the 2014 Quality Assurance Plan.Quality of PlanningIf we are successful in achieving quality in this area, what does that success looklike?Service planning documents across ODDS, VR and ODE reflect the needsand desires of individual; planning includes discussion anddocumentation around employment; plans across agencies reflect thesame outcome and are coordinated and complementary. Success is morethan just planning for and delivering the needed services. Quality ofPlanning is ultimately demonstrated by the outcomes individuals achieve.Key questions that will guide our work: Do plans, and does the planning process, meaningfully address employmentservices and reflect the expectation of integrated employment? Is planning and the documentation generated as consistent as possible acrossagencies to minimize the time the individual has to spend “starting over” orduplicating information or activities?See “A Review of Current Trends in Quality Assurance of Services Provided to Persons withDevelopmental Disabilities,” by Gary L. Siegel, Ph.D, Institute of Applied Research, St. Louis,Missouri, July 2000.57Employment First July 2015 Oregon Department of Human Services
What additional documentation would be helpful to ensure seamless transitionbetween programs? What monitoring system(s) will we rely on for oversight of planningdocuments? Do we provide our services in as timely a manner as possible across all servicesegments?A.Quality Assurance ActivitiesGoal 1: Continue to monitor and evaluate employment service planning across allagencies to assure informed choice and compliance with integrated employmentservice expectations1.Goal 1, Strategy 1: Timeliness of Service PlanningStrategy: Continue to monitor and evaluate timeliness of service planning across allprograms (ODDS, VR, ODE).Update: The cross agency working team brainstormed key time frames to monitor.Questions guiding this discussion were: What are the key timeframes to monitorrelating to employment service planning across the involved programs? How long areemployment outcomes maintained? The time frames the team decided to focus on firstwere:VR:Application to eligibility;Eligibility to plan;Application to employment outcome.ODDS:Length of time in employment path community and employment path facility;Length of time employed after transfer to ODDS employment supports.Next, the Employment First QA Analyst assessed the availability of data for the July2015 QA update with the Employment First and VR data analysts. For this report, datawas available for the VR time frames. For the ODDS time frames, data methodologywill be developed and reported on in future reports.8Employment First July 2015 Oregon Department of Human Services
Why are these identified time frames important measures to focus on for VR? Failingto provide services in a timely manner can be a barrier to ODDS clients accessing VRservices, and can ultimately affect the quality of the individual outcomes achieved andthe individual’s satisfaction with those services. For instance, the 2013 Oregon VR andState Rehabilitation Council Comprehensive Statewide Needs Assessment surveyshowed that both clients and VR staff believed that timeliness of service is a highpriority in improving employment outcomes. 6Application to EligibilityWhy is time from application to eligibility important to measure? Congress’sestablishment of the 60-day timeframe for VR application to eligibility was based onits intention that “eligible individuals should receive Vocational Rehabilitation servicesexpeditiously.”7 There are two presumptions that support this intention and help ODDSindividuals gain access to services quickly. One is a presumption of eligibility forSocial Security recipients and beneficiaries.8 The other is a presumption that eligibleindividuals can benefit from an employment outcome.9 These presumptions underscorethe intention that VR complete eligibility determinations quickly.One important source of feedback on VR customer satisfaction: the 2013 Oregon VocationalRehabilitation and State Rehabilitation Council Comprehensive Statewide Needs Assessment (the“VR Survey”), available f7RSA Policy Guidance Document on Vocational Rehabilitation, "Eligibility for VocationalRehabilitation Services Under Title I of the Rehabilitation Act of 1973, as Amended" (RSA IM-9317) (May 11, 1993). Many of the changes outlined in this Informational Memorandum were designedto “streamline the eligibility determination process and to enhance the access of individuals tovocational rehabilitation services,” both themes that continue today with the Workforce Innovationand Opportunity Act.834 CFR 361.42(a)(3); see also OAR 582-050-0020 (6).934 CFR 361.42(a)(2); see also OAR 582-050-0020 (5).69Employment First July 2015 Oregon Department of Human Services
2.VR Application to Eligibility (Graph 1)Graph 1The data in Graph 1 above will help to determine whether federal requirements fortimeliness of service delivery (VR application to eligibility) are being met. This datawill also help show how quickly ODDS individuals access VR services.Results:10 Federal regulations require VR to complete the eligibility determinationwithin 60 days of application. Exceptions are made for exceptional and unforeseencircumstances, extended evaluations, and trial work periods. 11 Of the 1,053 ODDSindividuals that closed in SFY 2015, 957 (91%) were given an eligibilitydetermination within the required time frame. This percentage does not includeindividuals who were granted extensions under the above exceptions. The total averagedays from application to eligibility were 38.1011All VR data published in this report represents ODDS clients at VR, not the general VR population.34 CFR 361.41 (b)(1).10Employment First July 2015 Oregon Department of Human Services
Eligibility to PlanWhy is the VR eligibility-to-plan timeframe another important measure to focus on forVR? WIOA provides that “[t]he individualized plan for employment must bedeveloped as soon as possible, but not later than 90 days after the date ofdetermination of eligibility . . . ”12 This measure will help to monitor whether legalrequirements for service delivery are met. It will also help programs to evaluatewhether changes are needed to existing systems in order to ensure ODDS clients areable to access VR services in a timely manner.3.VR Eligibility to Plan (Graph 2)Graph 2“Workforce Innovation and Opportunity Act” (PL 113-128, July 22, 2014), Section 413(b)(4)(B)(emphasis added).1211Employment First July 2015 Oregon Department of Human Services
Results: New proposed WIOA regulations require VR to complete the individualizedplan for employment within 90 days of eligibility.13 This requirement went into effecton January 1, 2015 for any applications submitted after that date (not all eligible casesawaiting plan after that date). Therefore, data pulled by plan date would not accuratelyshow results under each requirement. We report instead on the general results of theentire state fiscal year.Of the 1,097 ODDS individuals that closed in VR in state fiscal year 2015, 681 entereda VR Individualized Plan for Employment (IPE) at some point before closure. Of theseindividuals, 341 (50%) entered plan within 90 days of their eligibility determination,and 507 (74%) entered plan within 180 days of eligibility. The average was 140 days.Graph 2 shows the eligibility to plan results in 100 day increments. Reasonsindividuals fell after the prior 180 day requirement included mutual agreement by theclient and counselor, unforeseen circumstances, and lack of agreement over theproposed IPE, among other allowable reasons. These can be seen in OregonAdministrative Rule (OAR) 582-050-0000.Application to RehabilitationRehabilitation at VR means a successful closure: a successful employment outcomethat is maintained for at least 90 days (for “an appropriate period of time, but not lessthan 90 days, necessary to ensure the stability of the employment outcome ” 34 CFR361.56.) For the rest of this report, rehabilitation will be called ‘employmentoutcome’.Why is the timeframe from application to employment outcome important to focus onfor VR? Timely services and obtaining a job promptly after applying for VR servicescan be important for a successful career. According to a study by the Institute forCommunity Inclusion, getting a job quickly “boosts self-confidence and preventsapplicants from losing work skills as a consequence of inactivity. Moreover, employersmay prefer candidates whose work history shows limited gaps in employment.”14 TheProposed 34 CFR 361.45(e), found on page 21062 of the proposed rule titled “State VocationalRehabilitation Services Program; State Supported Employment Services Program; Limitations on Useof Subminimum Wage”, posted 4/16/2015 on the Federal Register (www.federalregister.gov).14Migliore, A. & Smith, F.A., 2007. “Timeframe from Application to Closure in IntegratedEmployment for Vocational Rehabilitation Customers with Developmental Disabilities” (Data Note14). University of Massachusetts Boston, Institute for Community Inclusion.1312Employment First July 2015 Oregon Department of Human Services
data in Graph 3 below will help track the overall time frame for successful outcomesfor ODDS individuals, which can be compared to outcomes for all non-ODDSindividuals.4.VR Application to Employment Outcome (Graph 3)Graph 3Results: Of 432 ODDS individuals closed with an employment outcome in state fiscalyear 2015, 139 (33%) reached that outcome within 300 days, and 299 (69%) within600 days. For number of days from application to employment outcome, the averagefor ODDS individuals was 514 days (17 months). As a comparison, the average fornon-ODDS individuals closed with an employment outcome in state fiscal year 2015was 516 days, showing almost no difference in this measure for ODDS individuals.13Employment First July 2015 Oregon Department of Human Services
The activities that occur during this time period include eligibility determination,development and implementation of the IPE,15 achieving the employment outcome,and maintaining the outcome for at least 90 days. This means the results include atleast 90 days on the job. Services that occur before closure may include jobdevelopment and placement, job coaching, supported employment, a school or trainingprogram, transition services, and other possibilities based on the needs and goals of theindividual.16It is also important to note that “the time elapsed between application and closure is notnecessarily a measure of individual success in employment outcomes. Applicants whoare students may delay looking for employment until after completing secondary orpostsecondary education. Moreover, some applicants may receive services beyond the90th day of employment, extending the date of case closure in comparison to jobentry.”175.Goal 1, Strategy 2: Enhance Employment-Related Aspects ofODDS QA Field ReviewsStrategy: Work with ODDS QA Staff to enhance the monitoring of the employmentrelated aspects of the current ODDS QA Field Reviews.Update: This item was accomplished with implementation of the new employment tabof the Developmental Disabilities Quality Assurance Field Review tool.18 This tab asksfive new questions related to the employment goal and Career Development Plan(CDP) of the individual file being reviewed. The questions are designed to determine ifconversations are occurring regarding employ
4 Employment First July 2015 Oregon Department of Human Services An annual increase of 40.3% for individuals successfully closing with competitive integrated employment. This QA Update shows a great deal of collaborative effort by multiple state agencies to move Employment First quality assurance and quality improvement efforts forward.
