Transcription
Student–Teacher LinkageVerification: Model Processand RecommendationsMarch 2012Jeffery WatsonUniversity of WisconsinMatthew GrahamWestatChristopher A. ThornUniversity of Wisconsin
The Center for Educator Compensation Reform (CECR) would like to thank the following peoplefor their time in reviewing drafts of this resource: Jenna Aurand Scott, Ph.D., Westat; Shen Lee,Westat; Carolyn Lampila, U.S. Department of Education.The work described in this paper was supported by the U.S. Department of Education through theCenter for Educator Compensation Reform. The opinions expressed are those of the authors anddo not necessarily reflect the view of the U.S. Department of Education, the Center for EducatorCompensation Reform, or the institutional partners of the Center. Comments and suggestionsare welcome.The Center for Educator Compensation and Reform (CECR) was awarded to Westat —in partnership with Learning Point Associates, an affiliate of American Institutes for Research;Synergy Enterprises, Inc.; J. Koppich and Associates; and the University of Wisconsin—by the U.S.Department of Education in October 2006.The primary purpose of CECR is to support Teacher Incentive Fund (TIF) grantees in theirimplementation efforts through provision of sustained technical assistance and development anddissemination of timely resources. The Department has also charged CECR with raising nationalawareness of alternative and effective strategies for educator compensation through a newsletter,a Web-based clearinghouse, and other outreach activities.This work was originally produced in whole or in part by the CECR with funds from the U.S.Department of Education under contract number ED-06-CO-0110. The content does not necessarilyreflect the position or policy of CECR or the Department of Education, nor does mention or visualrepresentation of trade names, commercial products, or organizations imply endorsement by CECRor the federal government.Allison Henderson, DirectorPhone: 888-202-1513E-mail: cecr@westat.com35011.0312.83670507
Student–Teacher Linkage Verification:Model Process and RecommendationsINTRODUCTION. 4CONCLUSION. 18I. PLANNING . 5APPENDIX A: USE-CASES FOR LINKAGEVERIFICATION. 20Determine what the verificationsystem must do. 5Actor/Goal Table (Use-Case Index). 20Engage stakeholders who knowdata systems best. 5Teacher Login. 21Balance the desire for nuanced data againstthe need for stakeholder buy-in. 6Verify Class Information. 23Train, support, and communicatewith educators. 7Add Student to Class. 25II. CONFIGURATION AND TESTING. 8Ensure that educators’ computers can runthe software. 8Pilot and improve the system. 8III. LINKAGE VERIFICATION. 9Step 1: Administrator reviews teacher rosters. 10Teacher Screening. 22Verify Class Roster. 24Add New Student. 25Principal Login. 26Unclaimed Course Resolution. 27Unclaimed Student Resolution. 28Unclaimed Student Resolution. 29REFERENCE. 30Step 2: Teachers review class rosters. 11Step 3: Administrators review changes, resolveconflicts, and account for unassignedstudents. 13Use data and educator feedback to improvethe verification process. 16Integrate quality control into district plansand staffing. 18Student–Teacher Linkage Verification: Model Process and Recommendations3
INTRODUCTIONAs momentum grows for tracking the role ofindividual educators in student performance,school districts across the country are implementingprojects that involve linking teachers to theirstudents. Federal programs like Race to the Topand the Teacher Incentive Fund (TIF) contributeto this trend by requiring participants to implementeducator evaluation systems that use studentperformance as one of the main components.Programs that link teachers to student outcomesrequire a verification process for student–teacherlinkages. Linkage verification improves accuracyby allowing educators to check for and correctstudent and course assignment errors. Without aneffective verification process, data errors are muchmore likely to affect the validity of performanceassessments, program evaluations, educationalresearch, professional development decisions, andany other initiative that uses student–teacher data.The culmination of an appropriately implementeddesign, collection, and verification process willproduce highly accurate links, although other factorssuch as teacher absences or team teaching may stillthreaten the validity of those links.This paper describes a model process for gatheringand verifying student–teacher linkage data andoffers recommendations at each stage. AppendixA contains use-case2 specifications that correspondto the model verification process. The use-casespecifications present more detailed steps that maybe useful to readers interested in designing their ownprocess or contracting for verification services.In order to accurately link students, teachers, andcourses, district leaders must first determine whattypes of links they want to make and the extent towhich their current data systems support such links.Typically, districts did not design their student datasystem to seamlessly and accurately link students,teachers, and courses; therefore, the systems mayrequire substantial upgrades.1 Once the studentinformation system (SIS) generates preliminarylinks, often by combining data from multiplesources, educators must verify those links.The model verification process begins with aplanning stage, followed by configuration andtesting. Once these steps are complete, teachersand administrators can begin verifying data.During verification, administrators and teachersconfirm or modify classroom assignments usingcomputer software or a web-based application.Verification takes place in three main steps: aninitial administrator review, a teacher review, anda final administrator check. In addition, users canenter additional data during verification, such asthe percentage of instructional responsibility forteam teachers. After teachers and administratorsverify the data, districts should gather and reportinformation about the linkage verification process.Districts should use this information to continuouslyimprove the accuracy and quality of their student–teacher links.1 Many of the necessary changes are encouraged by programs like Raceto the Top.2 A use-case document is a detailed diagram that depicts the processthrough which users of a system achieve a goal. For a more detailedexplanation, see Appendix A.Student–Teacher Linkage Verification: Model Process and Recommendations4
I. PLANNINGSince student–teacher linkages should be asaccurate as possible, careful planning and testingof any verification system should take place beforefull implementation. Otherwise, districts maynot identify flaws in the verification process untilafter they have used inaccurate data in high-stakesdecisions. In the case of a performance-basedcompensation system, for example, inaccurate datacould result in unfair rewards or punishments forteachers, which in turn could erode support amongeducators and the broader public.This section contains several recommendationsthat districts should consider when designing andplanning an electronic student–teacher linkageverification system. As district managers determinewhat features the verification system must contain,they should engage the stakeholders who knowthe data best. Districts should train teachers andadministrators on how to use the system and shouldprovide readily accessible support to educatorsduring the verification process itself.Determine what the verificationsystem must doThe first planning step is to determine whatfeatures the verification system must contain. Forexample, districts should decide which coursesrequire linkage verification, when and how oftenverification should occur, and other features thatdepend on project details. After districts determinethe verification system’s required capabilities,programmers can begin designing software thatmeets those specifications.Many verification system features, such as thestage where teachers certify that their class rostersare accurate, will be common to all verificationsystems. Other potential features will depend on thecapabilities of district data systems. If a district wantsto know something about classrooms that the SISdoes not currently capture, the district may eitherupgrade the SIS or add the data during verification.For example, if an SIS does not contain detaileddata on co-teachers, a district will have to choosebetween (1) adding co-teacher data to the SIS or (2)requiring teachers to enter those data ad hoc usingverification software.Engage stakeholders who knowdata systems bestLinking students and teachers often requiresdistricts to integrate data from multiple informationsystems. However, because districts did not designtheir information systems to link among students,teachers, and courses, each district data systemmay present its own unique set of challenges. Forexample, in one school district’s human resourcesdata system, 25 percent of teachers had inaccuratedata (such as employee identification numbers orincomplete names) before the district began tomanage its data quality. Another common challengeis that districts often do not manage courses forelementary grades because those classrooms areconsidered self-contained.Districts need to identify and overcome datalimitations with the assistance of personnel whoknow the data best. District stakeholders such asdepartmental middle managers, technical staff, andother school-level clerical personnel who regularlyuse the data are likely to be familiar with datalimitations and may also know of ways to mitigatethose limitations. Districts should take advantageof this knowledge as they plan to integrate new tasksand new data quality requirements into their use ofinformation systems.Student–Teacher Linkage Verification: Model Process and Recommendations5
Balance the desire for nuanced data againstthe need for stakeholder buy-inAlthough districts should strive to obtain accurateand appropriately detailed student–teacherlinkage data, they should also remember that userexperiences affect opinions of the verificationprocess. Districts must balance the desirabilityof collecting nuanced data against the need tomaximize stakeholder buy-in and participation,as well as the need to control administrative costs.Collecting the most detailed data possible is notalways feasible because in many cases, doing sowould make the verification process too greata burden for educators. For example, districtleaders may wish to gather highly specific data onthe percentage of instruction delivered by supportspecialists and resource teachers. However, ifinputting these data makes the process too longand difficult, educators may become frustrated withverification and be less likely to participate. In aperformance-based compensation system, drops ineducator support and participation could jeopardizethe whole initiative by increasing the likelihoodof high-profile errors. Although it is important tocollect nuanced data, districts have to balance thatneed against the need to win stakeholder support.One way to add nuanced data while managingproject complexity is to gradually increase the degreeof detail collected during verification. After teachershave become familiar with the verification processand district managers have had a chance to work outother potential problems or inefficiencies, managerscan establish procedures to collect more nuanceddata. Such a strategy would mitigate the risk ofoverwhelming educators in the first few years, wheninexperience with verification increases the risk oferror and inefficiency.District leaders should also be aware that theverification process will affect stakeholder buy-in.During the pilot verification and once the systemis implemented, district staff should monitoreducators’ experiences. Districts should collectvarious pieces of information such as data on systemresponsiveness and the number of calls to the helpdesk and should take appropriate action to addressstakeholder concerns.Student–Teacher Linkage Verification: Model Process and Recommendations6
Train, support, and communicatewith educatorsTeachers and administrators face many demandson their time, and linkage verification adds to thatburden. The estimated time to complete for themodel linkage process outlined below is 10 to 20minutes for individual teachers and about fivehours for principals at the elementary and middleschool levels. As such, district leaders should clearlycommunicate with teachers and administratorsregarding the purpose of the verification processand the importance of data quality for any highstakes use. When teachers see that (1) the districtuses the data to determine potentially high-stakesdecisions, and (2) the district wants teacher inputon data quality, teachers will be more likely to trustthe program’s outcomes measures. Dist
for their time in reviewing drafts of this resource: Jenna Aurand Scott, Ph.D., Westat; Shen Lee, Westat; Carolyn Lampila, U.S. Department of Education. The work described in this paper was supported by the U.S. Department of Education through the Center for Educator Compensation Reform. The opinions expressed are those of the authors and do not necessarily reflect the view of the U.S .
