WIXLIB

What Works Clearinghouse U.S. DEPARTMENT OF EDUCATIONWWC Intervention ReportA summary of findings from a systematic review of the evidenceTeacher Training, Evaluation, and CompensationTAP : The System forTeacher and StudentAdvancementProgram Description1TAP : The System for Teacher and Student Advancement(formerly known as the Teacher Advancement Program) is a comprehensive educator effectiveness program that aims to improvestudent achievement through supports and incentives that attract,retain, develop, and motivate effective teachers. The programprovides teachers with leadership opportunities and associatedsalary increases; ongoing, school-based professional development;rigorous evaluations; and annual performance bonuses based ona combination of teacher value added to student achievement andobservations of their classroom teaching.Research2The What Works Clearinghouse (WWC) identified one study of TAP that both falls within the scope of the Teacher Training, Evaluation,and Compensation topic area and meets WWC group designstandards. This one study meets WWC group design standards withreservations. The study included 7,661 students in grades 4–8 in34 Chicago elementary (grades K–8) schools.July 2015Report ContentsOverviewp. 1Program Informationp. 3Research Summaryp. 5Effectiveness Summaryp. 6Referencesp. 8Research Details for Each Studyp. 13Outcome Measures forEach Domainp. 15Findings Included in the Ratingfor Each Outcome Domainp. 16Supplemental Findings forEach Outcome Domainp. 18Endnotesp. 20Rating Criteriap. 22Glossary of Termsp. 23This intervention report presents findingsfrom a systematic review of TAP conducted using the WWC Proceduresand Standards Handbook, version 3.0,and the Teacher Training, Evaluation,and Compensation review protocol,version 3.1.The WWC considers the extent of evidence for having a teacher in a TAP -implementing school on the academicachievement of students in grades 4–8 to be small for three student outcome domains—science achievement,English language arts achievement, and mathematics achievement. There were no studies that meet WWC designstandards in the three other student outcome domains and the six teacher outcome domains, so this interventionreport does not report on the effectiveness of TAP for those domains. (See the Effectiveness Summary on p. 6for more details of effectiveness by domain.)EffectivenessHaving a teacher in a TAP -implementing school was found to have no discernible effects on scienceachievement for students in grades 4 and 7 or on English language arts and mathematics achievementfor students in grades 4–8.TAP : The System for Teacher and Student Advancement July 2015Page 1

WWC Intervention ReportTable 1. Summary of findings4,5Improvement index (percentile points)Rating of effectivenessAverageRangeNumber ofstudiesNumber ofstudentsExtent ofevidenceScienceachievementNo discernible effects 5na11,717SmallEnglish languagearts achievementNo discernible effects0na17,661SmallMathematicsachievementNo discernible effects–1na17,656SmallOutcome domainna not applicableTAP : The System for Teacher and Student Advancement July 2015Page 2

WWC Intervention ReportProgram InformationBackgroundLowell Milken and educational experts at the Milken Family Foundation established TAP in 1999. The programis managed by the National Institute for Excellence in Teaching (NIET). The organization’s address is 1250 FourthStreet, Santa Monica, CA 90401. Web: www.niet.org and www.tapsystem.org. Telephone: 310-570-4860.Program detailsNIET identifies clusters of potential TAP schools through its partnerships with districts, states, and universities.Identified schools that are interested in adopting TAP must submit an application, and NIET selects those applicantschools that demonstrate the capacity to implement TAP , the ability to fund the system, and strong faculty support(that is, approved through a faculty vote). Once a school is selected to implement TAP , leaders from those schoolsreceive NIET training, materials, and tools to implement all four of the program’s core elements. NIET also works withits TAP partner schools to obtain appropriate funding for services and to sustain TAP ’s core elements.The four core elements of TAP are: Multiple career paths. Schools implementing TAP create new opportunities for high-performing teachersto take on additional leadership responsibilities by becoming mentor or master teachers. These individualsserve together with principals and assistant principals on a TAP Leadership Team, which is responsiblefor participating in trainings, analyzing student data, setting student learning goals and achievement plans,evaluating traditional teachers (called “career teachers”), and providing individual and team coaching. Masterteachers also lead professional learning communities (called “cluster groups”) and provide oversight to mentor teachers, who in turn provide additional support to career teachers. Mentor and master teachers receiverelease time and additional compensation to perform their leadership duties. TAP currently recommendsannual salary augmentations of 5,000 to 8,000 for mentor teachers and 8,000 to 12,000 for master teachers, depending on local budgets. Ongoing applied professional growth. Teachers in TAP schools meet for 1 hour per week in grade- or subject-based groups led by mentor or master teachers. This collaborative mentoring and planning time is intendedto help teachers learn research-based instructional strategies to meet the specific needs of their students. Mentor and master teachers also provide career teachers with individual, classroom-based support through activitiessuch as demonstrating lessons, team-teaching, conducting observations, and providing feedback. Instructionally-focused accountability. Teachers in TAP schools are evaluated using three measures:classroom achievement growth, classroom observations, and school-wide achievement growth (or the latter twomeasures for teachers in non-tested subjects and grades). NIET trains and certifies members of the TAP Leadership Team to conduct observations using the TAP Teaching Skills, Knowledge, and ResponsibilitiesPerformance Standards. Each teacher is formally evaluated four times per year and receives feedback andcoaching in a post-conference following each observation. Student achievement growth is assessed usingteachers’ grade- and subject-specific value-added scores derived by students’ average growth trajectoriesper their state’s standardized test. Performance-based compensation. NIET expects TAP schools to develop a performance award pool ofabout 2,000 to 3,000 per teacher for use as bonuses to the teachers who attain a minimum score on theirevaluations. TAP schools also have the option to offer enhanced compensation to principals based on alocally-determined bonus structure. NIET can provide guidance on measures that can be used to determine aprincipal’s eligibility for and amount of compensation (e.g., school-wide achievement growth, the TAP Leadership Team Rubric, and other valid and reliable 360 instruments).TAP : The System for Teacher and Student Advancement July 2015Page 3

WWC Intervention ReportCostFor a typical school of 25 teachers and 600 students, the cost of TAP implementation is about 250 per student.Implementation costs include onsite and online support, the cost of a master teacher, salary stipends for otherteacher leaders in the school, and bonuses. However, the cost per student may vary depending on the local cost ofliving, student/teacher ratios, and whether existing infrastructure can be leveraged (e.g., reading coaches who canbecome master teachers). Some districts use local or federal funds (Title I, Title II, or School Improvement Grants)to cover TAP costs. NIET can assist schools in identifying ways to leverage existing resources and local or federalfunds to reduce the cost of TAP .TAP : The System for Teacher and Student Advancement July 2015Page 4

WWC Intervention ReportResearch SummaryThe WWC identified nine eligible studies that investigated the effects ofTAP teachers on academic achievement for students in grades 4–8.An additional 27 studies were identified but do not meet WWC eligibilitycriteria for review in this topic area. Citations for all 36 studies are in theReferences section, which begins on p. 8.Table 2. Scope of reviewed researchGrade4–8Delivery methodWhole schoolProgram typeTeacher levelThe WWC reviewed nine eligible studies against group design standards.One study (Glazerman & Seifullah, 2012) uses two designs to answer research questions: a cluster randomizedcontrolled trial and a quasi-experimental design. Both designs meet WWC group design standards with reservations.The study is summarized in this report. Eight studies do not meet WWC group design standards.Summary of studies meeting WWC group design standards without reservationsNo studies of TAP met WWC group design standards without reservations.Summary of study meeting WWC group design standards with reservationsGlazerman and Seifullah (2012) presented the implementation and impact findings resulting from Chicago PublicSchools’ phased roll-out of TAP (called “Chicago TAP”) across four cohorts that included 34 randomly assignedelementary (grades K–8) schools.The cluster randomized controlled trial component of the study featured two lotteries. One lottery randomlyassigned an initial group of 16 recruited schools to either implement TAP immediately (Cohort 1: eight schools)or delay implementing TAP for 1 academic year (Cohort 2: eight schools). The second lottery, conducted 2years later, randomly assigned another group of 18 schools to either implement TAP immediately (Cohort 3:nine schools) or delay implementation for 1 academic year (Cohort 4: nine schools). Thus, Cohort 2 served as anon-TAP comparison group for Cohort 1 during Cohort 1’s first year of implementation, and Cohort 4 served asa non-TAP comparison group for Cohort 3 during Cohort 3’s first year of implementation. Estimates of programimpact after 1 year of implementation were then made by measuring the changes in student achievement for theTAP group of schools (Cohorts 1 and 3 pooled together) compared to the changes for both delayed implementation cohorts (Cohorts 2 and 4 pooled together). The authors also reported supplemental achievement findings froma quasi-experimental analysis that compared schools that implemented TAP (Cohorts 1 through 4) to a matchedgroup of non-TAP schools.Student achievement was assessed each March using scores on the science, reading, and mathematics sectionsof the Illinois Standards Achievement Test (ISAT), with the prior year’s score being used as a pretest measure. Theanalytic samples for the cluster randomized controlled trial were: 1,717 grade 4 and 7 students (808 TAP and909 comparison) for science achievement; 7,661 grade 4–8 students (3,717 TAP and 3,944 comparison) forreading achievement, which falls in the English language arts achievement domain; and 7,656 grade 4–8 students(3,714 TAP and 3,942 comparison) for mathematics achievement.6Because the cluster randomized controlled trial analysis of the impact of TAP teachers on student achievementuses data from students who were present at the time of randomization of schools and those who joined theschools after randomization, the analysis is not eligible for the rating of meets WWC group design standardswithout reservations. The study demonstrated baseline equivalence of the analytic samples and, therefore,meets WWC group design standards with reservations.TAP : The System for Teacher and Student Advancement July 2015Page 5