Transcription
NCEE 2010-4036NCEE 2010-4037U.S. DEPARTMENT OF EDUCATIONU.S. DEPARTMENT OFStudy of Teacher Preparation inEarly Reading Instruction
Study of Teacher Preparation inEarly Reading InstructionSEPTEMBER 2010Terry SalingerLorin MuellerMengli SongYing JinCourtney ZmachMichele ToplitzAmerican Institutes for ResearchMark PartridgeAdam BickfordOptimal Solutions Group LLCNCEE 2010-4036U.S. DEPARTMENT OF EDUCATION
U.S. Department of EducationArne DuncanSecretaryInstitute of Education SciencesJohn Q. EastonDirectorNational Center for Education Evaluation and Regional AssistanceRebecca MaynardCommissionerSeptember 2010This report was prepared for the Institute of Education Sciences under Contract No. ED-04-CO0062/0001. The project officer was Tracy Rimdzius in the National Center for Education Evaluation andRegional Assistance.This publication is in the public domain. Authorization to reproduce it in whole or in part is granted.While permission to reprint this publication is not necessary, the citation should be: Salinger, T., Mueller,L., Song, M., Jin, Y., Zmach, C., Toplitz, M., Partridge, M., & Bickford, A. Study of Teacher Preparationin Early Reading Instruction (NCEE 2010-4036). Washington, DC: National Center for EducationEvaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.IES evaluation reports present objective information on the conditions of implementation and impacts ofthe programs being evaluated. IES evaluation reports do not include conclusions or recommendations orviews with regard to actions policymakers or practitioners should take in light of the findings in thereports.To order copies of this report, Write to ED Pubs, Education Publications Center, U.S. Department of Education, P.O. Box22207, Alexandria, VA 22304.Call in your request toll free to 1-877-4ED-Pubs. If 877 service is not yet available in your area,call 800-872-5327 (800-USA-LEARN). Those who use a telecommunications device for the deaf(TDD) or a teletypewriter (TTY) should call 800-437-0833.Fax your request to 703-605-6794.Order online at www.edpubs.gov.This report also is available on the IES website at http://ies.ed.gov/ncee.Upon request, this report is available in alternate formats such as Braille, large print, audiotape, orcomputer diskette. For more information, please contact the Department’s Alternate Format Center at202-260-9895 or 202-205-8113.
ACKNOWLEDGMENTSThis study represents a collaborative effort between Optimal Solutions Group LLC (Optimal)and the American Institutes for Research (AIR). We appreciate the willingness of 99 teacherpreparation programs to participate in the study and of the 2,237 pre-service teachers whocompleted the Pre-service Teacher Preparation Program and Knowledge Survey. Additionally,we are grateful for the interest in the study shown by 15 reading researchers, 19 reading teachereducators, and 28 recent college graduates, who also agreed to take the Pre-service TeacherPreparation Program and Knowledge Survey. Their efforts provided us comparison groupsagainst which to gauge the performance of the pre-service teachers.Our work was strengthened by our expert technical working group. Members included JoanneCarlisle, University of Michigan; Donald Compton, Vanderbilt University; Linnea Ehri, CityUniversity of New York; Geoffrey Phelps, University of Michigan; Ray Reutzel, Utah StateUniversity; and Joanna Williams, Teachers College.Work on this study consisted of many stages, and numerous people have been involved duringthe development of the measures, recruitment of institutions and students, administration of thePre-service Teacher Preparation Program and Knowledge Survey, and data analysis. At AIR,we wish to acknowledge David Baker, Rebecca Holland-Coviello, Andrea Muse, DerrickJohnson, Jim Kurtessis, Nancy Matheson, Mark Smith, Stephane Baldi, Elizabeth Sparano,Catherine Lopez, Melanie Skemer, Charlene Impey, Amy Holtzman, and Juliet Jason. AtOptimal, we recognize Mark Turner, Monica Leal Priddy, Eric Asongwed, Amy Peterson,Andrew Gendreau, Annie Oonjit, Brandon Newbury, Laurel Davis Rasinski, CharleneGatewood, Alexander Kyerematen, Deanna Bussard, Shelley Marshall, Helga Gross, AnnaHabash, Michelle Blair, Allison Buckley, Ja-Nee Jackson, Veronika Peleshchuk, DianaBehrendt, Grace Hong, Kelly Gent, and Christopher Hill.Finally, we wish to thank Michael Garet, Dan Sherman, Tsze Chan of AIR and Larry Dolan andToks Fashola of Optimal for technical reviews; and Holly Cutting Baker and Aida Gomez forhelping to make the report useful and understandable.iii
ContentsPageAcknowledgments . iiiExecutive Summary . ixPrimary Research Questions. ixStudy Design . ixData Analysis and Constructs . xiStudy Findings . xiiResearch Question 1 . xiiResearch Question 2 . xvChapter 1: Study Overview. 1Primary and Secondary Research Questions . 2Study Design . 3Organization of This Report . 5Chapter 2: The Study Sample . 6Three-Stage Sample Design . 6State Selection . 7Institution Selection . 8Pre-Service Teacher Selection . 8Computation of Final Weights . 9Exceptions . 10Recruitment of Study Sample . 11Chapter 3: Data Collection . 14The Program Survey . 14The Knowledge Assessment. 16Review and Modification of the Existing Assessment . 16Pilot Testing of the Knowledge Assessment . 17The Operational Data Collection Instrument. 18Counterbalanced Forms . 20Data Collection . 20Summary . 21Chapter 4: Findings . 22Definitions of Variables Used in the Report . 22Analyses to Answer the Study’s Primary Research Questions . 23Analyses to Answer the Study’s Secondary Research Questions . 32Summary . 43iv
References . 45Appendix A. Pre-Service Teacher Program Survey . A-1Appendix B. Program Survey Variables and Assessment Items . B-1Appendix C. Program Survey and Knowledge Assessment Pilot Testing . C-1Appendix D. Recruitment . D-1Appendix E. Demographic Information Gathered through Program Survey . E-1Appendix F. Data Cleaning Procedures . F-1Appendix G. Psychometric Analysis of the Program Survey Scales MeasuringCoursework Emphasis on, Filed Experience Exporsure to, and Feelingsof Preparedness to Teach the Essential Components . G-1Appendix H. Descriptive Statistics for Program Survey Items . H-1Appendix I. Knowledge Assessment Scoring, Scale, Distractor, and Forms Analysis . I-1Appendix J. Knowledge Assessment Means Tables .J-1Appendix K. Hierarchical Linear Model Used to Answer the Primary and SecondaryResearch Questions . K-1Appendix L. Administration of The Knowledge Assessment to Experts and Novices . L-1v
List of TablesTable ES-1.Table 2-1.Table 3-1.Table 3-2.Table 3-3.Table 4-1.Table 4-2.Table 4-3.Table 4-4.Table 4-5.Table 4-6.Table 4-7.Table 4-8.Table B-1.Table B-2.Table C-1.Table C-2.Table C-3.Table C-4.Table C-5.Table D-1.Table D-2.Table D-3.Table E-1.Table E-2.Table E-3.Table E-4.Table E-5.Table E-6.Table G-1.Distribution of items in operational Pre-Service Teacher Preparation Program andKnowledge Survey . xiCharacteristics of final pre-service teacher sample . 12Program survey item totals and internal consistency, by section . 16Items in the Knowledge Assessment, with statistics from pilot test. 18Distribution of items in Pre-Service Teacher Preparation Program and KnowledgeSurvey in the main study sample . 19National estimates of teacher education programs’ focus on the essential componentsof early reading instruction, by essential component and aspect of program . 24National estimates of pre-service teacher responses to question about having learned―what students need to know and be able to do‖ in the five essential components ofreading . 30Mean raw scores and percent correct for the Knowledge Assessment . 31National estimates of teacher education programs’ focus on the essential componentsof early reading instruction, by essential component, aspect of program, and by sectorof institution . 35National estimates of teacher education programs’ focus on the essential componentsof early reading instruction, by essential component, aspect of program, and by thehighest degree offered by institution . 36National estimates of teacher education programs’ focus on the essential componentsof early reading instruction, by essential component, aspect of program, and byprogram type. 38Regression coefficients between degree of program focus and pre-service teachers’knowledge, by essential component and aspect of program. 40Regression coefficients between pre-service teachers’ knowledge and their feelings ofpreparedness to teach, by essential component . 44Variables measured by the Pre-Service Teacher Program Survey, by section of thesurvey and item .
IES evaluation reports do not include conclusions or recommendations or views with regard to actions policymakers or practitioners should take in light of the findings in the reports. To order copies of this report, Write to ED Pubs, Education Publications Center, U.S. Department of Education, P.O. Box 22207, Alexandria, VA 22304. Call in your request toll free to 1-877-4ED-Pubs. If 877 .
