WIXLIB

MAY 2019Validity of the SAT for PredictingFirst-Year Grades and Retentionto the Second Year PAUL A. WESTRICK, JESSICA P. MARINI, LINDA YOUNG,HELEN NG, DORON SHMUELI, AND EMILY J. SHAW

ContentsAbstract. 4Introduction . 5SAT Score Relationships with First Year Grade-Point-Average. 6Methodology. 6Sample. 6Table 1: Institutional Characteristics of the 2017 SAT Validity Study Sample and Population ofFour-Year Institutions . 7Table 2: Student Characteristics of the 2017 SAT Validity Study Sample and 2017 GraduatingSeniors with SAT Scores . 8Measures. 8Descriptive Statistics . 9Table 3: Descriptive Statistics for Measures of Interest . 9Table 4: Corrected (Raw) Correlation Matrix of SAT Scores and HSGPA. 10Methods. 10Results. 10Table 5: Corrected (Raw) Correlations of Predictors with FYGPA . 11Figure 1: Mean FYGPA by SAT Total Score Bands . 12Figure 2: Mean FYGPA by HSGPA and SAT Total Score Bands . 13Figure 3: Probability of a 2.50 or Higher FYGPA Given HSGPA and SAT Total Score . 14SAT Score Relationships with Retention to Second Year. 14Methodology. 14Sample. 14Methods. 15Results. 15Table 6: Summaries of Study Measures for Students Returning to the Same Institution for theSecond Year. 15Figure 4: Mean Second-Year Retention Rate by SAT Total Score Bands . 16Figure 5: Mean Second-Year Retention Rate by HSGPA and SAT Total Score Bands . 17Table 7: Probability of Retention to the Same Institution by SAT Deciles and HSGPA Categories. 17Figure 6: Probability of Returning to the Same Institution in the Second Year given HSGPA andSAT Total Scores. 182

Figure 7: Retention Rates of Students Underperforming and Performing as Expected or Better,Total Sample and by Institutional Admittance Rate . 20Conclusion. 20References . 22Appendix A: Participating Institutions . 23Appendix B: Retention Analysis Sample and Population Institutional Characteristics. 25Appendix C: Retention Analysis Sample and Population Student Characteristics . 26Appendix D: Retention Analysis Sample and Population Descriptive Statistics for Measures of Interest . 27Appendix E: Percentage of Students Retained to the Same Institution, Institutional Characteristics . 28Appendix F: Percentage of Students Retained to the Same Institution, Student Sample Characteristics . 293

AbstractThis report represents the first national operational SAT validity study since the SAT was redesignedand launched in March 2016. This is among the largest SAT validity studies ever conducted and is basedon data from more than 223,000 students across 171 four-year colleges and universities. Results showthat the SAT is essentially as effective as high school grades in predicting students’ college performanceand, when these two measures are combined, offers the most accurate understanding of studentperformance than either measure used alone. Analyses also demonstrate that SAT scores are useful forunderstanding whether students will be retained to the second year of college. In particular, findingsshow: SAT scores are strongly predictive of college performance—students with higher SAT scores aremore likely to have higher grades in college.SAT scores are predictive of student retention to their second year—students with higher SATscores are more likely to return for their sophomore year.SAT scores and HSGPA are both related to academic performance in college but tend tomeasure slightly different aspects of academic preparation. Using SAT scores in conjunction withHSGPA is the most powerful way to predict future academic performance.o On average, SAT scores add 15% more predictive power above grades alone forunderstanding how students will perform in college.o SAT scores help to further differentiate student performance in college within narrowHSGPA ranges.Colleges can use SAT scores to identify students who may be in need of academic supportbefore they start college and throughout their college education by monitoring predicted versusactual performance and by helping to position these students for success.Having a more accurate understanding of students’ future performance helps ensure that colleges anduniversities will not overlook or miss students who will be successful on campus. Also, the combinationof SAT score information with HSGPA information helps institutions identify enrolling students who maybenefit from additional support and monitoring to ensure that they are academically successful in a waythat HSGPA alone cannot do. Future research will examine SAT validity by institutional and studentsubgroups and will continue to analyze and document the relationship between SAT scores and othercollege outcomes, including course grades, later college performance, and degree completion.4

IntroductionCollege Board launched the redesigned SAT in 2016 to better reflect the work that students do in highschool, focusing on the core knowledge and skills that research has shown to be critical for students tobe ready for college and career. This study represents the first national operational SAT validity researchstudy to examine the utility of SAT scores for admission decisions, focusing on the outcomes of first-yeargrade point average (FYGPA) and retention to the second year. This report is divided into two sectionswith the first section focusing on SAT score relationships with FYGPA and the second section focusing onSAT score relationships with retention outcomes.As noted above, College Board aimed to make the SAT an assessment that reflects the work thatstudents need to do to be ready for and successful in college. 1 Scholarly research and empirical dataderived from secondary and postsecondary curriculum surveys conducted by College Board and otherorganizations formed the evidentiary foundation for specifying the test content and domains of interest.The SAT has three sections: The Evidence-Based Reading and Writing section, the Math section, and anoptional Essay section. The Evidence-Based Reading and Writing section and the Essay sectionincorporate key design elements supported by evidence, including: The use of a range of text complexity aligned to college- and career-ready reading levels;An emphasis on the use of evidence and source analysis;The incorporation of data and informational graphics that students will analyze along with text;A focus on relevant words in context and on word choice for rhetorical effect;Attention to a core set of important English language conventions and to effective writtenexpression; andThe requirement that students interact with texts across a broad range of disciplines.The key evidence-based design elements incorporated into the SAT Math section include: A focus on the content that matters most for college and career readiness (rather than broadcoverage of a vast array of content and skills);An emphasis on problem solving and data analysis; andThe inclusion of “Calculator: Permitted” questions as well as “Calculator: Not Permitted”questions and attention to the use of the calculator as a tool.The SAT Evidence-Based Reading and Writing section and the Math section each report scores on a 200–800 scale. The Evidence-Based Reading and Writing section has two test scores, the Reading Test scoreand the Writing and Language Test score, each reported on a 10–40 scale. The Math section alsoproduces a Math Test score that is reported on a 10–40 scale. Several new subscores and cross-testscores provide richer information to students, schools, and institutions on student performance.More information on the development of the SAT can be found in Test Specifications for the Redesigned SAT (College Board, 2015) and SAT Suite of Assessments Technical Manual: Characteristics of the SAT (College Board,2017).15

Students earn points for correct answers to questions, but, unlike the previous SAT, they do not losepoints for incorrect answers. For the optional Essay, students now have 50 minutes instead of 25minutes to write a response to a reading passage.To gather early SAT validity evidence, a pilot predictive validity study was conducted in 2014-2015 tostudy the relationships between SAT scores and college performance across a sample of 2,050 studentsat 15 four-year institutions, using a pilot form of the redesigned SAT (Shaw, Marini, Beard, Shmueli,Young, & Ng, 2016). That study found that the redesigned SAT was as predictive of college success as theprevious SAT, that redesigned SAT scores improved the ability to predict college performance aboveHSGPA alone, and that there was a strong, positive relationship between redesigned SAT scores andgrades in matching college course domains, suggesting that the redesigned SAT is sensitive to instructionin English language arts, math, science, and history/social studies. The entering class of fall 2017 was thefirst cohort of students to have primarily taken the redesigned SAT and completed the first year ofcollege in 2017-2018 so that their corresponding college outcome information could be examined forvalidity research. Therefore, this first validity study will focus on the relationships between SAT sectionscores and FYGPA and retention to the second year for that cohort.SAT Score Relationships with First Year Grade-Point-AverageMethodologySampleCollege Board broadly recruited four-year institutions with at least 250 first-year students (at least 75 ofthose students had to have SAT scores) to participate in this initial SAT validity study. These institutionsprovided data through College Board’s secure online Admitted Class Evaluation Service (ACES )system. Ultimately, 171 institutions provided the complete student-level information needed for theanalyses that follow in this section of the report. See Appendix A for the list of participating institutions.Table 1 includes the characteristics of the institutions in the sample and shows that the sample is quitediverse with regard to region of the U.S., control (public/private), selectivity, and size. Compared to thepopulation 2 of four-year institutions for this study, the institutional study sample included more publicinstitutions, more selective institutions, and more “large” and “very large” institutions than thereference population. This is to be expected as there was a sample size minimum to participate in thestudy and more-selective institutions rather than less selective-institutions would be more apt to usethe SAT and examine the relationship between the SAT and college outcomes.2The population included four-year public or private nonprofit institutions that accepted 90% or fewer applicantsfor admission.6

Table 1: Institutional Characteristics of the 2017 SAT Validity Study Sample and Population of FourYear InstitutionsU. S. RegionControlAdmittance RateUndergraduate EnrollmentVar

This report represents the first national operational SAT validity study since the SAT was redesigned and launched in March 2016. This is among the largest SAT validity studies ever conducted and is based on data from more than 223,000 students