Transcription
A Good Start & Paying for Persistence:Building Knowledge to Improve EducationLashawn Richburg-HayesIHEP Summer InstituteBirmingham, ALJuly 22, 2008
2Workshop Outline Background on MDRC Opening Doors & evaluation Projects Learning Communities Performance-Based Scholarships Achieving the Dream Others
3Opening Doors Purpose: test the effectiveness of programs designed to increasestudent success 4 program strategies at 6 community colleges Learning communities: Kingsborough Community College (NY) Performance-based scholarships: Delgado Community College and LouisianaTechnical College – West Jefferson (LA) Enhanced student services: Lorain County and Owens Community Colleges(OH) Guidance course tied to basic academic skills instruction: Chaffey College(CA) Primary research questions - Do the interventions result in: Increased course completion and persistence? Higher rates of graduation or transfer? Increased employment and better jobs?
Alternatives Trend analysisWhat would the outcome be in the absenceof the program?4
Pre-Post angeParticipants2.02.60.6Problems with this approach: Not always feasible Pre-program period not representative Other factors possibly responsible5
Comparison timateddifference0.4Issue of selection bias remains6
7Random assignment is:(In simple designs)The assignment of program participantsrandomly to two groups: a treatment groupallowed to enter the program and a controlgroup that is not.The experience of the control group gives thecounterfactual.
8Random Assignment DesignTargeted students invited to participate instudyStudents give consentBaseline data collectedRandomassignmentProgram groupEnrolled ininterventionControl groupReceived regularcourses and services
9Pros of Random Assignment No selection bias Internal validity External validity Identification of useful or wasteful program Any systematic difference confidently attributedto program Increases knowledge of program
10Cons of Random Assignment Costs Suitability of questions Applicability Burden on sites Limited outcomes studied Control group contamination “Endogenously defined” subgroup analyses Inability to control no-shows
11Relevance to Summer Institute?Habit #2:Begin with the end in mind
12Success Stories Learning Communities Program at KingsboroughCommunity College (KCC) Groups of 25 freshman took 3 linked courses together:English (usually developmental), student development(taught by counselor), and a standard college course, suchas sociology or health.Performance-Based Scholarships in Louisiana Low-income parents received 1,000 for two semesters( 2000 total) on two conditions: enrolled at least half-timeand maintained “C” or better average. Scholarship paid in increments: 250 on enrollment, 250on passing midterms, 500 on passing courses.
13Two-Year Effects ofKCC Learning Communities Learning Communities improved: students’ college experience students’ progression through developmental Englishrequirements, and some educational outcomes while students were in thelearning community program, but the effects diminishedin subsequent semesters. The evidence is mixed about whether the program increasedpersistence.
14Impact of Learning Communitieson Students’ College Experience*******
Key Effects on Academic Outcomes DuringLearning Community o. of coursesattempted4.94.40.4***No. of coursespassed3.83.20.6***No. of regularcredits earned8.07.70.3No. ofdevelopmentalcredits earned3.52.60.9***15
Effects on English Skills Assessment TestsThrough Third SemesterOutcome (%) Program Control DifferenceGroupGroupAttemptedreading test17.813.34.5***Passed readingtestAttemptedwriting test88.686.02.5*46.839.17.7***Passed writingtest66.260.55.7**16
No Immediate Impact on Persistence,but Maybe a Long-Term Effect**17
Louisiana Results18 Scholarships led students to: enroll for more credits pass more courses and persist. Effects continued into 3rd and 4th semesters Hurricane Katrina precluded measuring longterm effects (e.g., graduation)
Effect of Hurricane KatrinaSpring 2004Summer 2004Fall 2004Spring 2005Cohort 41st semester2nd semesterCohort 3Cohort 2Cohort 1Spring Spring2006Summer2006Fall2006Spring200619
Selected Findings for First twoSemesters (n 1,019)TreatmentControlImpactRegistrationFirst semesterSecond semester82.264.876.849.85.5**15.0 ***Number of credits attemptedFirst semesterSecond semester8.66.28.05.00.6 **1.2 ***Total credits earnedFirst semesterSecond semester5.93.94.62.81.2 ***1.1 ***20
Selected Findings for First two Cohorts(n 537)TreatmentControlImpactRegistrationFirst semesterSecond semesterThird semesterFourth 11.8 ***7.2 **Number of credits attemptedFirst semesterSecond semesterThird semesterFourth semester7.24.95.42.76.73.54.02.10.51.4 ***1.4 ***0.6Total credits earnedFirst semesterSecond semesterThird semesterFourth semester4.53.43.11.63.42.12.21.51.1 ***1.3 ***0.9 **0.121
Impact of Scholarshipson Total Credits Earned12.0***10.08.06.04.0********2.00.01st semester after 2nd semesterrandomafter randomassignmentassignmentOpening Doors group3rd semesterafter randomassignmentTotal creditsearned afterrandomassignmentControl group22
23Lessons Learning Communities Significantly improved academic outcomes during the learningcommunities semester Most notable lasting effects are on perceptions of college experienceand completion of English exams (both reading and writing) One semester of learning communities gives students a boost, but maynot be transformative Performance-Based Scholarships Large, positive effects on all measures of academic achievement Positive effects endured even after scholarship ended Financial incentive tied to performance seems key
24Next Generation Projects Testing variations in other locations withdifferent populations is essential to determiningwhether the findings can be generalized (andeven strengthened) as the basis for large-scalereplication and widespread policy change. Learning Communities Demonstration Performance-Based ScholarshipDemonstration
25Learning CommunitiesDemonstration 5-year grant from Institute for Education Sciences at U.S. DOE Part of the National Center for Postsecondary Research, a partnershipbetween Community College Research Center at Columbia UniversityTeachers College (lead), MDRC, and University of Virginia Uses a random assignment design to ascertain the effectiveness of learningcommunities for students in need of developmental English or math Six community colleges across the nation: Queensborough Community College (NY)Hillsborough Community College (FL)Kingsborough Community College (NY)Merced College (CA)Houston Community College System (TX)Community College of Baltimore County (MD) Final report in fall of 2011.
26Variation ofLearning CommunitiesCoredevelopmental mathLinked with:Fall 07: developmental EnglishHCCdevelopmental readingSpring 08: college-level English composition;speech;business; or sociologystudent success courseKCCintegrative seminarQCCMercedHoustonCCBCtwo courses required for an occupational major:(accounting; business administration; mentalhealth and human services; early childhoodeducation; and pre-nursing/allied health)developmental English developmental reading; developmental math; orguidance coursedevelopmental mathstudent success coursedevelopmental English college-level health; psychology; speech; history;or readingcomputer information systems; or sociology
27Performance-BasedScholarship Demonstration Anchor funding from Bill & Melinda GatesFoundation Variations along six dimensions of the program: Target group Dollar amount of the scholarships Duration of the scholarships Types of counseling and other services accompanying the scholarships. Administrative arrangements for disbursing scholarships Schedule of scholarship payments
28Scope of PBSDemonstration 3 Waves phased in over 3 years First phase (08 – 09): 2-3 Ohio colleges, 2 New York City University of New Mexico Second phase (09-10) CaliforniaWashingtonArizona or FloridaNew York State colleges Third phase (10-11) Minority-serving institutions
29Wave I Sites
30Wave I Sites (continued)
31Policy Relevance Alternative to merit scholarships Target group customized for each state in orderto: direct the opportunity to the students with highestunmet need and inform policy discussions on state aid programs Interactions with public benefits minimized Demonstration authorized in HEA
32Contact InformationLashawn Richburg-HayesSenior Research Associate212. 340.7598rhayes@mdrc.org
Learning communities: Kingsborough Community College (NY) Performance-based scholarships: Delgado Community College and Louisiana Technical College - West Jefferson (LA) Enhanced student services: Lorain County and Owens Community Colleges (OH) Guidance course tied to basic academic skills instruction: Chaffey College (CA)
