WIXLIB

Large Scale Assessments as Change AgentsJo Ritzen, chair Empower European UniversitiesProfessor UNU/MERIT/Maastricht Graduate Schoolof GovernanceILSA Conference, PrincetonEducation Testing ServiceMarch 16-17-18, 2011Maastricht Graduate School of Governance

Introduction Transparency to keep you honest Personal experience of the recent past on largescale assessments as change agents Focus on “PISA”, the project international studentachievementMaastricht Graduate School of Governance

Transparency in educational outcomes as achange agent with the institution as the unit ofperformanceMaastricht Graduate School of Governance

Some examples of the Netherlands in thenineties: the higher education choice guide the inspectorate assessment of schoolsMaastricht Graduate School of Governance

Is change socially desirable? The value added debate Resistance to large-scale assessmentsMaastricht Graduate School of Governance

The Policy Impact of PISA PISA is an internationally standardised assessment PISA assesses the ability of 15-year-olds to applytheir knowledge in reading, mathematics andscience to real life problems Assessments take place every three yearsMaastricht Graduate School of Governance

Evaluation of the impact of PISA on policy Our model needs some adaption: consumer choiceis no longer relevant Based on respondents assessment of impact of PISAon policy:- Group A (lowest through third decile)- Group B (fourth through seventh decile)- Group C (eighth through tenth decile)Maastricht Graduate School of Governance

Approximate expectations Expectations are based on comparisons with“comparison” (neighboring?) countries (applicableto 2000 PISA) Expectations are based on expected changes overtime due to national improvement efforts(applicable to 2003 and 2006) A combination of both Countries which express great general ambitionsalso are more ambitious with respect to educationMaastricht Graduate School of Governance

The Academic Achievement Curve (selectedcountries)4035 % of sample3025OECDFINTURDEUSUK20151050below1123 Achievement levelMaastricht Graduate School of Governance456

Stakeholders responsible for implementing policies in light of the PISA resultsAll countries (n 659)Business communityTeacher associationsMedia representativesAcademics and researchersParentsSchool principalsLocal government officialsPolicy makers0%10%20%Extremely and Very30%Moderately40%50%Not very and Not at all60%Don't know70%80%90%100%No responseQuestion: Who would you identify as being responsible forimplementing policies in light of the PISA results in yourcountry? Please indicate the degree of responsibility for eachstakeholder and specify to which PISA assessment you refer.Maastricht Graduate School of Governance

Extent of key stakeholders' own responsibility for resultsBusiness community (12.8, n 33)Teacher associations (11.8, n 36)Academics and researchers (9.7, n 125)Parents (8.8, n 65)School principals (7.8, n 79)Local government officials (6.7, n 72)Policy makers (5.8, n 145)0%Very great and Great Extent10%20%Moderate extent30%40%Limited and No extent50%Don't know60%70%No response80%90%100%Not applicableQuestion: To what extent do [members of own stakeholder group]feel responsible for your country's results in PISA?Maastricht Graduate School of Governance

Large scale assessments can play a role as a changeagent if the information reaches the stakeholder andeven more so if stakeholders are challenged on theassessments If you want large-scale assessment to have animpact, then you should disseminate them to anddiscuss them with teachers, parents, school boardsetc. on the local level What to do with the PISA results? How torestructure the school such that better results willbe achieved?Maastricht Graduate School of Governance

Large scale assessments can be important changeagents provided the assessment addresses theprime concerns of stakeholders in education Most large-scale assessments dodge the valueadded question. This undermines their potential forwanted changeMaastricht Graduate School of Governance

Thank you!Maastricht Graduate School of Governance

Maastricht Graduate School of Governance Large Scale Assessments as Change Agents Jo Ritzen, chair Empower European Universities Professor UNU/MERIT/Maastricht Graduate School . of Governance. ILSA Conference, Princeton. Education Testing Service. March 16-17-18, 2011.