Transcription
PISA 2012 Assessmentand Analytical FrameworkMathematics, Reading, Science,Problem Solving and Financial LiteracyPr ogr am m e f or Int er nat ional St udent A s s es s m ent
PISAPISA 2012 Assessmentand Analytical FrameworkMathematics, Reading, Science,Problem Solving and Financial Literacy
This work is published on the responsibility of the Secretary-General of the OECD. The opinionsexpressed and arguments employed herein do not necessarily reflect the official views ofthe Organisation or of the governments of its member countries.This document and any map included herein are without prejudice to the status of orsovereignty over any territory, to the delimitation of international frontiers and boundariesand to the name of any territory, city or area.Please cite this publication as:OECD (2013), PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science,Problem Solving and Financial Literacy, OECD -enISBN 978-92-64-19052-8 (print)ISBN 978-92-64-19051-1 (PDF)The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The useof such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlementsin the West Bank under the terms of international law.Photo credits: khoa vu /Flickr /Getty Images Shutterstock /Kzenon Simon Jarratt /CorbisCorrigenda to OECD publications may be found on line at: www.oecd.org/publishing/corrigenda. OECD 2013You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databasesand multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided that suitableacknowledgement of OECD as source and copyright owner is given. All requests for public or commercial use and translation rightsshould be submitted to rights@oecd.org. Requests for permission to photocopy portions of this material for public or commercial useshall be addressed directly to the Copyright Clearance Center (CCC) at info@copyright.com or the Centre français d’exploitation du droitde copie (CFC) at contact@cfcopies.com.
ForewordThe OECD Programme for International Student Assessment (PISA), created in 1997, represents a commitment by thegovernments of OECD member countries to monitor the outcomes of education systems in terms of student achievement,within a common internationally agreed framework. PISA is a collaborative effort, bringing together scientific expertisefrom the participating countries and steered jointly by their governments on the basis of shared, policy-driven interests.Participating countries take responsibility for the project at the policy level. Experts from participating countries alsoserve on working groups that are charged with linking the PISA policy objectives with the best available substantive andtechnical expertise in the field of internationally comparative assessment. Through involvement in these expert groups,countries ensure that the PISA assessment instruments are internationally valid and take into account the cultural andcurricular context of OECD member countries. They also have strong measurement properties, and place an emphasison authenticity and educational validity. PISA 2012 represents a continuation of the data strategy adopted in 1997 byOECD countries. As in 2003, mathematical literacy is the focus of the PISA 2012 survey. The framework for assessingmathematics was fully revised for the PISA 2012 assessment and introduces three new mathematical processes thatform the basis of developments in the reporting of PISA mathematics outcomes. A computer-based assessment ofmathematics was also included in the 2012 cycle. The framework for assessing science was revised for PISA 2006 whilethe framework for assessing reading was revised for PISA 2009. Both of these frameworks remained unchanged in PISA2012. The analytic framework that formed the basis of the development of the various questionnaire instruments wasalso redeveloped for PISA 2012.Additions to the PISA 2012 assessment include a computer-based assessment of problem solving and an assessmentof financial literacy. In 2003, problem solving became an assessment domain in PISA but was not reintroduced in thePISA 2006 and 2009 cycles. However, a new framework was devised for problem solving in PISA 2012 and additionalassessment methodologies were implemented, allowing for the real-time capture of students’ capabilities. In particular,the PISA 2012 assessment of problem solving was computer-based, and interactivity of the student with the problem is acentral feature of the assessment. Financial literacy was included for the first time in the PISA assessment. Its frameworkprovides a common language for discussion about financial literacy, a working definition of the domain, an articulatedplan for developing items, and defines the relevant content, processes and contexts for the assessment of 15-year-oldstudents in this domain.This publication presents the guiding principles of the PISA 2012 assessment, which are described in terms of the skillsstudents need to acquire, the processes that need to be performed and the settings in which knowledge and skills areapplied. Further, it illustrates the assessment domains with a range of sample tasks.The framework development for mathematics was undertaken jointly by the Australian Council for Educational Research(ACER), and Achieve, Inc., a USA-based educational development organisation. The framework development for allother cognitive domains, as well as the context questionnaire, was undertaken by the Australian Council for EducationalResearch.The frameworks were developed by the expert panels, with the guidance of Raymond Adams, Barry McCrae, Petra Lietz,Juliette Mendelovits, Dara Ramalingam and Ross Turner from ACER. The mathematics expert group was chaired by KayeStacey from the University of Melbourne. The problem solving expert group was chaired by Joachim Funke from theUniversity of Heidelberg. The reading expert group was chaired by Irwin Kirsch of Educational Testing Service in theUnited States of America. The science expert group was chaired by Rodger Bybee, formerly of the Biological ScienceCurriculum Study in the United States. The financial literacy expert group was chaired by Annamaria Lusardi of TheGeorge Washington University School of Business, in the United States of America. The questionnaire expert groupwas chaired by Eckhard Klieme of the German Institute for International Educational Research (DIPF) in Germany. Themembers of the expert groups are listed in Annex B of this publication. The frameworks have also been reviewed byexpert panels in each of the participating countries. The chapters were drafted by the respective expert groups underthe direction of their chairs. The publication was prepared by the OECD Secretariat, principally by Michael Davidson,Sophie Vayssettes, Pablo Zoido, Giannina Rech, Elisabeth Villoutreix, Marilyn Achiron and Elizabeth Del Bourgo.The report is published on the responsibility of the Secretary-General of the OECD.PISA 2012 ASSESSMENT AND ANALYTICAL FRAMEWORK – OECD 20133
Table of ContentsINTRODUCTION. 13Overview . 13Basic features of PISA 2012 . 14What makes PISA unique . 16An overview of what is being assessed in each domain . 17Assessing and reporting PISA 2012 . 18The context questionnaires and their use . 20Collaborative development of PISA and its assessment framework . 20CHAPTER 1 PISA 2012 MATHEMATICS FRAMEWORK . 23Introduction . 24Defining mathematical literacy. 24 A view of students as active problem solvers in PISA 2012. 25 An explicit link to a variety of contexts for problems in PISA 2012. 27 A visible role for mathematical tools, including technology in PISA 2012. 27Organising the domain . 27 Mathematical processes and the underlying mathematical capabilities. 28 Mathematical content knowledge. 31 Contexts . 37Assessing mathematical literacy . 38 Structure of the PISA 2012 mathematics assessment . 38 Reporting proficiency in mathematics . 41 Attitudes towards mathematics . 42 Optional computer-based assessment of mathematics . 43Summary . 44Illustrative PISA mathematics items . 46Notes . 56References . 57CHAPTER 2 PISA 2012 READING FRAMEWORK . 59Introduction . 60Defining reading literacy . 60Organising the domain . 62 Situation . 62 Text . 63 Aspect . 66 Summary of the relationship between printed and digital reading texts and tasks . 69Assessing reading literacy . 69 Building tasks in the print medium . 69 Building tasks in the digital medium . 71Reporting proficiency in print and digital reading . 78 Print reading . 78 Digital reading . 80Summary . 80PISA 2012 ASSESSMENT AND ANALYTICAL FRAMEWORK – OECD 20135
TABLE OF CONTENTSIllustrative PISA print reading items. 81Notes . 94References . 95CHAPTER 3 PISA 2012 SCIENCE FRAMEWORK. 97Introduction . 98Defining scientific literacy. 99 Explanation of the definition. 100Organising the domain . 101 Situations and context . 102 Illustrative PISA science items. 103 Scientific competencies. 107 Scientific knowledge . 108 Attitudes towards science. 110Assessing scientific literacy. 111 Test characteristics. 111 Science assessment structure . 112Reporting proficiency in science. 113Summary . 114Notes . 116References . 117CHAPTER 4 PISA 2012 PROBLEM-SOLVING FRAMEWORK. 119Introduction . 120 Background to the 2012 assessment. 120 Problem solving in the OECD survey of adult skills. 121Defining problem-solving competency. 121 Definition of a problem . 121 Definition of problem solving. 122 Definition of problem-solving competency. 122 Scope of the assessment. 123Organising the domain. 124 Problem context. 124 Nature of problem situation. 124 Problem-solving processes . 125Assessing problem-solving competency. 127 Structure of the assessment . 127 Functionality provided by computer delivery. 127 Task characteristics and difficulty. 127 Distribution of items . 130Reporting problem-solving competency. 130Summary . 131Illustrative PISA problem-solving items. 131Notes . 135References . 136CHAPTER 5 PISA 2012 FINANCIAL LITERACY FRAMEWORK. 139Introduction . 140 The importance of financial literacy. 140 Expected benefits of financial education and improved levels of financial literacy. 141 OECD activities in relation to financial education. 1416 OECD 2013 – PISA 2012 ASSESSMENT AND ANALYTICAL FRAMEWORK
TABLE OF CONTENTS Financial education for youth and in schools. 142 The need for data. 143 The measurement of financial literacy in PISA. 143Defining financial literacy. 144Organising the domain . 146 Content . 146 Processes . 151 Contexts. 155 Non-cognitive factors. 157Assessing financial literacy. 158 The structure of the assessment .
mathematics was fully revised for the PISA 2012 assessment and introduces three new mathematical processes that form the basis of developments in the reporting of PISA mathematics outcomes. A computer-based assessment of mathematics was also included in the 2012 cycle. The framework for assessing science was revised for PISA 2006 while
