WIXLIB

6 2. Overall conceptual framework guiding the assessment of students’interaction with ICT in PISA 20212.1. Overall framework13. At the heart of the PISA 2021 ICT framework is the relationship between two majordimensions of ICT – access and use – and students’ outcomes (cognitive performance,well-being, and ICT attitudes and competencies). However, the framework also aims toidentify how these links depends on contextual factors or background characteristics, andon existing policies and practices related to ICT. Figure 2.1 provides an overview of theunderlying logical framework used to elaborate the PISA 2021 ICT framework.14. Students’ use of ICT resources is conditional upon the availability, accessibilityand quality of those resources. Conversely, the amount and type of ICT resources madeavailable to students is also influenced by how and why ICT are used. Documentingaccess to ICT therefore aims to answer the following policy question: To what extent isstudent engagement with ICT determined by the availability of diverse and functionalICT resources? Consequently, the use of ICT is considered as a second step – a logicalcontinuation. As such, the assessment of ICT use aims to answer the following question:Given the available ICT resources, how are students’ uses of different kinds of ICTrelated to teaching practices and to students’ cognitive performance, well-being and ICTskills?15. Although this framework recognises the diverse ways in which ICT are used inschool, its prime interest lies in documenting students’ use of ICT. Yet, ICT are integratedin schools without necessarily involving students’ use of them. For example, schoolprincipals can use ICT to administer and manage financial and educational resources; andteaching staff can rely on ICT to improve overall instruction, identify and monitorstudents’ strengths and weaknesses, or communicate with parents. Since these practicesare likely to affect students’ experiences, they are covered by this framework.16. Nevertheless, the structure and scope of PISA better fits a thorough examination ofthe use of ICT at the student level rather than at the school or teacher level. Indeed,optional questionnaires for teachers were distributed in only 17 of thecountries/economies that participated in PISA in 2015; and students cannot be matchedwith a specific teacher, as the sampling of teachers occurs at the school level. Thusinformation based on teachers’ reports only contributes to between-school analyses, andstudent-level information is needed to capture within-school variations in students’ ICTuse.17. The framework also acknowledges the influence of contextual factors, and policiesand practices, on both access to and use of ICT resources, and on students’ outcomes.Contextual factors include the general background characteristics of the educationsystem, schools and students’ households. They include, for example, the level ofeconomic development of a country; students’ grade level in secondary school; theintegration of ICT literacy in the curricula; whether the school is public or private; thesocio-economic and cultural background of students and parents; and even teachers’qualifications. These elements are not specific to ICT, and overlap with the informationfound in the PISA background questionnaires at the student, parent, teacher, school andsystem levels. Although not directly related, these factors are likely to shape andPISA 2021 ICT FRAMEWORK OECD 2018

7constrain the degree of access to ICT resources, and how they are used. They are alsolikely to affect the relationship between access to and use of ICT, on the one hand, andstudent outcomes, on the other.18. In addition, specific ICT-related policies and practices could directly influenceaccess to and use of ICT resources. Such policies include, for example, the existence ofspecific funding for ICT resources in schools, principals’ attitudes towards ICT use as aninstructional tool, and guidelines and support for teachers in using ICT in the classroom.These policies and practices could also be developed as a response to students’ cognitiveperformance or their attitudes towards ICT.19. This logical framework applies to students’ engagement with ICT both in andoutside of the classroom, and covers a variety of stakeholders, including students,teachers, principals and parents.2 This does not imply that students’ access to and use ofICT is conceived in an entirely similar way in both contexts. Indeed, there are crucialdifferences between having a teacher present as an intermediary to students’ engagementwith ICT in the classroom and using ICT for leisure or learning purposes outside theclassroom. For example, teachers can select relevant digital resources ex ante, explainhow to use them efficiently and ensure students are focusing on the learning tasks. Yet,the specificities of ICT use in and outside of the classroom fit into the broader conceptualapproach presented in Figure 2.1. The particularities of both contexts are detailedthereafter.20. As mentioned earlier, the PISA 2021 ICT framework examines the relationshipbetween students’ use of ICT and three outcomes: students’ cognitive achievement, wellbeing and their level of ICT competencies. The assessment of students’ cognitiveachievement in mathematics, reading and science and students’ well-being is not specificto this framework; it relies on the respective frameworks dedicated to each outcomeduring previous PISA cycles. By contrast, the PISA 2021 ICT framework develops aspecific strategy for documenting students’ ICT competencies, which are regarded asincluding both knowledge and skills related to ICT use and attitudes toward ICT. Theframework includes ICT skills that were identified using existing ICT competenceframeworks (e.g. Fraillon et al., 2015; Carreto, Vuorikari and Punie, 2017; Fraillon,Schulz and Ainley, 2013). The objective is to provide directions for the development ofa proper assessment of ICT literacy in future PISA cycles.21. However, the assessment of ICT competencies in PISA 2021 will not rely on a test,as is the case with mathematics, science and reading. Instead, it will rely on students’self-reported attitudes and self-efficacy measures regarding ICT use for learning andleisure. Assessments of both students’ attitudes and self-efficacy will build on similarmeasures developed in previous PISA cycles in relation to the major domain being tested.Particular attention will be given to ensure the validity of the self-efficacy measure, whichcan be challenging, as seen in previous assessments. Indeed, students’ confidence inperforming advanced ICT tasks is weakly associated with Computer and InformationLiteracy (CIL) achievement scores as measured in the International Computer andInformation Literacy Study (ICILS) test (Fraillon et al., 2014). Ensuring the validity ofthe self-efficacy measure requires the coverage of a wide array of tasks pertaining todifferent dimensions of digital competencies (by including technical skills as well asskills related to communication and information literacy among others).PISA 2021 ICT FRAMEWORK OECD 2018

8 Figure 2.1. PISA 2021 ICT conceptual frameworkSource: Authors2.2. Relationship between access to and use of ICT resources22. As shown in Figure 2.1, the availability, accessibility and quality of ICT resourcespartly shape teachers’ and students’ practices with ICT, both in and outside of theclassroom. Indeed, the total amount of ICT equipment available per student is likely toaffect decisions on whether and how to use ICT resources. One could imagine that havingfewer than one computer per student at school would mean that students use computersin group exercises, for example. Similarly, the ease with which ICT can be accessedduring class could affect the work arrangements and frequency of use.23. In addition, the quality of ICT resources for learning – encompassing dimensionsas diverse as technical capacity and performance, teacher (or pedagogical) and studentusability (e.g. ergonomics and ease of use), practicability and adaptability – would likelyaffect the range and relevance of the activities that could be conducted with the availableICT equipment. For instance, slow Internet connections would prevent students fromusing demanding online digital learning resources, while students working on poorlymaintained computers would likely encounter software compatibility or obsolescenceissues. In addition, an educational software could be accessible and engaging to studentsbut not flexible enough to fit a teacher’s pedagogical approach or not in line with thecurriculum. Access to and use of ICT outside of school for learning are vulnerable tosimilar constraints. However, the assessment of the quality of ICT resources for otherPISA 2021 ICT FRAMEWORK OECD 2018

9purposes, such as leisure, has to rely on different measures, although some of the aspectsmentioned above could still be relevant.24. Conversely, students’ and teachers’ use of ICT can also affect decisions about theselection of and attention devoted to ICT resources. Indeed, the extent to which studentsuse ICT resources for learning science or mathematics could guide the selection ofspecific software and hardware requirements, and mandate a certain level of Internetbandwidth, for example. Similarly, if teachers rely on ICT mainly for personalising thepace of students’ learning, for obtaining instant feedback or for collaborative groupexercises, that could affect their school’s decision about whether to purchase a computerfor each student, to invest in individualised learning software, or instead to invest inonline collaborative games and Intranet installations.25. Moreover, teachers’ attitudes towards the use of ICT as a tool for instruction arelikely to influence the amount and types of resources they use in class and available inschools. A similar relationship apply for students’ use of ICT outside of the classroom.Indeed, students’ use of ICT for leisure could be affected by their (and their parents’)attitudes and practices. For example, parents aware of the opportunities and risks of ICTuse could encourage their children to play educational or collaborative video games orsimply limit the access to the game console or computer.26. The strength of the inter-dependence between access to and use of ICT resources,and the relative importance of each of the effects described above, strongly depend onhow responsibilities are shared across the different levels of the education system. In acentralised system, access to ICT resources may be almost entirely determined at thesystem level, with a limited relationship to actual ICT use at the school level. By contrast,if the school enjoys more autonomy in acquiring educational resources and encouragingteaching practices using ICT, teachers may be more involved in selecting the equipmentthey need to pursue their pedagogical strategies. Including ICT skills in the curriculumor adhering to security regulations may also affect schools’ and teachers’ ability to useand provide access to ICT resources. Indeed, the curriculum might favour the use ofparticular ICT resources while security regulations might prevent schools and teachersfrom conducting specific activities.2.3. ICT use in the classroom27. The conceptual framework can be further refined by describing how ICT use bystudents and teachers in the classroom can influence student outcomes. The aim is topostulate hypotheses that will serve as a basis for investigation. This section details threemain ways that ICT use in school may be related to student outcomes (Figure 2.2).28. First, students can use ICT resources to learn a traditional subject, such asmathematics, reading or science. ICT-assisted instruction can affect students’ cognitiveperformance (and other outcomes) through its interaction with teaching strategies andstudents’ engagement with learning. Teachers’ instructional practices prior to ICTintegration into teaching are likely to affect how ICT is used in the classroom.Conversely, the integration of ICT can also change the use and modalities of teachingstrategies, pedagogical practices and classroom arrangements (including teacher orstudent-centred instruction, traditional or enquiry-based teaching, and assessment andfeedback practices for pedagogical purposes). Thus, the relative weight and overallcombination of the different teaching practices (i.e. teacher-directed instruction, student-PISA 2021 ICT FRAMEWORK OECD 2018

10 focused instruction, teacher support, and feedback and adaptive teaching) are expectedboth to guide how ICT are used and to vary with the use of ICT in the classroom.29. Moreover, teachers’ use of ICT can affect subject-specific teaching strategies, suchas using simulations in mathematics. Using ICT as a tool to learn a specific subject canalso affect students’ engagement with learning, as manifested in the time spent receivinginstruction, and students’ concentration on, efforts in and attitudes towards the subject.Teachers may introduce more ICT tools into classrooms precisely in order to attractstudents’ attention.30. The practical use of ICT for learning also entails challenges. Teaching practicesthat rely on ICT require teachers to have specific skills and know-how about the learningprocess and classroom management with digital tools. Integrating digital resources inteaching could also initially require a substantial investment of time and effort inpreparing course materials, classroom management and monitoring the success of theteaching approach. Moreover, the use of ICT for different activities could make it morechallenging to maintain a quiet, peaceful and respectful classroom in which students canconcentrate on academic tasks, and listen to the teacher and other students. Anotherrelated challenge is the potential misuse of ICT by students who could spend a substantialamount of their time in class on social networks, games and other distracting activities.31. Second, students can benefit from specific teaching about ICT – either in adedicated course or time period – aimed at improving ICT-related competencies. Thebenefits for students will depend on the availability of specific ICT educational resources.In particular, students could benefit from ICT teachers or teachers with particularqualifications for delivering ICT courses. They can also have access to learning resourcesspecifically designed to develop ICT competencies.32. In this regard, more and more education systems are integrating computationalthinking (i.e. “a problem-solving methodology that expands the realm of computerscience into all disciplines, providing a distinct means of analysing and developingsolutions to problems that can be solved computationally”, ACM et al., 2016) intosecondary education curricul

well-being, and ICT attitudes and competencies). However, the framework also aims to identify how these links depends on contextual factors or background characteristics, and on existing policies and practices related to ICT. Figure 2.1 provides an overview of the underlying logical framework used to elaborate the PISA 2021 ICT framework. 14.