Transcription
STUDY ON SOCIAL AND EMOTIONAL SKILLS- AN OVERVIEW Dr Miloš KankarašProject ManagerDirectorate for Education and Skills, OECD
Skill development across the life llsCognitive skills LITERACY NUMERACYMixedcognitive & SEskills MENTAL FLEXIBILITY COGNITIVE CONTROL WORKING MEMORYMixedcognitive & SEskills EMPATHY & TRUST EMOTIONAL REGULATION SOCIAL SKILLSSocial &EmotionalSkillsSocial &EmotionalSkillsEarly Learning and Child Well-being(IELS)5 years old CRITICAL THINKING CREATIVITY META-COGNITION THE “BIG FIVE” 15 SKILLSMixedcognitive & SEskillsSocial &EmotionalSkills READING MATH SCIENCE COLLABORATIVE PS GLOBAL COMPETENCE META-COGNITION SELF-EFFICACY SELF-ESTEEM GOAL ORIENTATION CURIOSITYSocial and Emotional Skills(SSES)International StudentAssessment (PISA) & SSES10 years old15 years oldCognitiveskillsMixedcognitive & SEskillsSocial &EmotionalSkills LITERACY NUMERACY PROBLEM SOLVING ADAPTIVE PS META-COGNITIVE STRATEGIES THE “BIG FIVE” 15 SKILLS VOCATIONAL INTERESTSInternational Assessment of AdultCompetencies (PIAAC)16-65 years old
Study topicsSkill-relatedfactors at homeRelevant characteristicsof family environmentAssessment of SEskillsChildren’s social and emotionalskills will be assessedBackgroundcharacteristicsStudents and familybackgroundSkill-relatedfactors in schoolRelevant characteristicsof school and widercommunity environmentLife outcomesAcademic achievementhealth and personalwell-being
Selected skills 19 skills selected forthe initial testing 15 skills will beincluded in the mainstudy Grouped in 5 broaddomains plus“compound” skillsTASKPERFORMANCECRITICAL ATIONTHE‘BIG STCO-OPERATIONENGAGINGWITH OTHERSOPENMINDEDNESS
Our Formula For Data QualityCombining methods1. Students’ reports2. Reports by informed others3. Behavioural indicators4. Administrative data5. Anchoring vignettesCombining sourcesInformation from:1. Students2. Parents3. Teachers4. PrincipalsCombining contexts1. Family environment2. School environment3. Peer environment4. Wider community
Contextual informationChildren Socio-demographicbackground Daily activities Relations with parents Relations with peers Personal well-being School life Perceptions of socialand emotional skillsParents Family background Home environment Parents skills and wellbeing Parent-child relations Parental styles Parent’s attitudes andopinionsTeachers Teachers’ background Teaching pedagogicalpractices School climate Role of social andemotional skills inteachers’ educationand work practices Perceptions of socialand emotional skillsPrincipals School structure andorganisation Student body andteachers School resources School climate Role of social andemotional skills in schoolprogramme Principal’s attitudes andopinions
PART A: ContextualquestionnairePART B:IndirectAssessment assessmentof sParentsSTRUCTURE OFINSTRUMENTSStudentsStructure of instruments
Direct Assessment instruments Self-reports on typical behaviour Main characteristics:– Respondents report about their usual or typical behaviours,perceptions and feelings– Each construct/skill is assessed through the use of multiple“indicators” or “items” – statements of typical behaviour– Information from all indicators/items is used to create one overallscore per respondent– Fast, online administration
TRIANGULATION Three different, and mutually independent sources of information aboutstudents' SE skills: Students, parents and teachers Direct assessment: students’ reports Indirect assessment: reports by parents and teachers– Found to be reliable and valid methods– Found to add new information to one another Possibility of mutual validation and increased reliability of estimates ofstudents’ SE skills Efficient: use of the same instruments and items across the three informants
ANCHORING VIGNETES Innovative assessment option Used for improving the cross-cultural comparability “Reference bias” can exist across nations and cultures Anchoring vignettes provide common reference point AV ratings are used to recalibrate results acrosscultures based on this common standard
CompletelyagreeAgreeNeither agreenor disagreeDisagreeCompletelydisagreeANCHORING VIGNETTES:EXAMPLE FOR CONSCIENTIOUSNESS [Mia] studies hard and gets very good grades. She always does herhomework, finishes her assignments, and is never late to class.[Mia] is a hard-working person.123450[Peter] usually gets good grades. He sometimes has trouble payingattention in class, but usually completes his homework on time.[Peter] is a hard-working person.12345-[Tom] often forgets to do his homework, is sometimes late to class,and does not care about his grades.[Tom] is a hard-working person.12345Vignette statements
BEHAVIOURAL INDICATORS Concrete behaviours that are characteristic of personswith particular skills Provide more real-life context for SE skills Easy to understand and to relate to by policy makers andnon-experts Useful for testing and improving the validity of direct andindirect assessment measures Interesting possibility of comparing concrete behavioursacross contexts/informants
Sample selectionTarget populationTwo age groups of school students:10-year-olds and 15-year-oldsSample size3,000 students for each of the two agegroups in each participating city or countrySample design1. Initial random selection of schools2. A follow-up random selection of studentswithin selected schools
FINAL LIST OF PARTICIPANTS Final list of participants confirmed:– 11 participants from 4 continents– students speaking 11 languages, including 3 minoritylanguages French, Swedish and Spanish– 1st & 2nd languages for 4 billion people (would needadaptations in some cases)– 3 sites using two language forms (Ottawa, Houston & Helsinki)– 3 sites using two Spanish language forms (dialects)
Participating cities and countriesSintra, PortugalHelsinki, FinlandOttawa, CanadaMoscow, RussiaHouston, USARome, ItalyDaegu, South KoreaManizales, ColombiaBogota, ColombiaTurkeySuzhou, China
Additional informationOnline administrationBoth cities and countriesCentrally managed online platform forassessment of students’ SE skillsThe study will include a diverse list of participants,including different administrative units/policy levelsClass-based administrationUse of administrative dataPossible administration of directassessment at the class levelExisting administrative data could be usefulas behavioral correlates and outcomesFollow-up of childrenAnchoring vignettesCities and countries can decide tofollow-up children and havepossibility of longitudinal data in thenext waveAnchoring vignettes will be used to improvecomparability of results across cultures
Phases of the study2016/2017Design &preparationConceptual frameworks,survey documentation,instrument development2018Field test2018Item trialsFull test of instrumentsand study procedures.Testing of initial set ofdirect and indirect setof items.v2019Main study2020Analysis &ReportingDelivery of the mainstudyData analyses andreport drafting
KEY DOCUMENTS PREPARED Conceptual framework - published:– Defining the concepts, their internal structure, development/malleability,cross-cultural comparability Assessment framework – updated draft prepared:––––Why is this study needed/what are objectives:What are study topics?How will factors of interest be measured?Who will conduct the study and when will the study be conducted? Technical Standards – Field Test version shared with participants:– Defining procedures, methodologies and standards that will guide studyimplementation
FOUR EMPIRICAL PHASES COMPLETED Cognitive interviews conducted in December 2017 Online study of parents conducted in February2018 Item Trials administered in April/May 2018 in 6participating sites – 300 students per cohort Field Test being completed - 500 students percohort
Contact usDirectorate for education and skillsVisit our yOnSocialAndEmotionalSkills.htmStudy on Social and Emotional SkillsInformation about the StudyStudy brochuresStudy documentsMilos.Kankaras@oecd.orgRelated OECD projects and publicationsNews and events
assessment of students' SE skills Both cities and countries The study will include a diverse list of participants, including different administrative units/policy levels Use of administrative data Existing administrative data could be useful as behavioral correlates and outcomes
