WIXLIB

Module 12 (Concepts): Cross-Classified Multilevel ModelsModule 12: Cross-ClassifiedMultilevel ModelsGeorge LeckieCentre for Multilevel ModellingPre-requisites Modules 1-5,11ContentsWhat are Cross-Classified Multilevel Models?. 3Introduction to the Example Dataset . 5C12.1 Understanding Cross-Classified Data Structures . 9C12.1.1Two-way cross-classified data structures . 9C12.1.2Adding a random interaction classification . 13C12.1.3Three-way cross-classified data structures . 17C12.1.4More complex cross-classified data structures . 20C12.2 A Cross-Classified Variance Components Model . 23C12.2.1Specifying the two-way cross-classified model . 23C12.2.2Interpretation of the intercept and the random effects . 27C12.2.3Testing for cluster effects . 29C12.2.4 Calculating coverage intervals, variance partition coefficients (VPCs)and intraclass correlation coefficients (ICCs) . 33C12.2.5Predicting and examining cluster effects . 38C12.2.6Example: Secondary schools crossed with primary schools . 39C12.3 Adding a Random Interaction Classification . 42C12.4 Adding Predictor Variables . 44C12.4.1Adding level 1 and higher-classification predictor variables . 44C12.4.2Example: Secondary schools crossed with primary schools . 45C12.5 Adding Random Coefficients . 50C12.5.1Adding higher-level random coefficients . 50C12.5.2Example: Secondary schools crossed with primary schools . 52C12.6 Adding Further Classifications . 55C12.6.1A cross-classified model with four higher classifications . 55Centre for Multilevel Modelling, 20131

Module 12 (Concepts): Cross-Classified Multilevel ModelsC12.6.2 Example: Secondary schools crossed with primary schools,neighbourhoods and LAs. 56Further reading . 58References . 59If you find this module helpful and wish to cite it in your research, please use thefollowing citation:Leckie, G. (2013). Cross-Classified Multilevel Models - Concepts. LEMMA VLEModule 12, htmlAddress for correspondence:George LeckieCentre for Multilevel ModellingUniversity of Bristol2 Priory RoadBristol, BS8 1TXUKg.leckie@bristol.ac.ukCentre for Multilevel Modelling, 20132

Module 12 (Concepts): Cross-Classified Multilevel ModelsWhat are Cross-Classified Multilevel Models?In the previous modules we illustrated two- and three-level models for analysinghierarchical data structures whereby lower level units, such as students, arenested within higher level units, such as schools, and where these higher levelunits may in turn be nested within further clusters (or groupings) such as schooldistricts, regions or countries. With hierarchical data structures each lower levelunit belongs to one and only one higher level unit. For example, each studentattends one school, each school is located within one school district and so on.However, social reality is more complicated than this and so social and behaviouraldata often do not follow strict hierarchies. Two types of non-hierarchical datastructures which often appear in practice are cross-classified and multiplemembership structures. In this module, we describe cross-classified data structuresand cross-classified multilevel models which can be used to analyse them. Weleave discussion of multiple membership data structures and multiple membershipmultilevel models until Module 13.In cross-classified data, lower level units do not belong to one and only one higherlevel unit. Rather, lower level units belong to pairs or combinations of higher levelunits formed by crossing two or more higher level classifications with one another.An example in educational research arises in studies of student attainment wherestudents are nested within schools and are separately nested withinneighbourhoods. Schools and neighbourhoods are not typically nested within oneanother as not all students from the same school live in the same neighbourhood,nor do all students from the same neighbourhood attend the same school. Rather,schools and neighbourhoods are crossed with one another, with each studentpotentially belonging to any combination of school and neighbourhood. Studentsare described as nested within the cells of the two-way cross-classification ofschools by neighbourhoods. An example in health services research arises in studiesof hospital patient outcomes. Hospitals and general practitioners (GPs, i.e. familydoctors) are cross-classified as GPs tend to refer their patients to differenthospitals depending on patient need while hospitals typically treat patients whohave been referred by many different GPs. There is nothing to stop data structuresbeing even more complex and having three or more higher classifications and weshall consider examples of such data in this module. Many further examples of crossclassified structures are described in C4.4 of Module 4.It is important to incorporate cross-classified structures in to our models whenthey arise in the data and lead the higher level clusters to differ substantially fromone another on the response variable under study. Naively fitting the nearestequivalent hierarchical model to cross-classified data will lead us to misattributeresponse variation to the included levels (van Landeghem et al., 2005; Moerbeek,2004; van den Noortgate et al., 2005; Tranmer and Steele, 2001). This in turn maylead us to draw misleading conclusions about the relative importance of differentsources of influence on the response. For example, fitting a students-withinschools two-level model of student attainment and ignoring the fact that studentsare simultaneously, but separately, nested within neighbourhoods will likely leadus to overstate the importance of schools as a source of variation in studentattainment. Some of the variation that we attribute to schools may be betterCentre for Multilevel Modelling, 20133

Module 12 (Concepts): Cross-Classified Multilevel Modelscharacterised as neighbourhood-to- neighbourhood differences in attainment. Ournaïve analysis would therefore overstate the importance of schools on studentattainment and would ignore the role of neighbourhoods (i.e. neighbourhoodpolicies, practices, context and compositional effects). Furthermore, byincorrectly modelling the dependency in the data we will likely obtain biasedstandard errors for the predictor variables, particularly those measured at higherlevels. We therefore run the risk of making incorrect inferences and drawingmisleading conclusions about the relationships being studied. For example,including neighbourhood-level predictor variables in our students-within-schoolstwo-level model, but ignoring neighbourhood as a level in the model will typicallylead us to severely underestimate the standard errors on these neighbourhoodlevel variables. When we then go on to test the significance of these variables, wewill run the risk of making type 1 errors of inference.Centre for Multilevel Modelling, 20134

Module 12 (Concepts): Cross-Classified Multilevel ModelsIntroduction to the Example DatasetWe will illustrate multiple membership models in the context of the same schooleffectiveness application which was analysed in Module 11. Readers familiar withthis application may wish to skip the next two paragraphs.In educational research, there is considerable interest in measuring the effectsthat schools have on students’ educational achievements. Measuring the effectsthat schools have on their students is after all a necessary first step to learninghow schools’ policies and practices combine to generate differences betweenschools. Governments are also often interested in measuring school effects,typically for school accountability purposes, but often to also provide parents withinformation to help guide school choice. However, in nearly all education systems,there are substantial differences between schools in their students’ attainments atintake (i.e. when students first arrive at their schools). For the purposes ofresearching the effects of schools’ policies and practices, holding schoolsaccountable, or informing school choice, schools should not be compared simply interms of their average exam results as these differences will, at least in part, bedriven by these initial differences.Traditional studies of school effects attempt to measure the ‘true’ effects thatschools have on their students by fitting two-level students-within-schoolsmultilevel models to students’ exam scores where covariate adjustments are madefor students’ initial scores, and typically for a range of other student backgroundcharacteristics. The school-level residuals from these models are then argued tomeasure the effects that schools have on their students having adjusted for thenon random selection of students into schools. These effects are interpreted asmeasuring the influences schools have on their students’ academic progress(improvement or change in attainment) while they attend their schools. In schooleffectiveness research these influences are referred to as ‘value-added’ effects.In Module 11, we used three-level models to explore the stability of school effects(across cohorts) on students’ academic progress during secondary schooling in theEnglish education system. However, it is easy to think of further sources ofclustering or influence on child learning which might also be important to considerand explore. One interesting example is the role that schools attended in anearlier phase of education may continue to exert on students after they have leftthese schools. For example, we might ask: Does primary school attended (ages 4 to11) predict student academic progress during secondary schooling (ages 11 to 16)?Such long lasting or continued effects of schools attended in earlier phases ofschooling are sometimes referred to as carry over school effects. However, as notall students from the same primary school typically go on to attend the samesecondary school, the data can no longer be described as hierarchical. Rather, thedata are described as cross-classified with students (level 1) nested within thecells of a cross-classification of secondary schools and primary schools (bothconceptually at level 2). In this module, we shall introduce cross-classifiedmultilevel models to analyse the potential role of primary schools on students’progress during secondary schooling in the English education system.Centre for Multilevel Modelling, 20135