Transcription
Research Report DFE-RR108(a)Evaluation ofevery child awriter: report 1Ros Fisher and Liz TwistUniversity of ExeterNational Foundation for EducationalResearch
This research report was commissioned before the new UK Government tookoffice on 11 May 2010. As a result the content may not reflect current Governmentpolicy and may make reference to the Department for Children, Schools andFamilies (DCSF) which has now been replaced by the Department for Education(DFE).The views expressed in this report are the authors’ and do not necessarily reflectthose of the Department for Education.
AcknowledgementsWe would like to express our gratitude to all participating staff, parents and pupils in the localauthorities and schools involved for their willingness to take part in the research either byproviding data, completing surveys or being involved as a case study. Without them thisresearch would not have been possible.This study would also not have been possible without the involvement of a number ofresearchers and we are grateful to them for their participation and insights. Dr CarolineJenkinson at the University of Exeter was the key researcher responsible for data collectionin the case studies.A number of secretaries have supported the work of this project over the three years.Special thanks are due to Nicola Keogh at NFER and Meridith Griffin at the University ofExeter.We have received valuable feedback on data instruments and draft reports from members ofthe DCSF steering group, in particular, Emma Rogers. We are grateful to our projectmanager, Konstantina Dimou for her support, understanding and flexibility throughout thelifespan of this project.Research teamUniversity of ExeterDr Ros FisherProfessor Debra MyhillDr Caroline JenkinsonMeridith GriffinProject Director and ResearcherResearcherResearch FellowAdministrative assistantNational Foundation for Educational ResearchLiz TwistResearcherLouise CooperResearcherHelen FrancisResearch Data ServicesRose McClureResearch Data ServicesJaswinder AthwalData captureNicola KeoghProject administratorFrancesca SaltiniStatisticianDr Ben StylesStatisticianContact detailsDr Ros Fisher: R.J. Fisher@Exeter.ac.uk2
Table of ContentsExecutive summary11.11.21.31.422.12.22.32.45Overview of the Study14IntroductionThe Policy and Research Context1.2.1 Pupils’ progress in English1.2.2 The effectiveness of literacy interventions1.2.3 The importance of literacy1.2.4 The process of writing1.2.5 The social aspect of writing1.2.6 Every Child a WriterAims and objectives of the research study1.3.1 The impact of involvement in Every Child a Writer1.3.2 The delivery of Every Child a Writer1.3.3 Perceptions of cost effectivenessMethodology1.4.1 Quantitative strand1.4.2 Qualitative strand1.4.3 Perceptions of cost effectiveness1.4.4 Limitations of the ummaryOn Standards2.2.1 Has ECaW raised standards of writing in schools, and if so to whatextent?2.2.2 What has been the impact of one-to-one tuition for selected pupils,and what is the impact on other pupils in Years 3 and 4?2.2.3 Are any subgroup variations observable?2.2.4 Perceptions of impactOn Teaching2.3.1 Has ECaW improved the quality of pedagogy of the whole classteaching in schools that took part in the programme?2.3.2 Has ECaW had an impact on teacher confidence and practices?2.3.3 What impact has ECaW had on the prevalence and effectiveness ofguided writing?2.3.4 What impact has ECaW had on the use of available teachingmaterials for writing, underpinned by use of Assessing Pupils’ Progress(APP)?Pupil attitudes and perceptions2.4.1 Has ECaW had an impact on pupil’s attitudes to writing, theirconfidence, and their perception of themselves as writers?324242426262731313240414444
2.532.4.2 Do pupils feel their writing has improved, and if so how?2.4.3 Are any subgroup variations observable?Across the school2.5.1 Is there an identifiable impact of ECaW more broadly across .113.123.133.143.15450SummaryHow have Leading Teachers worked in their own schools and in theschools they support?Do they feel adequately supported?What characterises good and bad practice by Leading Teachers?What are the conditions that support or hinder their work?What are the characteristics of tutors?How do teachers view the support they have received from LeadingTeachers?How do they view the classroom materials provided through ECaW?What are their perceptions of the value of ECaW?What delivery challenges have schools and local authoritiesencountered, and how have they addressed them?3.10.1 Information flow3.10.2 Teacher workload3.10.3 Factors impeding the delivery of ECaWHas one-to-one tuition been delivered effectively?How have schools managed the process of appointing tutors?What role has the named Local Authority Lead Consultant for ECaWplayed?3.13.1 What characterises effective/ineffective practice?3.13.2 What are their perceptions of the value of ECaW?How have schools engaged with parents and carers, and to whateffect?How do schools plan to sustain any improvements made as a resultof ECaW?Perceived cost 717273757678SummaryIs ECaW a cost effective way of raising pupils’ attainment in writingand in English overall at act5.2.1 On standards8181814
5.35.45.55.65.75.2.2 On teaching5.2.3 On pupil attitudes and perceptions5.2.4 Across the schoolDelivery5.3.1 Leading teachers as a model for professional development5.3.2 One-to-one tuition5.3.3 Local authority involvementPerceptions of cost effectivenessImplications for writing pedagogy5.5.1 Classroom observation5.5.2 Writing samplesMajor enablers and challenges5.6.1 Enablers5.6.2 ChallengesChanges to programme and delivery that may maximise impact5828283838485858585858787878889
AppendicesAppendix 1Appendix 2Appendix 3Appendix 4Appendix 5Appendix 6ReferencesAttainment and attitude modellingResearch methodologyPupil questionnaire dataSPSS modelling outputCase study methodology9193107111135138List of Tables and FiguresTable 1.1Table 2.1Table 2.2Table 2.3Table 2.4Table 2.5Table 2.6Table 2.7Table 3.1Table 3.2Table 3.3Table 3.4Figure 2.1Figure 2.2Figure 2.3Figure 2.4Figure 2.5Figure 2.6Questionnaire response ratesPerceived impact of ECaW on writing attainmentPerceived impact of ECaW on attainment of children receiving oneto-one tuitionPerceived impact of ECaW on writing confidence and enjoyment ofchildren receiving one-to-one tuitionTeachers reporting increased subject knowledge in specific areasTeachers’ reported use of resources in the teaching of writingPerceived impact of ECaW on children’s enjoyment and confidencein writingPerceived impact of ECaW across the schoolSupport received from Leading TeachersLeading Teachers’ perceptions of adequacy of support forSupported TeachersLA Lead Consultants’ perception of the demands of participation inECaWLA Lead Consultants’ evaluation of ECaW componentsProgress in writing as predicted by the main modelECaW teachers’ confidence in teaching modes of English, phase 1and phase 2 questionnairesSupported Teachers’ reported frequency of use of differentteaching strategiesLeading Teachers’ reported frequency of use of different teachingstrategiesReported time spent on teaching writing strategies in an averageweek (phase 2)Perceptions of teachers reporting changes in their 373839
Executive SummaryIntroductionIn 2008, the University of Exeter and the National Foundation for Educational Research werecommissioned by the then Department for Children, Schools and Families (DCSF) nowDepartment for Education, to conduct an evaluation of the Every Child a Writer initiative. Thestudy employed quantitative and qualitative methods to evaluate impact and explore processand practice over the second year of the project.AimsThe research aimed to evaluate:The impact of involvement in Every Child a Writer (ECaW) on standards of writing in theschools: on teaching, of both class teachers and one-to-one tutors; on pupils’ attitudes andperceptions; and on the whole school.The delivery of ECaW including investigating processes which supported/hindered theeffectiveness of ECaW, and identify features of effective and ineffective practice.Perceptions of cost effectiveness.MethodologyThe research involved a sophisticated blend of quantitative and qualitative researchmethodologies. The mixed methodology research design adopted sought to address thespecific aims of the research. There were essentially four major strands to the researchdesign: a quantitative analysis of the impact of the initiative on pupils’ progress in writingusing a quasi-experimental design with a comparison sample of pupils in schoolsnot involved in ECaW a quantitative analysis of questionnaires completed by teachers, headteachers,pupils and Local Authority Lead Consultants a series of case studies exploring the context in which ECaW is being introducedand the perceptions of the initiative and its impact on those involved a qualitative study of the writing produced by a sample of pupils involved inECaW.7
Quantitative strandThe quantitative element of the evaluation of ECaW comprised three main elements: the collection and analysis, including multilevel modelling, of teacher assessmentdata concerning pupils’ writing skills in both ECaW and comparison schools a series of questionnaires – collecting information about teachers’, headteachers’and lead ECaW consultants’ perceptions of the ECaW programme and its impactor anticipated impact on teaching strategies and on pupils’ attainment a pupil questionnaire to both ECaW and comparison schools – collectinginformation about pupils’ perceptions of themselves as writers and their attitudesto writingTeacher questionnaires were administered at two time points during the academic year2009/10. In November 2009, four different questionnaires were administered, distinguishingbetween recipients in schools who took part in ECaW in 2008/09 (cohort 1) and those inschools who began ECaW in 2009/10 (cohort 2), and between Supported Teachers andLeading Teachers. In June/July 2010, questionnaires were again sent to teachers involved inthe project. There were two different questionnaires, one for Supported Teachers and onefor Leading Teachers. Questionnaires were also sent to headteachers and Local Authority(LA) Lead Consultants in June/July 2010.Qualitative strandCentral to the qualitative strand were ten case studies of both cohort 1 (five) and cohort 2(five) schools. Each case study focused on one LA. Selection of case study schools providedas broad as possible a range of characteristics. These characteristics cover demographicssuch as geographical location as well as economic and social background; schools makingexceptional progress as well as those experiencing difficulties; schools with the LeadingTeacher based in the school and those where this is not the case. Data included interviewswith LA staff, teachers, one-to-one tutors, parent focus group and pupils; classroomobservation of literacy teaching; and analysis of writing samples from pupils in case studyschools. The case studies were conducted in the spring and summer terms of 2010.8
Key FindingsImpactImpact on standards Statistical analysis of pupils’ attainment data showed that the rate of progress inwriting in ECaW schools was no greater than that in comparison schools. The perception of staff in ECaW schools was that ECaW had had a positiveimpact on the attainment and confidence, at least, of pupils receiving one-to-onetuition. There was some evidence that staff see the impact on pupils as broader than thecriteria used in national curriculum teacher assessment. This could be becauseelements that may support longer term gains such as increased confidence andenthusiasm are not measured in national curriculum assessment. Furthermore,ECaW focuses on sentence construction and text cohesion which form only onepart of the teacher assessments of writing.Impact on teaching Perceptions of staff in ECaW schools were that ECaW had had a positive effecton teachers’ practice and confidence. Recently qualified teachers in particularfound the extra support and contact with colleagues from outside their ownschools very supportive. Leading Teachers also found the experience of benefitto their professional development both in relation to the teaching of writing and toleadership experience. A key aspect of impact was that Supported Teachers had been encouraged toplan their teaching according to the needs of their pupils as indicated by thewriting that they produce rather than according to some external programme. ECaW had improved access to materials that support planning and assessmentsuch as Assessing Pupils’ Progress (APP) for Supported Teachers. However,evidence from the observation of teaching and examination of the writingsamples indicated that some teachers did not follow their plan but resorted towell established routines. There was evidence of increased and improved use of guided writing in ECaWclasses. Based on research evidence into the teaching of writing, it is likely thatimproved use of guided writing targeted on the needs of pupils (rather than onthe level they have attained) will have a positive impact when it becomes wellestablished. Some teachers were still reluctant to work regularly on guidedwriting at the expense of overseeing the whole class as they write.9
There was evidence that some teachers focused more on encouraging pupils toinclude particular grammatical features such as connectives, adjectives, etc intheir writing rather than focusing on how these are used to promote meaning andeffect.Impact on pupil attitudes and perceptions There was no evidence of improved attitudes to writing over the year of theevaluation of pupils in ECaW classes. This is in line with other findings that showattitudes declining as pupils grow older. Nevertheless, staff in schools and in LAs all spoke positively about the impact onenjoyment and confidence of pupils receiving one-to-one tuition.Impact across the school There was little evidence of the impact of ECaW across the school except in afew, mainly Leading Teacher, schools. Although survey data and some case study interviews indicated that staff outsideyears 3 and 4 were informed about ECaW, evidence from the case studyinterviews indicate that this may be limited.DeliveryLeading Teachers as a model for professional development Leading Teachers have found the experience beneficial to themselves and theirown schools. However, many have found the time away from their own class themost difficult aspect. Evidence from the different strands indicates that the positive experience forLeading Teachers was impressive. Headteachers of schools with a Leading Teacher also recognised the benefit ofthe experience of this role for these particular teachers. Data from the headteachers’ questionnaire indicates that 95% of headteacherswere confident in their ability to judge the quality of the teaching of writing.However, LA staff felt that headteachers may overestimate quality, particularlywhen it relates to the more specialised area of teaching writing. On the whole effective relationships have been established between LeadingTeachers and teachers they support. This was seen to be crucial to the successof the initiative.10
Most teachers were positive about the impact of the initiative on their ownteaching and professional confidence. There was some concern about the practicality of increasing the number ofschools involved and finding good quality Leading Teachers to support them.The selection of Leading Teachers who have a good understanding of theprocess and skills of writing is essential and may be a threat to the continued rollout of the initiative. A major challenge to the successful delivery of ECaW identified by teachersrelated mainly to the initial stages of the set up, including training, and the flow ofinformation from the centre to teachers and headteachers. Another potentialhindrance was concern about the amount of time that Leading Teachers spentout of class. ECaW has increased teacher knowledge and use of available materials tosupport the teaching of writing. Opinions varied as to the usefulness of thesematerials with Leading Teachers being most positive. Where teachers alreadyhad their own way of planning and assessment there was some reluctance totake on new ways, particularly when the rest of the school would continue to usethe established way. However, many schools had appreciated the support ofECaW in implementing APP.One-to-one tuition The one-to-one tuition element of ECaW was viewed very positively by allparties. Both headteachers and class teachers reported that one-to-one tuitionhas had a positive impact on individual pupils. There was also a sense that the gains for pupils who were not making goodprogress may be longer term and reach beyond writing skills in that classteachers and parents reported an increased confidence and willingness toparticipate in classroom activities on the part of these pupils. A key finding from the focus group interviews was the fact that, in each of the tenschools, parents who did not have children receiving one-to-one tuition wereunaware of ECaW. Parents/carers whose children had received one-to-one tuition were positiveabout it. They reported that their children were making progress and also, that inalmost every case, they enjoyed the sessions and liked being singled out for it.11
Local authority involvement Local authorities played an important part in the set up of ECaW and in ongoingsupport and monitoring. Limited time was available for support and monitoring of ECaW and some LAstaff expressed concern that the initiative took time and resources from other LAbased programmes.Perceptions of cost effectiveness On the whole participants indicated that they perceived ECaW as providing valuefor money although opinions varied as to which aspect and how much. It was notpossible to evaluate this in any quantifiable way.Classroom teaching Support by Leading Teachers in planning was useful but evidence fromobservations of teaching indicates that further opportunity to discuss lessonsresulting from the planning would be helpful. In only some areas were SupportedTeachers allowed to observe Leading Teachers teaching in their own schools.This modelling of practice could be advantageous. Some teachers were still not fully confident in the planning and organisation ofguided writing. Evidence from classroom observation of some ECaW teachers indicated thattheir subject knowledge of writing is insufficient to support understanding of thepurpose and effect of linguistic features. Resources used in ECaW schools were varied. Leading Teachers encouragedthe use of a range of resources, not all of which supported a developingunderstanding of the purpose of writing as opposed to merely acting as amnemonic. The selection of targets for pupils receiving one-to-one tuition through ECaWwas not always fully discussed with parents and tutors. These targets did notalways focus on the key element of ECaW: sentence construction and cohesion.Evidence from writing samples from ECaW classesThe evidence from the writing samples shows only a snapshot in time. The evaluation hasno evidence of change in practice other than as reported by teachers and others involved inthe school. Summary points below relate to both Leading and Supported Teachers.12
The best writing samples came from lessons where teachers focused onmeaning and communicative effect. Such lessons were observed both fromsupported and leading teachers. Scaffolding was used extensively by teachers. In some cases scaffoldingappeared too strong or ‘supportive’ and may have been limiting student learningand creating over-dependence. Lesson plans and the teacher feedbac
Research Report DFE-RR108(a) Evaluation of. every child a writer: report 1 . Ros. Fisher and. Liz. Twist University of Exeter . National Foundation for Educational Research . This research report was commissioned before the new UK Government took office on 11 May 2010. A
