WIXLIB

oOn an annual basis, test results and low income data shall be used todetermine the distribution of available state funds to supportremedial esistance prograrv.The purpose of this report is to provide an overview and summary of theThe mastery testimplementation of the fourtv-grue Connecticut Mastery Test.assesses how well each student 1:: performing on those skills identified bycontent experts and practicing oduc:Iwrb as important for students enteringfourth grade to have mastered.

FOREWORDThe Connecticut Mastery Test is a critical element in Connecticut's agenda toattain educational equity and excellence.The testing program assessesessential skills in mathematics and language arts, including listening,reading and writing, for grades four, six and eight students. Studentachievement is measured and reported in relation to specific learningobjectives that students reasonably can be expected to have mastered by theend of grades three, five and seven.The Connecticut Mastery Test provides valuable educational information whichcan be used to improve instruction and elevate the achievement ofConnecticut's students.The test results are reported in a manner thatidentifies how well each student is succeeding in relation to clearly definedand meaningful standards.It is my hope that educators throughout the stateuse the results as a tool to gain a better understanding of the learningoccurring in our classrooms and the ways to increase learning in the ruture.Connecticut is committed to an annual cycle of assessment in order to promote:othe monitoring of individual student achievement;othe evaluation of instructional program effectiveness;oeducational goal setting; andoremedial assistance program improvement.I encourage you to carefully review the mastery test results provided at thestudent, classroom and district levels.The Department is prepared to assistlocul school districts in the areas of curriculum and professional developmentand test interpretation.Gerald N. TirozziCommissioner of Education

OVERVIEW OF THE MASTERY TESTING PROGRAMIn the spring of 1984, the Connecticut General Assembly amended the EducationEvaluation and Remedial Assistance (EERA) legislation to authorize thecreation of mastery tests in the basic skill areas of mathematics and languagearts, including listening, reading and writing skills.The tests were to beestablished for grades four, six and eight.The goals of the mastery testing program are:oearlier identification of students needing remedial educatIon;otesting a more comprehensive range of academic skills;osetting high expectations and standards for student achievement;omore useful test achievement information about students, schools anddistricts;oimproved assessment of suitable equal educational opportunities; andocontinual monitoring of students in grades four, six and eight.The type of test that best addresses these goals is a criterion-referencedtest.Criterion-referenced tests are designed to assess the specific skilllevels of students. Such tests usually cover relatively small units orcontent.Their scores have meaning in terms of what each student knows or cando.Test results are used to identify the areas of strengths and weaknessesof each student.MASTERY TEST CONTENTThe CMT is designed to assess essential language arts/reading, writing andmathematics skills that can reasonably be expected to be mastered by moststudents by the end of the third, fif:h and seventh grades.The specificskills to be tested within these content areas were identified by committeesof educators from throughout the state.In addition, surveys were sent tomany teachers, administrators and parents to determine the appropriateness ofthese skills for the Mastery Test. A complete description of the proceduresused in the development of the fourth-grade CMT can be found in Appendix A(p. 31).MathematicsThe Mathematics Advisory Committee recommended a grade four mathematics testthat assessed twenty-five (25) specific objectives in four domains:(1) Conceptual Understanding; (2) Computational Skills; (3) ProblemSolving/Applications; and (4) Measurement/Geometry.There are four 'est itemsper objective for a total of 100 items on the mathematics test. A detailedlist of domains and objectives is given in Appendix B (p. 35).Language ArtsThe Language Arts Advisory Committee recommendedl',3-item grade fourlanguage arts test that covers two domains: Readi1i".14'ening andWriting/Locating Information. Nine (9) objectives were recommended by theLanguage Arts Advisory Committee.

The general content of Reading/Listening consisted of narrative, expositoryand persuasive passages on a variety of topics measuring a student's abilityin: (1) Literal Comprehension; (2) Inferential Comprehension; and (3)Audiotapes were used to assess students' listeningEvaluative Comprehension.comprehension ability in: (1) Literal Comprehension and (2) Inferential andThe Degrees of Reading Power (DRP) test was alsoEvaluative Comprehension.TheDRP test included eight (8) passages andused to assess reading.fiUty-six (56) test items. It was designed to measure a student's ability tounderstand nonfiction English prose at different levels of reading difficulty.The general content area of Writing/Locating Information consisted of threeFirst, there was a writing sample for direct, holistic assessmentcomponents.Each student was asked to write a composition on aof student writing.Writingwas then judged on a student's demonstrated abilitydesignated topic.Second, theto convey information in a coherent and organized fashion.mechanics of good writing, which was defined as (1) Capitalization andPunctuation, (2) Spelling, Homonyms and Abbreviations and (3) Agreement, wasassessed in a multiple-choice format. Third, Locating Information (Srhedules,Maps, Index and Reference Use and Dictionary Meaning), measured students'A detailed listability to find and use information from the sources listed.with objectives and number of items per objective is given in Appendix C(p. 37).FUTURE DEVELOPMENTThe Connecticut State Department of Education (CSDE), in conjunction withcontent consultants and various CMT advisory committe.rs, has begun theMP current CMT is underdevelopment of the second generation of the CMT.review to determine which skills are appropriate for inclusion on the newIn addition, new content areas and other forms of assessment techniquestest.(e.g., performanLe assessment and short-answer questions) are beingIt is anticipated that the second generation CMT will beconsidered.Items for thisadministered for the first time statewide in the fall of 1993.set of exams will initially be piloted in the fall of 1991 followed by asecond pilot in the fall of 1992.SETTING MASTERY STANDARDS BY OBJECTIVEThe essence of the Connecticut Mastery Test (CMT) is the establishment of aspecific mastery standard against which each student's knowledge andThe mastery test incorporatescompetency on each objective can be compared.appropriate and challenging expectations for Connecticut public schoolThe goal of the CMT Program is for each student to achieve masterystudents.The objectives being tested were identified as appropriateof all objectives.and reasonable for students at each of the grades tested.Mese tests aredesigned to measure a student's performance on these specific objectives.The process of establishing the mastery standards by objective used aThe firststatistical method that required two decisions to be utilized.decision defined a student who mastered a particular skill as one who had a95% chance of correctly answering each item within the objective. The seconddecision wa that the specific standard for each objective would identify 99%

of the students who mastered the skill. By applying the two decision rulesstated above to a binomial distribution table, mastery standards wereestablished for the 25 matheratics objectives and the 9 language artsobjectives.The mastery standards are as follows:oIn mathematics, for each of the 25 objectives, a student must answercorrectly at least 3 out of 4 items.oIn language arts, for the 9 multiple-choice objectives with varyingnwibers of items, a student must answer correctly the followingnumbers of items:# Items Correctfar MasteryWRITING MECHANICS(1) Capitalization & Punctuation(2) Spelling(3) Agreement9 out of 127 out of 911 out of 15LOCATING INFORMATION(4) Schedules, Maps, Table of Contents,Title Page and Dictionary8 out of 11LISTENING COMPREHENSION(5) Literal(6) Inferential and Evaluative5 out of 79 out of 13READING COMPREHENSION(7) Literal(8) Inferential(9) Evaluative9 out of 1210 out of 147 out of 10No mastery standards were set for the two holistic language arts measures,neither the Degrees of Reading Power (DRP) test nor the Writing Sample, sincethese measures are not composed of objectives on which mastery could beassessed.SETTING REMEDIAL (GRANT) STANDARDSIn addition to mastery standards, Section 10-14 m-r of the Connecticut GeneralStatutes requires that the Connecticut State Board of Education establishstatewide standardI for remedial assistance in order to meet tworesponsibilities:oto identify and monitor the progress of students in need of remedialassistance in language arts/reading and mathematics as part of theEERA field assessments; and

oto distribute EERA funds based on the number of needy studentsstatewide, as well as for use in the Chapter 2 and Priority SchoolDistrict Grants.Students who score below the remedial standard(s) are eligible for servicesprovided for in EERA legislation. Remedial standards were established by theState Board of Education acting on the recommendations of committees that'he standard-settingrepresented Connecticut citizens and educators.committees recommended the following remedial standards:1.In mathematic?, a student who answers fewer than 69 of the 100 items(697.) correctly is required to receive further diagnosis by the localschool district and, if necessary, to be provided with remedialassistance.2.In reading, a student whose Degrees of Reading Power (DRP) unit scoreis lower than 41 is required to receive further diagnosis and, ifnecessary, to be provided with remedial assistance.3.In writing, a student receiving a total holistic score less than 4 isrequired to receive further diagnosis by the local school districtand, if necessary, to be provided with remedial assistance.The mastery and remedial standards were established by the State Board ofEducation on June 23, 1985. For a detailed explanation of the remedialstandard-setting process, see Appendix D (p. 39),STATEWIDE ACHIEVEMENT GOALSIn addition to mastery and remedial standards, statewide achievement goalshave been established in the content areas of mathematics, reading (DRP) andwriting.These goals represent high expectations and high levels ofachievement for Connecticut public school students.The achievement goals are as follows:oIn mathematics, all students must master 22 of 25 objectives tested.oIn reading, a student must score a Degree of Reading Power (DRP) unitscore of 50 with 707. comprehension.oIn writing, a student must score a total holistic score of 7 on ascale of 2 to 8.STUDENT GROWTH OVER TIMEThe Connecticut Mastery Test (CMT) program is designed to providecriterion-referenced information about the level of student mastery ofobjectives in grades four, six and eight.However, the basic scores reportedfor the mastery tests do not provide a system for evaluating achievementgrowth from grade four to grade six to grade eight.This is so becausemastery decisions are based on student performance (mastery/non-mastery) on

objectives that are unique to grade level.Mastery of objectives cannot becompared directly across grade levels and tests because of the differences inthe number of objectives, curriculum content and levels of difficulty. Inorder to make valid interpretations across grade levels, the mastery testperformance must first be linked using a procedure called vertical equating.Purpose of Vertical EquatingVertical equating is a psychometric technique for comparing tests at allability levels.This is accomplished by putting them on a new scale which iscommon to the tests.Vertical equating is based on twc assumptions.Thefirst is that learning is continuous. The second is that instruction in eacharea is related to increased achievement in that area.These assumptionsenable test developers to create a scale score that covers a wide range ofcontent over several grades.The type of equating that leads to thedevelopment of t4ke "growth scales" is known as vertical equating.Thedevelopment of growth scales is a common practice and has been usedsuccessfully in the development of a variety of achievement test batteries.The purpose of vertical equating is to provide one scale score system whichcan be used to compare performance across multiple grade levels. This scoresystem enables test users to interpret test score information over timewithout altering the basic nature of the testing program.This achievementgrowth can be monitored over time on the basis of student performance on theCMT across grades.Development of Vertical ScalesIn order to develop a vertical scale, pelformance on the grade four, grade sixand grade eight mastery tests was statistically linked.This was accomplishedduring the 1987 administration of the CMT using representative statewidesamples of approximately 5,000 sixth-grade students and approximately 7,000eighth-grade students. Each group of students at grade six and grade eightwas administered the appropriate on-grade level test form of the CMT alongwith one below-grade level section of the CMT. Specifically, each group ofeighth-grade students took the grade eight test as usual and a part of thegrade six test.Likewise, each sixth-cjrade group took the grade six test asusual along with a section of the grade four test.Each sample of studentstook only one below-level section of the CMT involving approximately one hourof additional testing time.Performance on the below-level items was notcounted toward the CMT scores of individual students.For each of theselinking samples, item difficulty estimates were obtained for the on-grade andbelow-grade level items by analyzing all items together as one test. Onceitems from the on-grade and below-grade level tests were linked, itemdifficulties from each level of the CMT were adjusted to a common metric toproduce the vertical scale.Vertical scales were established in the content areas of mathematics and thereading comprehension section of the language arts test.For each grade andcontent area, every correct score corresponds to a specir:'. value on a commonscore scale (vertical scale).Each of the vertical scales was constructed sothat each scale score point represents the same theoretical achievement levelwhether derived from a score on the grade four test, a score on the grade six

test or a score on the grade eight test. This allows valid interpretations ofgrowth across time using tests differing in content, length and itemdifficulty. All items on the mathematics and reading comprehension tests wereused in the development of the vertical scales. The writing and language artstests were not scaled because of the nature of these assessment processes.The Degrees of Reading Power (DRP) test employs DRP unit scores which arealready on a common scale across grades, obviating the need for any other(For more information see Congero, W.J., 1989, The Developmentdevelopment.of Vertical Scales to Enhance the Evaluation of Assessment Data. Paperpresented at the annual conference of the National Council of Measurement inThis paper is available through the StudentEducation, San Francisco, CA.Assessment and Testing Unit of the Bureau of Evaluation and StudentAssessment.)Scaled scores can be used to measure growth over time because CMT scores fromThese scalesall three grade levels have been placed on a common scale.provide a means of monitoring students' academic progress from grade toBefore the scales were developed, it was difficult to assess thegrade.performance of groups of test takers as they moved from grade to grade becauseof differences in test length, curriculum content covered and levels ofdifficulty on the fourth-, sixth- and eighth-grade tests.Since students who took the fourth-grade test in 1987 subsequently took thesixth-grade test in 1989, change in test performance can be assessed acrossSimilarly, change in performance can be assessed for 1990two years' time.sixth graders who took the grade four test in 1988. A summary of the overallgrowth in performance for these two groups of students in the content areas ofmathematics and reading comprehension can be found in the 1990-91 Grade 6Students who took the fourth-grade testSummary and Interpretations Manual.in 1985 subsequently took the sixth-grade test in 1987 and the eighth-gradetest in 1989. Similarly, students who took the fourth-grade test in 1986subsequently took the sixth-grade test in 1988 and the eighth-grade test inA summary of the overall growth in performance for these groups of1990.students in the .'r.tent areas of mathematics and reading comprehension can befound in the 1990-91 Grade 8 Summary and Interpretations Manual.NORMATIVE INFORMATIONThe CMT program is designed to provide detailed information about fourth-,sixth- and eighth-grade students' mastery of specific skills and objectives.The provision of national norms with CMT results is intended to enhance theusefulness and flexibility of mastery test information by offering a bridge toconventional norm-referenced testing programs. The decision to providenormative information with the CMT does not change the essential purposes ofour criterion-referenced testing program. The CMT will continue to be usedfor diagnostic and other instructional purposes with results reported at thestudent, classroom, school, district and state levels.In particular, national norms provide greater:Test Economy.By providing national norms with CMT r

an act concerning Education Evaluation and Remedial Assistance (URA). This law provides that: o. By May 1, 1985, each local or regional board of education shall. have. developed and submitted for State Board of Education approval, a new. plan of educational evaluation and remedial assistance. Each plan had to address the following: o