Transcription
NGSS Large Scale AssessmentAlignment:Expectations, Challenges, andSolutionsJune 26, 2019CCSSO NCSA
PresentersApril McCrae – Delaware Department of EducationTJ Smolek – Michigan Department of EducationSara Cooper – Nebraska Department of EducationKevin King – WestEd
IntroductionHow well can large scale assessment measure what is valuedabout the NGSS? Alignment in the context of NGSSLeverage states’ experience in developing NGSS alignedassessments to advance NGSS alignment thinking.3
Delaware Assessment
Delaware’s Philosophy
Delaware Science Assessment
Delaware’s PhilosophyDriving Claim: Students can make sense of a novel phenomenon by accurately engaging the threedimensions of scientific inquiry: disciplinary core ideas, practices and crosscutting concepts.
Item and Item Cluster AlignmentRequirements: All items must be aligned to at least 2 of the 3 dimensions(Exception: ER must be 3-D) Items are limited by the assessment boundary statementassociated with a PE, when present. Item clusters are typically developed to align to a PE bundle of 2–3PEs. The items that compose an item cluster must, as a set of items,include alignments to all three dimensions of each PE in the PEbundle. Additional Science and Engineering Practices (SEPs) and CrosscuttingConcepts (CCCs), beyond those specified in the PEs in the PE bundle, may beincluded in the alignment of items in an item cluster.
Item and Item Cluster Alignment Alignment begins at the PE level. At the dimension level, the sub-bullets (elements) of the specific dimensionsrepresented in a PE should be the basis of alignment decisions. Dimension decisions should also be informed by the progressions documents(NGSS appendices E, F, and G), SEPs and CCCs, outside the specified PEs, can—and, in many cases, should—be included in the alignment. Any additional alignment(s) should be capturedas metadata.
Item Organizations For Summative Assessments
Alignment Considerations
Michigan Assessment
Opportunity toLearnforTeachersOpportunity toLearnforStudents
Evidence Centered Design:Michigan Science Assessment ClaimsEquity Claim: Non-dominant and dominant groups of students have the opportunity todemonstrate grade band proficiency through the use of engineering, local contexts, andrelevant phenomena.Scientific Literacy Claim: Students demonstrate grade band proficiency in using the threedimensions to critically evaluate scientific and technological information in order todesign solutions to problems and investigate phenomena.Student Level Claim:Student has demonstrated gradeband proficiency in:Life Science,Earth Science, &Physical ScienceTopic Bundles using all dimensionsrepresented in the standards.District/State Level Claim:Students have demonstrated gradeband proficiency to explain thepresented phenomenon (local orglobal) and design solutions toproblems using all dimensionsrepresented in the given topicbundle.
Example High School Topic BundleSEPDCICCCDev/Use ModelsESS1.A Universe and StarsPatternsConstruct ExplanationsESS1.B Earth and Solar SystemScale, Prop. Quant.CommunicateMath and Comp Thk.Systems/Sytm. Models
Example Item Cluster MapPhenomenonTopic BundleStructure and Properties of MatterStimulus5-LS1-1LS1.CSEP 7CC 5Item 1LS1.CPS3.DCC 55-LS2-1LS2.ASEP 2CC 4ItemClusterItem 2LS2.BCC 5SEP 2LS1.CSEP 2Item 35-PS3-1LS2.APS3.DSEP 2CC 5Item 4LS1.CCC 4LS2.BSEP 7Item 5CC 4
Michigan Alignment Considerations Alignment is at the Topic Bundle Level All items must be at least 2D All Stimuli must present a phenomenon Alignment is not bound to PEs
2020 Operational Test Multiple forms per grade (5, 8, 11) 7-8 Item Clusters per form 2-Physical ScienceOperational 2- Earth Science 2- Life Science 1-2 – Field Test Item Cluster(s) Estimated Time: 10-15 minutes per cluster
Questions for StatesQ&A focused on alignment to clarify the states’ contexts.19
Foundational Alignment Questions1. What is the overarching claim for the test?2. What are we aligning to?3. What is the role of an item specificationsdocument?4. What is the question that we are asking todetermine alignment?5. What is the process of making analignment judgment?20What dowe want tomeasure?Does thestudents’responseprovideevidence?Is the evidencesufficient tomeasure whatwas intended?
General Alignment QuestionsWhat dowe wanttomeasure?1. What is the overarching claim for the test?2. What are we aligning to?3. What is the role of an item specifications document?Delaware21MichiganBlueprintsElem 3 clusters 10 SAI;Secondary 4 clusters 14 SAI6 Clusters 1-2 FT ClustersItem SpecificationsPE BundlesTopic BundlesSampling PlanTiered Assessment System—Summative Single form 5, 8,Bio4 forms; matrix by studentReporting CategoriesSingle scoreLife, Physical, EarthDoes thestudents’responseprovideevidence?Is the evidence sufficientto measure what wasintended?
General Alignment Questions4. What are the questions that we areasking to determine alignment?What do we wantto measure?5. What is the process of making analignment judgment?22Does thestudents’responseprovideevidence?Is the evidencesufficient tomeasure what wasintended?
What do you think?Intended AlignmentPS1.A – Each pure substance has characteristic physical and chemical properties that can beused to identify it.PS1.B – Substances react chemically in characteristic ways. In a chemical process, the atomthat make up the original substances are regrouped into different molecules, and these newsubstances have different properties from those of the reactants.SEP 4 – Analyzing and Interpreting DataCCC 1 – PatternsItem Level Claim: The student uses the data table to determine if a chemical reaction occurredand provides the evidence to support their determination.23
24
25
26
What do you think?AlignedNot AlignedHow do you know?27
Perspectives to ConsiderSEP Guidance and QuestionsCCC Guidance and QuestionsGeneral Expectations28
SEP Questions1. Which practice is most specifically measured by the item inquestion?2. Is the targeted practice aligned with a grade levelappropriate progression(s)? Appendix F: “Practices Matrix” for SEPs lists all possible practice progressions by gradeband.3. At what level is the practice being measured by thisspecific item?29
CCC Questions1. Are/is a CCC(s) mentioned in the wording of this item?2. Are/is a CCC(s) alluded to within the item withoutrequirement of CCC evidence collection for a correctresponse?3. Are/is a CCC(s) directly measured through this item as aknowledge or framing construct developed by students?Do you feel this item is “ALIGNED” with a CCC?30
General Expectations1. Sense making1. to what degree is the item cluster/item allowing students to sensemake?2. Phenomena1. to what degree is the phenomena appropriate, and meeting theexpectations of a phenomena?31
What do you think?Intended AlignmentPE 05-PS3-1: Use models to describe that energy in animals’ food (usedfor body repair, growth, motion, and to maintain body warmth) was onceenergy from the sun.SEP – Analyzing and Interpreting DataCCC – Energy and matterItem Level Claim: The student is asked to analyze data (SEP) from afigure and select a statement to construct an explanation (SEP) using thecrosscutting concepts of energy and matter and applying the core idea ofenergy in chemical processes (DCI).32
33
What do you think?AlignedNot AlignedHow do you know?34
Final thoughts on NGSS alignment challenges35
We are five years in,what is the next frontier for NGSS assessments?Making good ones 36
April McCrae –Delaware Department of Education . Concepts (CCCs), beyond those specified in the PEs in the PE bundle, may be included in the alignment of items in an item cluster. Item and
