Transcription
Streamlining InstitutionalAssessment for Students and FacultyUsing Blackboard’s AssessmentSolution
Your PresentersMarlonMote,DirectorTina Babb,DirectorDirector of InstitutionalEffectivenessSACS Accreditation Liaison Institutional Effectiveness,Accreditation and AssessmentImplementation of institutionwide assessment and guidanceon accreditation standardsJoined AC in 2010 and promotedto current role in January 2017Tmbabb@actx.eduI.E., Accreditation, and Planningwhich includes ISLOs and PLOsAssessment Joined TCC in 2015experience in I.R., I.E.,Assessment, andAccreditationProvides leadership fordistrictwide assessment at 6campuses – 51K CreditStudentsRuthNewberry,PhDPrincipal Education Consultant forAssessment and Competency-baseEducation Joined Blackboard in 2014 withmore than 25 years Higher EdExperience Has assisted more than 40 clientsinternationally and in the US withassessment Ruth.Newberry@blackboard.comMarlon.Mote@tccd.edu2
Audience Poll What is your role at your campus?– Faculty– Assessment Professional– Responsible for General Education Assessment– Technical Support Staff How many of you use Blackboard Learn as your LMS? How many of you use Blackboard Outcomes to support yourassessment data collection and review processes? How many of you are using another software to support yourassessment data collection and review processes?3
Please Hold Questions to the End4
Welcome and IntroductionsTopicsJuried General Education AssessmentMeeting Challenges with Blackboard’sJuried SoftwareAmarillo College –Simplifying the ProcessTarrant County Community College –Building Buy-in Across MultipleCampusesClosing the Loop and ContinualImprovement5
Assessment ofStudent LearningIs CriticalStudent Success Where are students struggling? Where are students doing well?General Education ProgramExcellence What do we need to change toimprove student learning?Data Driven Decision Making forGeneral EducationImprovements How do we prioritize changes?6
Key ChallengesFacingInstitutionsFaculty Focused on Course-LevelGeneral Education Effectiveness What do we need to change toimprove student learning?Relevance & Improvements How does the data help me improvemy course?7
How Does Blackboard’s Assessment SolutionHelp?Adds Efficiencies to the Process Leverages what instructors and students already doin Blackboard Course Sites. Streamlines data collection for AssessmentProfessionals. Scales across the programs and campus.8
Leverage what Instructors and Students Already Do Contains Yours Students and InstructorsAssociates Students and Instructors to Courses andOrganizations Courses Contain Learning Activities and Student Work(Direct Evidence of Learning) Add General Education, Program, and DisciplineAccreditor Learning Outcomes into Goals AreaAssociate Activities to THECB SLOs 9
Select the Blackboard Software toFacilitate Your Assessment ProcessPrimary AssessmentJuried Assessment Outside the Course Review/ Committee Review Student work collected fromAssignments Work sampled and randomlydeployed to reviewers Committee reviews workusing rubric specific to SLOAND/ OR Inside the Course Review /Instructor Review Collects instructor scoredcourse rubrics (grading orsecondary) with criterionaligned to SLOs Collects exams withquestions aligned to SLOs10
Blackboard Facilitates Juried Process:Alignments and CollectionCompositionBiologySociologyAnalysis EssayAssignmentAligned to -Bio DiscoveryRpt AssignmentAligned to -CommunityAnalysisAssignmentAligned to -Written ComCritical ThinkingInfo LiteracyWritten ComCritical ThinkingInfo LiteracyWritten ComCritical ThinkingInfo Literacy11
Evidence Repository Through Evaluation andReporting for Juried AssessmentEXAMPLE: Fall 2018 Collections40 Composition 102 sections w/ Analysis Essay 840 artifacts23 Biology 211 sections w/ Disc Report 483 artifacts18 Sociology 235 sections w/ Community Paper 378 artifacts1701 Artifacts Randomly SampledArtifacts Randomly Deployed to Teams of 5 w/2 reviews for each artifactWrittenCommunicationRubricCritical ThinkingRubricInformation LiteracyRubric12
Tina BabbDirector of Institutional Effectiveness andSACSCOC Accreditation Liaison for AmarilloCollege in Amarillo, TX
Learning Outcomes for this SessionLeverage thistechnology to buildpractices around theassessment ofstudent artifacts atboth institutionaland program levelsExamine acontinuousimprovementstrategy for bothgeneral educationand programassessment thataligns withBlackboardOutcomesAssessmentExamine thereporting outputsthat triggerdiscussions aboutimprovements thatcan be made14
Outcomes Assessment Pilot 2015-17CHALLENGES– Manual assessment of hard-copystudent artifacts– Lacked a central repository forhousing historical assessment data– Development & maintenance ofrubrics– Data collection and analysiscompleted separately– Lacked structure for goal setting(Didn’t realize we needed them!)As a result, introduced CurriculumMapping in 2015 for Program levelassessment15
Amarillo College Assessment ComponentsAlignment with Blackboard Outcomes Assessment16
Building Practices with Outcomes AssessmentGeneral Education Competencies – AC’s Core Curriculum17
Building Practices ContinuedContinuous Improvement Strategy and Closing the Loop18
Continuous Improvement – “Closing the Loop” Assessment Cycle and Schedule: An assessment cycle is completed when alearning outcome (either institutional student learning outcome or programstudent learning outcome) has been re-assessed. Re-assessment allows for theCollege and the program to determine if improvements that are made haveimpacted student learning and in what manner.19
Development of an Assessment Planning TemplateWe needed to develop an instrument to align with the behind-the-scenes workneeded to create evaluation sessions within Blackboard Outcomes Assessment 20
Stats on Artifact Collection and AssessmentFall 2017 CollectionsSpring 2018 CollectionsFall 2018 Collections Personal and SocialResponsibilityCompetencies Personal and SocialResponsibility andTeamwork Competencies Critical Thinking andCommunicationCompetencies 1774 randomlysampled artifactsacross 94 coursesections 249 randomly sampledartifacts across 11course sections 2503 randomlysampled artifactsacross 187 coursesections 3-6 faculty perevaluation session 2-4 faculty perevaluation session 3-6 faculty perevaluation session21
Ok, So How Do We Perform the Actual Assessment? Department Chairs or Coordinators identify faculty members involved in theevaluation sessions via our planning template At close of semester, IE creates an evaluation session through BlackboardOutcomes:– Artifact collection across identified course sections– IE inputs formula for sampling– Align the rubric, assign the faculty and generate link to faculty email Faculty have the entire semester to assess Video: You Are Now A Faculty Evaluator (Do we want to show this?)22
Data Analysis and Reporting2 reports are generated at the close of each evidence collection/evaluation: Raw data .xls file Data visualization graphs in .pdf23
Data and Analysis Reporting-Continued24
Marlon MoteDistrict Director of Institutional Effectiveness,Accreditation and Planning and SACSCOC LiaisonFort Worth
Learning Outcomes for this SessionOverall Assessmentof ISLOs at TCC 2015to 2019 and theimpact to our workFocus onCommunicationand EmbeddedAssignments andwhy it is differentBlackboardConsultation/Deployment of Outcomesas a districtwideeffortLessons Learnedand ContinuousProcessImprovementFaculty ProfessionalDevelopment andwhy it matters togive clearinstructionsSignatureassignments andmore intentionalcollection ofartifacts26
Overall Assessment of ISLOs at TCC 2015 to 2019 State Core Objectives – TCC ISLOs Adoption of AAC&U Leap Value Rubrics Paper-based Assessment to Digital using Outcomes 2nd Cycle of Assessment for 6 ISLOs Communication – Written and Oral – Spring 2016 andSpring 201927
Blackboard Consultation/Deployment of Outcomes Consulted with Bb in Fall 2015 with Dr. Newberry Faculty Professional Development to DeployCommunication Assessment in Spring 2016 Twenty-five faculty engaged in discussion andassessment activity Assessment Team members (includes faculty) evaluatedwritten communication assignments from across all Coredisciplines28
Bb Outcomes SP 2016 - Written Communication29
Bb Frequency Distribution – Written Communication30
Assessment Action Items – Closing the LoopMonthActivityResultAugust 2016Assessment Table at Chancellor’sBreakfast. Assessment Progressdiscussed at South Campus 1st WeekBack.Launch Assessment LeadershipInstitute (ALI) at NTCCC Outcomesand Assessment Conference.Shared updates with faculty and staff about assessment and engaged faculty toparticipate in District Assessment Team. Speech faculty providerecommendations for improvement.Math ACT review of math samplesfor communication and adjustmentto rubric.Foreign Language ACT review ofcommunication samples andadjustment to rubric.Assessment review with Biology ACTto create Teamwork assessment.Table at Academics Day forassessment.Discussion with Assessment SubTeam for Computer Literacy.Created new assignment design to implement in Spring 2017 for writtencommunication. Commitment to pilot Empirical and Quantitative ReasoningAssessment.Created new assignment design to implement in Spring 2019 for er2016January 2017February2017Nineteen TCC Faculty start first session of seven for ALI. Participation fromSign Language, History, Government, Biology, Geology, Kinesiology, Art, andAcademic Foundations faculty.Recommendation made by Biology ACT for courses to assess and plans todevelop embedded assessment with lab work.Review opportunity for C&I proposal to include COSC 1305 and BCIS 1301 backinto the TCC Core as an optional component area. Eighteen Texas institutionscurrently have one or the other or both. Reinforces the importance ofcomputer literacy as a foundational skill. It is also an institutional learningoutcome for TCC.31
Faculty Professional Development Systematic approach to deployment with Bb Faculty engaged in multiple discussions of the value ofassessment How authentic assessment is more valuable and whystudents perform better when there are high stakes Dispelling the myth of judgement of individual facultyperformance with assessment32
Focus on Communication and Embedded Assignments Authenticity Relevance Ease of Deployment Ownership of Content Ownership of Improvement33
Lessons Learned and Continuous Process ImprovementSlow start to identify faculty participantsThe clear instructions matter to faculty,students, and reflects the quality ofassessment outcomesAssessment is a marathon and not a sprintAssessment is not a one and doneMaking connections with faculty and staff Integrative Learning – Active LearningStrategies Learning Environments – Space andTechnology Pedagogy/Andragogy Signature Assignments with multiple ISLOsIdentified departments that still cling to“committee style” assessment and hard-copyartifactsTraining was and is imperative for increasingfaculty participationIdentified communication gap between Chairs& Coordinators with their facultyAlignment of assignments with rubrics –importance of faculty knowing they areselected for assessment prior to semesterstart!Assessment is ongoing – We demonstratecompliance to SACS by showing what we havedone, not by focusing assessment aroundaccreditation cycles. “Culture of assessment”34
Some Challenges Going Forward Faculty Professional Developmentfor Signature Assignments Broader inclusion of onlinesections and dual enrollmentassessment Launch of the Computer LiteracyAssessment Explore opportunities ofassessment with E-Portfolios Keeping a place at the table andcombating initiative fatigue Additional training on how toleverage this application for moredepartments Need to increase facultyparticipation and include parttime faculty Teamwork competency is a toughone to assess especially withonline courses Challenge: Visual communicationartifacts35
QuestionsThank You
Joined Blackboard in 2014 with more than 25 years Higher Ed Experience Has assisted more than 40 clients internationally and in the US with assessment Ruth.Newberry@blackboard.com Ruth Newberry, PhD. Principal Education Consultant for Assessment and Competency-base Education Joined TCC in 2015 experience in I.R., I.E .
