WIXLIB

Making it Easy for Faculty to Document EmbeddedAssessment Activities and Use the Results for ProgramImprovementRosalind Humerick, Vice President for Research and Institutional EffectivenessSt. Johns River Community College

Describe the steps taken to achieve widespread faculty participation in the embeddedassessment processGain a general understanding of the onlineassessment system developed by SJRCCRecall how the online system can captureboth qualitative and quantitative data fromindividual faculty members

Describe the process used by SJRCC toanalyze sample aggregate data and generatedepartmental discussionsEvaluate the strengths and weaknesses of thecurrent online systemHave the opportunity to use the onlinesystem at your home institution

SACS, SJRCC & InstitutionalEffectiveness

Core Requirement 2.5 The institution engages in ongoing, integrated, andinstitution-wide research-based planning andevaluation processes that (1) incorporate asystematic review of institutional mission, goals,and outcomes; (2) result in continuing improvementin institutional quality; and (3) demonstrate theinstitution is effectively accomplishing its mission.(Institutional Effectiveness)

Comprehensive Standard 3.3.1 The institution identifies expected outcomes, assessesthe extent to which it achieves these outcomes, andprovides evidence of improvement based on analysisof the results in each of the following areas:(Institutional Effectiveness) 3.3.1.1 educational programs, to include student learningoutcomes 3.3.1.2 administrative support services 3.3.1.3 educational support services 3.3.1.4 research within its educational mission, if appropriate 3.3.1.5 community/public service within its educationalmission, if appropriate

Comprehensive Standard 3.5.1 The institution identifies college-level generaleducation competencies and the extent to whichgraduates have attained them. (College-levelcompetencies)

What is a Student Learning Outcome (SLO)? It identifies the measurable knowledge, skills,behavior and/or attitudes the learner has as a resultof engaging in a learning activity or program (KSBA) What is a core competency? The intended result of student learning experiencesacross courses, programs and degrees.

Located in North East Florida Three campus locations Orange Park Campus Palatka Campus St. Augustine Campus Faculty (Fall 2009) 100 full-time 150 part-time

-092009-109199928287978850926798861010110938

Fall 2009Orange Park2114Palatka1334St. Augustine1925Distance Learning962Other Offsite956

Category of InstructionAdvanced & ProfessionalPost Secondary VocationalCollege PrepEducator Preparation InstitutePost Secondary Adult VocationalApprenticeshipContinuing Workforce EdAdult Basic EducationAdult Sec./GED 3.4388.835.086.64.54528.7Source: Florida College System Fact Book 2008, 2009, 2010Retrieved from: http://www.fldoehub.org*Note: A & P, PSV, College Prep, and EPI are credit, Credit FTE 30 credit hrs, Non-credit FTE 900 contact/clock hrs.

ka739.4760.3778.8St. Augustine990.51133.51182.5Distance 17.54090.04462.6Orange ParkSource: Internal SJRCC Database*Does not include Adult Education

Visiting team had the following concernsabout the College’s IE program: Was not broad-based and did not involveparticipation by vast number of faculty and staff Faculty in some departments were not aware ofdepartmental goals as had been set at the divisionlevel Some administrators were not aware of goals andevaluations as they had been set at departmentlevel

College initiated college-wide faculty retreatsin 2003 to address assessment of educationalprograms.College submitted a Follow-Up report inSeptember 2004 demonstrating that eachacademic unit had completed a cycle ofreviewing goals, evaluating the results ofassessments, and using the results.

Requested to submit Part IV: AdditionalReport showing continued compliance withCS 3.3.1 (Institutional Effectiveness) Had to discuss: Formulation and assessment of learning outcomes forall educational programs Expected outcomes for all administrative andeducational support services and describe extent towhich those outcomes are achieved Report submitted September 2009

Strategic Planning Three year Cycle Vision, Mission, and College-Wide Goals for 20092012 Strategic Plan approved by District Board ofTrustees in July 2009 Operational Planning One year Cycle Use Strategic Plan to develop annual unit plans All units assess 3-5 goals/program outcomes per year Assessment of program outcomes must include themeasurement of student learning outcomes.

Institutional Effectiveness and Strategic Planning Framework (Simplified)Florida State Board of Education Goals&K-20 Strategic Areas of Focus (2008)Florida College System Mission/Goals&Florida Community College System Priority Goals (2005)St. Johns River Community College Mission StatementSt. Johns River Community College Priority Goals(2009-2012)Instructional Units:Mission StatementKey Functions:oUnit goalsProgramsoProgram OutcomesInstructional units will assess 3-5 goals/program outcomesper year. Assessment of program outcomes must includethe measurement of student learning outcomes. Theanalysis of assessment results and evidence of the use ofthese results to improve programs and student learningmust occur on an annual basis.Non-Instructional Units:Mission StatementKey Functions:oUnit goalsNon-instructional units will assess 3-5 unit goals per year.The analysis of assessment results and evidence of the useof these results to improve administrative and supportservices must occur on an annual basis.

2003 - Present

2003-2006 For reporting used a five column form based on theNichols model and related “goals” to collegemission/objectives 2006-2007 Academic year Focused on strengthening the framework for theassessment of student learning outcomes

2007-2008 Academic Year Piloted the assessment of course level student learningoutcomes developed in 2006-2007 2008-2009 Academic Year Expanded assessment activities to more courses anddisciplines Developed an online planning and assessment system 2009-2010 Academic Year Online planning system used by more departments andfaculty including adjunct and dual enrollment faculty

Quick Facts about Usage Fall 2008 16 courses 52 faculty 225 separate entries Fall 2009 33 courses 95 faculty 389 separate entries

Terminology: What is a student learning outcome? What is the difference between a learning outcome and alearning output? Why can’t we use course grades? What is embedded assessment? Bloom’s taxonomy Rubrics Methods for assessment of SLO’s: Quizzes, tests Pre/post testing Portfolios

Assessment is at the program or course level,it is not at the individual faculty or studentlevelData collected for institutional effectivenesswill not be used as part of an individualfaculty member’s evaluation. Data is onlydistributed in aggregate.

At the program level General Education Program Specific e.g. nursing At the course levelCourse outlines to list the program outcomesthe course contributes to, as well as theindividual course level SLOs

At SJRCC we started with Arts and Sciences Each department chose one course to pilot Most started with the assessment of one studentlearning outcome from this course Decided how to measure the chosen SLO Encouraged to consider artifacts they were alreadycollecting for student grades Most used multiple choice embedded questions Some used a rubric (essays and math/science)

Decided on the level of achievement for SLOEach Arts and Sciences full-time facultymember participated in the pilot ( 50)Faculty submitted: Pre-semester Learning Outcomes Report Form Post-semester Learning Outcomes Report Form Student Learning Outcomes Reporting Form forquantitative data

CourseSemesterStudentLearningOutcomeCheck One: On campus OnlineBrief description of howlearning outcome isaddressed in the coursematerials and informationprovided to students, e.g.readings, lecture,multimedia, labs) Off site dual enrollment Off site otherBrief description ofwork performed bystudents that willdemonstrate orassess the learningoutcome (e.g.,exams, home-work,projects, labs)List of materialsthat will becollected toevidence thelearningoutcomeFrom Barbara E. Walvoord : Assessment Clear and Simple: A Practical Guide for Institutions, Departments, and General Education

CourseSemesterStudentLearningOutcomeCheck One:Identify strengthsand weaknessesin your teachingmethods forcourse materialsrelated to thisgoal On campus OnlineIndicatestrengths andweaknesses instudentachievementof the learningoutcome Off site dual enrollment Off site otherWhat changes, ifany, do you planfor the next timeyou teach thiscourse toimprove studentlearningWhat actionscan thedepartmenttake toimprovestudentweaknessesyouidentifiedFrom Barbara E. Walvoord : Assessment Clear and Simple: A Practical Guide for Institutions, Departments, and General Education

CourseSemesterQuestion NumberCheck One: On campus OnlineBrief Description Off site dual enrollment Off site otherTotal Correct/ TotalResponses

CourseSemesterQuestionNumberBriefDescriptionCheck One: On campus Online1 Beginning 2 Developing(Report #)(Report #) Off site dual enrollment Off site other3 Competent(Report #)4 Accomplished(Report #)

All departments decided to use an embeddedassessment methodDiscussion about when to assess (during thesemester versus at the end of the course)Faculty wanted to isolate students who hadpassed the course from those who haddropped or withdrawn.

Best to have two meeting per year: One in August to review results of previous year’sassessment results and confirm the assessmentactivities for the current academic year One in April to look at the preliminary results of thecurrent year’s activities and to tentatively plan forthe following academic year. Some kind of summary sheet for departmentmeetings was needed to accurately reflectfaculty discussions concerning assessmentresults and planning activities.

Collating the assessment results was timeconsuming Faculty put their names on submissions! The amount of paperwork was overwhelming!

Development of an onlinesystem to easily documentembedded assessmentactivities by faculty

Online system accessed through the MySJRCCportal and available 24/7For each SLO assessed, faculty membersenter: Qualitative information (pre-semester and post-semester reports) Quantitative information from embeddedassessment Either graded as a rubric or as correct/incorrect

My office collates quantitative and qualitativedata and supplies these to the departments Departments discuss the results and use theresults for improvement Administrator or department head completes an“annual outcome summary” documenting the useof resultsDepartments plan their next year assessmentactivities

Planning and Assessment System

Enter qualitative information by clicking onthe “Edit” icon

Enter quantitative data by clicking on “EditResults”

My office collates quantitative and qualitativedata and supplies these to the departmentsDepartments discuss the results and use theresults for improvement

Administrator or department head completesan “annual outcome summary” documentingthe use of resultsDepartments plan their next year assessmentactivities

Simulated Experience

Choose a student learning outcome from thecollege algebra outline in your packetFill out the pre-semester learning outcomesreport for this SLONow look at the assessment results andcomplete the post-semester learningoutcomes report for this SLO

Look at the data you have been provided forFall 2008 and Spring 2009 for MAC 1105Discuss the results and: Complete one “Summary Sheet for Discussion ofAssessment Results” per groupand Complete one “Summary Sheet for Planning NextYear” per group

What my office does

Courses are added by the developer in the ITdepartmentMy office adds student learning outcomes forthe courses from course outlines that aresubmitted to us