Annual Academic Assessment Report Cover Sheet

Transcription

Annual Academic Assessment Report Cover SheetAssessment reports are due the 1st Wednesday after the Fall TermEmail to: assessment@unlv.eduProgram Information:Program AssessedDepartmentCollegeDepartment ChairMSW – graduate programSchool of Social WorkGreenspun College of Urban AffairsCarlton CraigAssessment CoordinatorMatt GygerDate Submitted12/20/2018Contact Person for This ReportNameMary-Ann unlv.eduPlease attach a narrative (not to exceed 4 pages, excluding appendices) addressing thefollowing: What are the student learning outcomes? Please provide a numbered list.Which learning outcomes were assessed?How were they assessed? (Programs must use at least one direct assessment of studentlearning.)Undergraduate programs should assess at least one University Undergraduate LearningOutcome (UULO) each year, which may or may not overlap with a program learningoutcome.Graduate programs should assess at least one outcome related to one of the followinggraduate level requirements each year:o student engagement in research, scholarship, creative expression and/or appropriatehigh-level professional practice.o activities requiring originality, critical analysis and expertise.o the development of extensive knowledge in the field under study.What was learned from the assessment results?How did the program respond to what was learned?Please limit the narrative portion of your report to no more than four pages. You may attachappendices with data, tables, charts, or other materials as needed. Please explain the relevantconclusions from any appendices in your narrative. . Please contact the Office of AcademicAssessment if you have questions or need assistance.10/20141

A. Masters of Social Work (MSW) learning e Ethical and Professional BehaviorEngage Diversity and Difference in PracticeAdvance Human Rights and Social, Economic, and Environmental JusticeEngage In Practice-informed Research and Research-informed PracticeEngage in Policy PracticeEngage with Individuals, Families, Groups, Organizations, and CommunitiesAssess Individuals, Families, Groups, Organizations, and CommunitiesIntervene with Individuals, Families, Groups, Organizations, and CommunitiesEvaluate Practice with Individuals, Families, Groups, Organizations, and CommunitiesB. MSW learning outcomes/competencies and graduate level requirements assessed forthis reporting period (fall 2017 – spring 2018):Learning outcomes:1. Demonstrate Ethical and Professional Behavior4. Engage in Practice-informed Research and Research-informed Practice9. Evaluate Practice with Individuals, Families, Groups, Organizations, and CommunitiesGraduate Level Requirements1. Student engagement in research, scholarship, creative expression and/or appropriatehigh level professional practice.C. MSW assessment measures: Both direct and indirect assessment measures are used fordata collection and are listed below.Direct ed1-9MSW Foundation Survey: Asks MSW students’ theirperceptions regarding how well they are prepared toaccomplish 31 practice behaviors established by the Councilof Social Work Education (CSWE) after their foundational yearin the MSW program. Students are also asked theirperceptions of MSW program components, including bothquantitative ratings and qualitative comments andsuggestions.Indirect1-9MSW Exit Survey: Asks MSW students’ their perceptionsregarding how well they are prepared to accomplish 41 CSWEpractice behaviors upon graduation. Additionally, students’satisfaction is captured regarding practice behaviors specificto either the direct practice or management and communitypractice concentrations.Indirect1-9MSW Alumni Survey: Asks MSW alumni their perceptionsregarding how well they were prepared to accomplish thepractice behaviors a year after they have graduated.Indirect1-9Assessment Instrument (e.g., survey, exit exam)Licensing Exam Pass Rate: This information is collectedfrom the Nevada State Board of Examiners for Social Workersfor the previous year10/2014Expected Measures(results that wouldindicate success)Pass rate isequivalent to orabove the nationalpass rateFor each question,we are using 7 points[out of a 10-pointscale] as ourbenchmark, treatingeach of the practicebehaviors within alearning outcomewith an equal weight(see Appendix A),and adopting 60% asthe target percentageof students achievingthe learning outcome.2

Assessment Instrument (e.g., survey, exit exam)Implicit Curriculum Survey (Indirect): Thirty-item survey tomeasure five areas of implicit curriculum as defined by theCouncil for Social Work Education. These five categories are:Faculty Availability and Advising; Resources and Equipment;Cultural Diversity; Student Development and AdministrativeGovernance. Six items are purported to measure examples ofeach category. The survey also allows respondents to providecomments regarding the five areas.Direct ssed1-9Expected Measures(results that wouldindicate success)We are using 3 pointswhich denotes“adequate” (out of a 1to 5 scale) as thebenchmark.D. MSW Assessment ResultsLicensing Exam Pass RateA direct measure of the School’s MSW program is its graduates’ national licensing exam passrates. The following is a summary of the data from 2014 to 2016. For 2016, masters licensingexam pass rates were equal (71%) compared to national rates (70%), and the clinical exam passrates for our students (78%) was higher compared to national pass rates (67%).UNLVSSW Graduates’Pass RatesMasters201576% (N 80)201671% (N 76)201781% (N 85)NationalPass Rates69% (N 19,718)70% (N 20,043)70% (N NA)UNLVNationalSSW Graduates’ Pass RatesPass RatesClinical20152016201769% (N 48)78% (N 50)64% (N 18,265)67% (N 18,868)84% (N 43)68% (N NA)MSW Foundational Survey, MSW Exit Survey & MSW Alumni SurveyMSW Foundation Survey: For this year’s analysis, we are continuing to use the median scores ofthe practice behaviors because we believe that presents a better measure of central tendency byremoving outlier responses. The benchmarks used for all the MSW surveys that measure thelearning outcomes/competencies is a score of “7” or higher, and 60% or more of the students’noted a mean score of “7” or higher for each competency; see Appendix A. For these 54foundation year students, all expected measures were met.An independent-samples t-test was conducted to compare the 31 mean scores for the practicebehaviors between the 2018 and 2017 MSW Foundation survey cohorts. For eight of the practicebehaviors, means for this year were significantly higher than last year; see Appendix B.When asked about their experiences in the program, 22 students provided feedback. Somereported positive experiences with the program (N 7) and faculty (N 4). Please note, in thisanalysis themes reported have two or more responses. When it came to challengingexperiences, the only theme that had more than one response was the perception the programshould be more challenging (N 3). When it came to suggestions for improving the program, 30students provided recommendations. Some students noted improved class availability andscheduling (N 7). A few noted difficulty transitioning from the research courses, 716 and 726(N 3), while others noted improved communication in the program (N 3). A couple of studentsnoted standardized teaching and expectations from faculty (N 2), more focus on macro practice(N 2), and faculty being more supportive of students (N 2). A quote is provided below for eachone these themes. A recommendation specific to field was the revision of the weekly journalingquestions (N 4).10/20143

MSW Exit Survey: Seventy-three (69 DP & 4 MCP) participants completed the MSW Exit Surveyof which all of the expected measures/benchmarks were met (See Table 1). That is, all ninecompetencies had a mean score of “7” or higher, and 60% or more of the students’ noted a meanscore of “7” or higher for each competency.An independent-samples t-test was conducted to compare the 31 mean scores for the practicebehaviors between the 2018 and 2017 MSW Exit survey cohorts. A significant difference wasfound for one that is part of the competency specific to engagement; see Appendix C. The meanfor this practice behavior was significantly lower than the previous year.When asked about their experiences in the program, 34 provided feedback. Similar to thefoundation students, many noted positive experiences with the program (N 14) and faculty (N 4).A perspective the MSW Exit Survey respondents added was the positive experience of peerconnections (N 4). When it came to challenges, feedback was given similar to the FoundationSurvey respondents in that the program should be more challenging (N 4). Feedback given bygraduates whose experiences were different from the first year students was that some notedissues around administrative structures that were perceived as disorganized (N 8), difficulties inthe scheduling and access to courses (N 5), and faculty/staff not offering adequate support (N 3)Fifty-six (56) of these graduates offered suggestions. Some were similar to the FoundationSurvey respondents, such as improved class availability and scheduling (N 14); improved“communication” in the program (N 5); standardized teaching and expectations (N 3); and moresupports for students (N 4). These respondents also provided other suggestions beyond whatthe Foundation students offered. A majority of the suggestions centered on curriculum andcontent, specifically around adding clinical and intervention courses specific to direct practice(N 14). A couple noted offerings in trauma focused care (N 2) along with a mezzo focus (N 2).Some students noted there should be more administrative staff (N 8), improving access toadvisors (N 4), offering a course on licensing preparation (N 3) and coordinating more studentevents (N 3). Some offered suggestions that were specific to field (N 12), with a few notingmore support in the foundation year with the practicum process (N 3), and reducing the amountof hours required for practicum (N 3).MSW Alumni Survey: Of the 59 emails sent inviting alumni to complete this survey, 29%responded (N 17). Five (5) of the nine competencies had a score above 7, and only onecompetency had 60% or more of the students with a mean higher than 7 (see Appendix A).However, caution should be used in interpreting this finding with the low response rate.When asked about their experience in the program, six (6) provided feedback with four (4) notingthat it was positive. When it came to suggestions for improving the program, seven (7) providedfeedback. However, no common themes were identified.Student engagement in research, scholarship, creative expression and/or appropriate high levelprofessional practice. All students are required to undertake a critical analysis of researchregarding the effectiveness of a social work intervention in SW 795, as the Capstone Experiencerequired by the Graduate College; students then may be assisted in preparing the critical analysisfor publication as appropriate. In addition, the program supports a Thesis Option, which onestudent completed and defended within this academic year. Students are encouraged and haveparticipated in the Greenspun College of Urban Affairs Research Symposium poster session inspring semester each year. Also, as a part of the required social work curriculum, field practicumsin professional practice occur each semester of the full-time MSW program, with the foundationyear at 15 hours per week or 450 hours, and the concentration year in advanced professionalpractice for 20 hours per week, or 600 hours, totaling 1050 hours of supervised professionalpractice in field agencies while in the MSW program.10/20144

Implicit Curriculum SurveyNew this year is the addition of an implicit curriculum survey as part of both the MSW Foundationand Exit surveys. The implicit curriculum measure is a 30-item measure with a five-point Likerttype scale ranging from poor to excellent developed by the Director of the UNLV School of Socialwork in order to measure five areas of implicit curriculum as defined by the Council for SocialWork Education. These five categories are: Faculty Availability and Advising; Resources andEquipment; Cultural Diversity; Student Development and Administrative Governance. Six itemsare purported to measure examples of each category. The highest mean was for CulturalDiversity for both the Foundation (M 3.74) and Exit surveys (M 3.69). Similarly, the lowest meanwas for Administrative Governance; Foundation (M 3.34) and Exit (M 3.01). See Appendices D& E.5. How did the program respond to what was learned?Issues related to administrative governance seem to have decreased, and the continued use ofthe implicit curriculum measures will be helpful in measuring any such changes. The mainadministrative office has one administrative assistant, a half-time administrative assistant, andthree part-time student workers, and the response time is quicker and more helpful to students,applicants, and visitors alike.In terms of the admissions process, it has been very helpful to return to a model of a separatefaculty member serving as the MSW Admissions Coordinator/Chair of the Admissions Committee;with that as well as quicker response times, continued monthly applicant information sessions,earlier deadline dates, and a streamlined process, we expect to have higher applicantsatisfaction. We also addressed the ‘early bird’ pre-admission anxiety of students—as well astheir questions about the program and the ‘next steps’ to take—by having a series of preorientation meetings in the Spring semester (prior to the MSW Orientations in May/June for theAdvanced Standing MSW students and in August for the regular MSW Program).The Martini Lunches with the students continue each semester, planned for the time betweenfield seminars on Fridays to maximize its availability to students, two for the foundation-yearstudents in SW 719 and SW 729, and two for the concentration-year students in SW 739, SW759, SW 749, and SW 769. In contrast to former years, there were no complaints and fewquestions raised, mainly appreciation for the direct access to information on class schedules andgraduation and licensing processes. The MSW Program has added sections of courses onFridays and Saturdays in the past few years, which are well accepted and filled. While noting thecouple of comments in written surveys regarding the lack of available sections of courses or theoverlapping of courses, these comments reflect the lack of available sections of courses fittinginto the individual student’s schedules rather than a lack of sections for students, so no changeshave been implemented with the exception of talking more explicitly about the scheduling processduring the Martini Lunches.As requested by students, more clinically-oriented elective classes have been added to theschedules, the most recent being one on trauma-informed practice and one on alternativepractice therapies in the Summer Term. With a larger number of electives and clinically-orientedelectives offered in both semesters of the regular academic year, issues related to the availabilityof elective courses have been resolved. Additionally, the development of a Trauma-InformedPractice and Policy concentration has been supported through curricular and faculty processes inthe School, and is being readied for review now through the University processes, with traumainformed concentration classes planned to begin in Fall 2020; this change in our MSW Programhas been met with a great deal of excitement in our student body and applicants as well.10/20145

Appendix A ‐ Learning Outcome/Competency MeansCompetency123456789Demonstrate Ethical and Professional BehaviorEngage Diversity and Difference in PracticeAdvance Human Rights and Social, Economic, andEnvironmental JusticeEngage In Practice-informed Research and Researchinformed PracticeEngage in Policy PracticeEngage with Individuals, Families, Groups, Organizations, andCommunitiesAssess Individuals, Families, Groups, Organizations, andCommunitiesIntervene with Individuals, Families, Groups, Organizations,and CommunitiesEvaluate Practice with Individuals, Families, Groups,Organizations, and CommunitiesMSW FoundationalSurvey (N 48)% of studentsTotalwith "7" orhigher8.6092%9.1792%MSW Exit Survey (N 73)8.409.00% of studentswith "7" orhigher84%86%TotalMSW Alumni Survey(N 17)% of studentsTotalwith "7" 8871%7.2568%6.7547%Note: for the range, 1 "very poor preparation" and 10 " very good preparation

Appendix A - Learning Outcome/Competency Means DetailCompetency 1 : Demonstrate Ethicaland Professional BehaviorMSW Foundation Survey Spring 2018MSW Exit Survey 2018MSW Alumni Survey 2018MedianRange1-10St. Dev.nMeanMedianRange1-10St. Dev.nMeanMedianRange1-101Make ethical decisions by applyingthe standards of the NASW Codeof Ethics, relevant laws andregulations, models for ethicaldecision-making, ethical conductof research, and additional codesof ethics as appropriate to 477.003-102.092Use reflection and self-regulationto manage personal values andmaintain professionalism inpractice Average8.40Average7.40% of students with"7" or higher92%% of students with"7" or higher84%% of students with"7" or higher53%345Demonstrate professionaldemeanor in behavior;appearance; and oral, written, andelectronic communicationUse technology ethically andappropriately to facilitate practiceoutcomesUse supervision and consultationto guide professional judgment andbehaviorBenchmark Mean 7Benchmark for students having amean of "7" or higher for thecompetency 60%St. Dev.nMeanNote: For the range, 1 represents "very poor preparation" and 10 represents "very good preparation."1

Appendix B - MSW Foundation Survey 2017 versus 2018 – SignificantDifferencesMSW FoundationSpring 2017nMeanSt. Dev.MSW FoundationSpring 2018nMeanIndependent Samples TTestSt. Dev.t df p 004Competency 1 : Demonstrate Ethical and Professional Behavior5. Use supervision andconsultation to guide537.852.4648professional judgment andbehaviorCompetency 2 : Engage Diversity and Difference in Practice6. Apply and communicateunderstanding of the importanceof diversity and difference inshaping life experiences inpractice at the micro, mezzo,and macro levels548.191.894487. Present yourself as learnersand engage clients and547.651.824488.61.412.934constituencies as experts of theirown experiencesCompetency 4: Engage In Practice-informed Research and Research-informed Practice12. Apply critical thinking toengage in analysis ofquantitative and qualitativeresearch methods and 4513. Use and translate researchevidence to inform and improvepractice, policy, and 01.018Competency 7: Assess Individuals, Families, Groups, Organizations, and Communities19. Collect and organize data,and apply critical thinking tointerpret information from clientsand y 8 : Intervene with Individuals, Families, Groups, Organizations, and Communities25. Use inter-professionalcollaboration as appropriate toachieve beneficial 827Facilitate effective transitionsand endings that advancemutually agreed-on goals557.132.373488.061.7192.3197.816.023 page

Email to: assessment@unlv.edu Program Information: Program Assessed MSW – graduate program Department School of Social Work College Greenspun College of Urban Affairs Department Chair Carlton Craig Assessment Coordinator Matt Gyger Date Submitted 12/20/2018 Contact Person for This