Transcription
Cyber-teaching, Cyber-learning, and nowCyber-evaluation!How to Evaluate Clinical Competence using Distance TechnologyMargaret Clark Graham, Rita Kaspar, Carol Nikolai, Alice M Teall, Joni TornwallPresentersMargaret GrahamRita KasparCarol NikolaiAlice TeallJoni TornwallOSU Online NP Programs Received HRSA Funding to start onlineprogram Core courses were offered asynchronously Began offering assessment andpharmacology synchronously Developed clinical courses synchronouslyOSU CON Cyber‐Evaluation NONPF1
OVERVIEWMethods to Evaluate Clinical Competence from a DistanceWe’ve Come a Long Way, Baby!OSU CON Cyber‐Evaluation NONPF2
Evaluation Strategies Non-exam evaluation methods– Preceptor calls, blogs Exam evaluation methods– On-call phone triage exam– 2-step objective exam with remote online proctoring &conference call line– OSCEs across the curriculum– 2-step OSCE utilizing surveys and conference calling– Comprehensive final exam, individual onlyNON-EXAM EVALUATION METHODSOSU CON Cyber‐Evaluation NONPF3
Preceptor CallsBloggingSUMMER: use of social media in health careAUTUMN: wellness and health coachingSPRING: chronic illness and primary care practiceON-CALL EXAMSOSU CON Cyber‐Evaluation NONPF4
On-Call Exam Competencies Knowledge of contentCritical thinkingCommunication skillsAbility to do health coachingTime managementOn-Call ExamOn-Call Exam Specifics Schedule set upHire mock patientsDevelop clinical scenariosTwo calls in two hoursCall is recorded for gradingOSU CON Cyber‐Evaluation NONPF5
Scheduling PatientsScheduling StudentsOn-Call GradingSTUDENT NAME:10101010101010AvailabilityStudent either answers call or calls patient back within 3 minutesCommunicationStudent asks open‐ended questions and is professionalAssessmentStudent asks pertinent history questions related to the problemManagementStudent management plan is reasonable, patient‐centeredHealth coachingStudent provides follow up suggestions, clarifies questions, concernsDocumentationStudent provides accurate charting noteReflectionStudent is able to correctly note areas of learning needs, strengthsFaculty Comments:OSU CON Cyber‐Evaluation NONPF6
TEAM-BASED LEARNINGTwo-Step ExamsTwo-Step Exams Team-based learning Also known as:– Two-part testing, blended exam, cooperative exam One exam is given with two attempts– Individual exam (weighed 50% to 90%)– Group exam (weighed 50% down to 10%)Two-Step Exam: Part I Individual exam is proctored– Using proctoring centers and sending exams was notpractical solution– Trial of using online proctoring company did not meetour students’ needsOSU CON Cyber‐Evaluation NONPF7
Two-Step Exam: Part II Part II of the exam is the Group exam– On campus for large or small classesOnline Faculty Proctoring Individual exam is now proctored by faculty– Students log in to their web conferencing softwareTwo-Step Exam: Part II Part II for distance students can be given usingconference callingOSU CON Cyber‐Evaluation NONPF8
Instructional Technology Tips about using Adobe Connect for proctoringInstructional Technology Alternatives to Adobe ConnectObjective Structured Clinical ExamsImplementing OSCEs across the curriculumOSU CON Cyber‐Evaluation NONPF9
Standardized Patient Clinical Exams orObjective Structured Clinical Exams (OSCEs) Students came to campus– Twice a term– Used hired patients– Used moulage for “abnormals”OSCEs Using Digital Recording on Campusand in the CommunityFirst Attempts at Online OSCEs Achievements– Students could use webcams– Faculty could implement from adistance and could give immediatefeedback Difficulties– Teaching each patient the case– Faculty in front of the computer forextended periods of time– Recording the web-conference(Skype)OSU CON Cyber‐Evaluation NONPF10
Online OSCE using Adobe ConnectOSCE Preparation uses Course Management SystemOSCE Preparation uses Course Management System Post OSCE checklist for the semesterProvide details regarding volunteer neededPost schedule, instructions for the studentsRequire students to practiceRequire students and volunteer patients to signrelease forms Prepare faculty resources, hidden posts– Results of lab or imaging tests– Pictures or files for abnormal resultsOSU CON Cyber‐Evaluation NONPF11
OSCE Preparation: Faculty ScheduleOSCE Checklist Across CurriculumSee pdf OSCE Checklists Across Curriculum IntroductionHOPIROSHealth HistoryFamily, Social, Cultural AssessmentPhysical ExamDiagnosisManagement PlanPatient / Family EducationTherapeutic CommunicationOSU CON Cyber‐Evaluation NONPF12
Advanced Assessment StudentOnline OSCE: Novice NP StudentOnline OSCE: Advanced NP StudentOSU CON Cyber‐Evaluation NONPF13
Online OSCE: Patient-Focused Educationhttps://youtu.be/5ZNkkMhf9LAPRACTICE Using ChecklistOSCEUsing the survey tool to evaluate student performanceOSU CON Cyber‐Evaluation NONPF14
Self ReflectionIn a typical self critique “I thought that I did well”. “I thought that my verbal and nonverbal techniques weretherapeutic”. “I felt like I was able to establish rapport”. “I performed all necessary physical exams”. “I should have been more thorough on medication teaching”.The Joni Tornwall EnlightenmentOSU CON Cyber‐Evaluation NONPF15
Individual Response: PEIndividual Response: Dx testsOSU CON Cyber‐Evaluation NONPF16
Individual Response: Follow-upIndividual Response: Free Text “I forgot to ask for pulse ox. I guess I considered it.In the future I will be more proactive.” “I also forgot to percuss the lungs. I even told myselfto do it and still forgot.” “I failed to fully explain all the treatment, but that’sbecause I decided to send him to the ED.”Class Stats: PEOSU CON Cyber‐Evaluation NONPF17
Class Stats: DiagnosisClass Stats: Treatment PlanOSCETwo Step OSCEs – using a team approachOSU CON Cyber‐Evaluation NONPF18
Part II of the OSCE 20% of total OSCE gradeTalk on conference line, work on Google docFocus on diagnosis and managementComplete within a pre-set timeframeThe EvaluationCompleted Part II of the OSCEOSU CON Cyber‐Evaluation NONPF19
Completed Part II of the OSCEBenefits of Team-Based OSCEs Clarification of difficult case Reduced post-exam questions to faculty Exams become a learning tool throughgroup processLessons LearnedInnovateImplementRefineOSU CON Cyber‐Evaluation NONPF20
Concluding Thoughts:Two-Step ExamsPast:Test centers, tripsto campus, costsNow:Onlineproctoring, freeConcluding Thoughts:On-Call & OSCEsPast:On campus, costs of models,minimal collaboration, examsecurity issue, evaluationNow:Remote, no cost,collaboration in Part II,exam integrity, effectiveresults analysisJoni TornwallCarol NikolaiMargaret Clark GrahamAlice M TeallRita KasparOSU CON Cyber‐Evaluation NONPF21
AcknowledgementsWe acknowledge the support of the Division of Nursing Bureau of Health Professions(BHPr), and the Health Resources Services Administration Department of Health andHuman Services (DHHS). The funding for this project was received through the granttitled: Nurse Practitioners Managing Co‐occuring Mental and Physical Disorders. Thecontent and conclusions presented are those of the authors and not the position, policyor endorsement of the Division of Nursing, BHPr, DHHS or the US Government.OSU CON Cyber‐Evaluation NONPF22
OSU CON Cyber‐Evaluation NONPF 3 Evaluation Strategies Non-exam evaluation methods - Preceptor calls, blogs . Using the survey tool to evaluate student performance. OSU CON Cyber‐Evaluation NONPF 15 . security issue, evaluation Now: Remote, no cost, collaboration in Part II, exam integrity, effective
