Transcription
4/12/2011Fi t FFirstFridaysidWWebinarbiSSeries:iMedical Education Group (MEG)Needs Assessment in Grant Making: AnInteractive WebinarApril 1, 2011ProvideInsights intoMEGOperationsShareUp-To-DateInformationHow CanPfizer ImproveProcesses?ShareBest PracticesWebinarSeriesGoalsRespond toOutstandingQuestionsFromProviders1
4/12/2011Agenda: Needs Assessment Welcome Needs Assessment in Grant Making: AnInteractive Webinar — Robert EE. KristofcoKristofco, MSWMSW, FACMEFACME,and Susan Connelly, PharmD, MBA Q and A Closing RemarksNew CGA – Women’s HealthApplications Due May 1, 2011 Through continuing professional education and practiceimprovement initiatives, enable primary care providers to focus thedialogue with their adult female patients and advance thei di id li d care andindividualizedd ttreatmenttt ffor conditionsditiththatt are ttraditionallyditillunder-discussed and more prevalent in women:–––––depressionfibromyalgia and pain managementmenopause – vasomotor symptoms and other related health-effectsurogenital atrophy, urinary incontinence and bladder infectionssexual health Applicants are strongly encouraged to include use of innovativeeducational methods such as patient simulationsimulation, academic detailingor point-of-care learning as part of their plans. Practice-based orteam-based learning approaches are also considered importantcomponents in a successful proposal. Projects should focus onholistic approaches to patient-centered care of female patients thattranscend traditional knowledge-based updates and similaractivities.2
4/12/2011Needs Assessment in Grant Making:An Interactive WebinarRobert E. Kristofco, MSW, FACME,andSusan Connelly, PharmD, MBA, CCMEPPfizer Inc.ObjectivesAt the completion of this session participantsshould be better able to:1. Describe the characteristics of quality needsassessment2. Identify needs assessment data that best informprogram planning and3. Discuss the importance of using the highest qualityneeds assessment data available for educationalcontent and format development3
4/12/2011Needs Assessment“Evidence from systematic reviews of theliterature shows that programs in continuingmedical education that are predicated on wellconducted needs assessments are effectivein changing doctors' behaviors.”Source: Fox RD, Bennett NL, Learning and change: implications forcontinuing medical education. BMJ.1998;316:466–9.Needs Assessment“Exclusive reliance on formal needsassessmentt couldld renderd educationdtianinstrumental and narrow process rather thana creative, professional one”Source: Grant J, Stanton F, The effectiveness of continuingprofessional development. Edinburgh: Association for the Study ofMedical Education,2000. (ASME medical education booklet.)4
4/12/2011The Challenges“A fundamental gap remains between thelearning needs of the individual practitionerand the priority educational needs identifiedfby bodies for continuing medical educationfor course offerings.”Source: Norman G, The need for needs assessment in continuingmedicaldi l education.dtiBMJBMJ. 20042004;328(7446):999.328(7446) 999Defining Terms“Educational needs can be defined as theinterests or perceived needs of a whole targetaudience and can be identified throughsurveys, focus groups, analysis of regionalpractice patterns, and evaluations of CMEprograms.”An example of an educational needs assessment is a discrepancy or “gap”analysis, in which current practice behavior is compared with an ideal oraccepted standard of practice.Source: Norman G, The need for needs assessment in continuingmedical education. BMJ. 2004;328(7446):999.5
4/12/2011Needs Assessment in Grant Review The needs assessment is minimal or does not exist The needs assessment has a literature review but does not goyarticulatingg the science area of need and/or includes onlyybeyondbroad generalized data Goes beyond basic literature review and begins to link the sciencefoundation to an actual need for education The needs assessment has specific localized quantitative datasources to document practice gaps In addition to having documentation of an actual practice gap, theprovider has also established the need for education as a strategy inpotentially helping to close the gapGilbody SM. Psychol Med. 2002;32:1345-1356; Grant J. BMJ. 2002;324:156-159.Harrison LM. Public Health Rep. 2005;120(1):28-34. Brazil K. Int J Palliat Nurs. 2005;11(9):475-480. Turner S.Occup Med. 2004;54:14-20. Ratnapalan S. Can J Clin Pharmacol. 2004;11(1):150-155.Needs Assessment Examples Examples provided are, in some cases, smallselections from lengthy requests Excerpts selected to illustrate various points We have our opinions, we want yours 6
4/12/2011You make the call Example 1 Practice Gap 1: PCPs are not identifying patients with RA in the earlystages after disease onset, thus delaying the initiation of DMARDtherapy and leading to greater patient disability and worse outcomes– “ Interviews with rheumatologists, PCPs and patients, along with outcomemeasures from programs given in 2008 and 2009, indicated that even PCPswho are aware of the need for early RA diagnosis do not generally know howto correctly identify RA in its early stages and need to have more instructionon this topic.”– emphasized by, a patient with RA. She stated, “As funny as itsounds, I consider myself lucky that my disease onset was very aggressive andsevere, so I looked like a textbook picture of RA. It took less than 8 weeksfrom the time of my first symptoms to my diagnosis. I’ve known manypatients who had a less fortunate disease course. I have a friend—we’re thepsame age, have the same diagnosis, but it took about 2 years from onset todiagnosis, and we don’t look like we have the same disease. She has physicaljoint damage that I don’t have, and a more complicated profile—not justbecause I was diagnosed early, but I also started aggressive therapy veryearly.”You make the call Example 1 emphasizedPractice Gap 1: PCPsare not identifyingpatients withRARA.in theSheearlyby,a patientwithstages after disease onset, thus delaying the initiation of DMARDstated,“Asfunnyas it sounds,I considermyselftherapy andleadingto greaterpatient disabilityand worseoutcomesluckythat myonsetvery aggressive– “ Interviewswithdiseaserheumatologists,PCPswasand patients,along with outcomefromso programsgiven likein 20082009, indicatedthat evenandmeasuressevere,I lookedaandtextbookpictureof PCPsRA.who are aware of the need for early RA diagnosis do not generally know howIt tooklessthan8weeksfromthetimeofmyfirstto correctly identify RA in its early stages and need to have more instructionsymptomsto my diagnosis. I’ve known manyon this topic.”– emphasizeda patientwith RA. Shediseasestated, “As funnyas it Ipatientswhoby,had a lessfortunatecourse.sounds, I consider myself lucky that my disease onset was very aggressive andhavea friend—we’rethe picturesameof age,havethe8 weekssamesevere,so I looked like a textbookRA. It tookless thanfrom the timeof mysymptomsdiagnosis.fromI’ve knownmanytodiagnosis,butit firsttookaboutto 2myyearsonsetpatients whoandphad a less fortunatediseaselikecourse.I havea friend—we’rethediagnosisdiagnosis,dont lookwehavethe onsetsamesame age, have thewesamedon’tdiagnosis,but it took about2 years fromtodisease.jointI don’tdiagnosis,Sheand wehasdon’tphysicallook like we havethedamagesame disease.thatShe hasphysicaljoint anddamageathatI don’tcomplicatedhave, and a more complicatedprofile—nothave,moreprofile—notjust justbecause I was diagnosed early, but I also started aggressive therapy verybecauseearly.” I was diagnosed early, but I also startedaggressive therapy very early.”7
4/12/2011Polling Question In this example, the provider included a directquote from a patient.Do you feel this (check all that apply)1.2.3.4.55.Can stand alone in support of the educational needIs a nice story to illustrate the educational needHelps to triangulate the educational needIs good but so much more is neededDoes not add much to support the need foreducation6. Does not capture the scope of the educationalneedYou make the call Example 1 Practice Gap 1: PCPs are not identifying patients with RA in the earlystages after disease onset, thus delaying the initiation of DMARDtherapy and leading to greater patient disability and worse outcomes–“ Interviews“ Interviews with withrheumatologists,PCPs and patients,along withoutcomerheumatologists,PCPsandmeasures from programs given in 2008 and 2009, indicated that even PCPspatients,alongoutcomemeasuresfromknow howwho are awareof thewithneed forearly RA diagnosisdo not generallyto correctly identifyRA in inits earlystagesand 2009,need to havemore instruction programsgiven2008andindicatedthaton this topic.”evenPCPswhoareawareoftheneedforearlyRA– emphasized by, a patient with RA. She stated, “As funny as itdiagnosisdo notgenerallyhowwastoverycorrectlysounds, I considermyselflucky that my knowdisease onsetaggressive andsevere, soRAI lookedlike aearlytextbookpicture ofandRA. It needtook lesstothan8 weeksidentifyin itsstageshavemorefrom the time of my first symptoms to my diagnosis. I’ve known manyinstructiononthistopic.”ppatients who had a less fortunate disease course. I have a friend—we’re thepsame age, have the same diagnosis, but it took about 2 years from onset todiagnosis, and we don’t look like we have the same disease. She has physicaljoint damage that I don’t have, and a more complicated profile—not justbecause I was diagnosed early, but I also started aggressive therapy veryearly.”8
4/12/2011Polling Question In this example, the provider cites interviews withHCPs and patients along with outcomes fromprevious activities. Do you feel this statement (check all that apply)1. Is a qualitative summary of data that supports aneducational need2. Is a general summary that provides little informationto support an educational need3. Is good start but so much more is neededYou make the call Example 1 continued– “ Survey outcomes from programs in 2008 and 2009 supported theseconclusions. In answers to prepre‐activityactivity questions, 74% of the 3789 participantsindicated that they felt confident to extremely confident in identifyingpatients with RA; however, fully 85% were not able to correctly identifydiagnosis criteria. This indicates that many PCPs incorrectly believe that theycan identify RA in its early stages, even while they do not have a properunderstanding of diagnostic criteria, and may therefore be missing manypatients who have early disease.” Educational Need: PCPs need to understand how to correctly identify theearly signs of RA in order to diagnose the disease soon after onset. Simplemethods for making a provisional diagnosis of RARA, such as the squeezetest, should be incorporated into practice. The role of early, aggressiveDMARD therapy in preventing disease progression should also beunderstood9
4/12/2011You make the call Example 1 continued “ Surveyoutcomes from programs in 2008 and– “ Survey outcomes from programs in 2008 and 2009 supported these2009009supportedsuppoted thesetto esecoconclusions.c us o 74%s Inaanswerss eparticipantss toconclusions.In answersprepre‐activityactivityquestions,of nfidentinidentifyingpre-activityquestions, 74% of the 3789 participantspatients with RA; however, fully 85% were not able to correctly identifyindicatedthattheyfeltthatconfidentto extremelydiagnosis criteria. Thisindicatesmany PCPs incorrectlybelieve that theycanidentifyRAinitsearlystages,evenwhiletheydo nothavehowever,a properconfident in identifying patients withRA;understanding of diagnostic criteria, and may therefore be missing manyfully85%notable to correctly identifypatientswho werehave yidentifyPCPsEducationalNeed:PCPs needto understandhowthatto correctlytheearlysigns of RA inorder to diagnosethe diseasesoon afterRAonset.incorrectlybelievethat theycan identifyin SimpleitsmethodsprovisionalRA,suchas theasqueezeearlyl forstages,t making aevenwhilehildiagnosisththey ofddoRAnotthhavepropertest, should be incorporated into practice. The role of early, DMARD therapy in preventing disease progression should also beunderstoodtherefore be missing many patients who have earlydisease.”Polling Question The example continues by providing moredetail related to information collected fromprevious activities.Do you feel this (check all that apply)1. Is a quantitative summary of data that supports aneducational need2. Is good start but so much more is neededgeneral summaryy that pprovides little3. Is a ginformation to support an educational need4. Is generalized data that does not truly support aneducational need10
4/12/2011You make the call Example 2Summary Table of Integrated ElementsPolling Question Example 2 included a summary table ofintegrated elements.Do you feel this (check all that apply)1. Nicely illustrates the linkage between needs,design, and outcomes2. Can be useful if fully supported by a dialogexplaining each element3. Is a simplified summary that overly generalizesinformation4. Can falsely give the impression of understandingby the provider11
4/12/2011You make the call Example 3 “A clear need exists for improved maintenance treatment forindividuals with bipolar disorder living in rural environmentsas demonstrated by the negative outcomesoutcomes, including relapse,relapsewhich many of these patients experience .” “ The APA guidelineX and guideline watchY for the treatmentof bipolar disorder are readily available but underused. Only30% to 40% of individuals with bipolar disorder receivetreatment based on current guideline recommendations.Z ” “One state that has a disproportionate share of shortageareas is ,, where the vast majority of its 256 counties, mostof them rural, are federally designated as mental healthprofessional shortage areas (please see map). In these areas,primary care clinicians are largely responsible for the care ofpatients with bipolar disorder”You make the call Example 3 “A clear need exists for improved maintenance treatment forindividuals with bipolar disorder living in rural environmentsasdemonstratedthe negativeoutcomesoutcomes, includingrelapserelapse,“OneOnestate thatbyhasa disproportionateshare ofwhichmanyofthesepatientsexperience .”shortage areas is , where the vast majority of “ The APA guidelineX and guideline watchY for the treatmentitsbipolar256 counties,mostof themrural,federallyofdisorder arereadilyavailablebut areunderused.Onlydesignatedmental withhealthprofessionalshortage30%to 40% ofasindividualsbipolardisorder receivetreatmentbased seeon currentareas (pleasemap).guidelineIn theserecommendations.areas, primaryZ ” “Onethat hasa disproportionateshare forof shortagecare statecliniciansarelargely responsiblethe care ofareas is ,, where the vast majority of its 256 counties, mostpatientsti t rural,withitharebipolarbi federallyl didisorder”d ” as mental healthofthemdesignatedprofessional shortage areas (please see map). In these areas,primary care clinicians are largely responsible for the care ofpatients with bipolar disorder”12
4/12/2011Polling Question Example 3 includes state specific data.Do you feel this (check all that apply)1. Illustrates a local educational need12. Appropriately draws conclusions about the need foreducation based on state deficiencies in care3. Is good start but so much more is needed4. Is just a broad generalization based on populationdata5. Inappropriately draws conclusions about the needfor education based on state deficiencies in careYou make the call .Example 4 “ .In 1974, MRSA infections accounted for just 2% of thetotal number of staph infections; by 1995 it was 22%, and2004 had increased 63%. A recent study by the Centers forDisease Control and Prevention (CDC) found that MRSA killsmore people in the United States than any other infectiontracked by the CDC, including AIDS. [1] The CDC estimatedthat 94,360 invasive MRSA infections occurred in the UnitedStates in 2005; about 20% ‐ 18,650 ‐ of these were associatedwithi h death.d h [1] AnotherA h studyd foundfd thath hhospitalizationsi li iiinthe United States due to MRSA more than doubled between1999 and 2005. [5]”13
4/12/2011You make the call .Example 4 “ .In 1974, MRSA infections accounted for just 2% of thetotal number of staph infections; by 1995 it was 22%, and2004 had increased 63%. A recent study by the Centers forDiseaseControlandfoundPrevention(CDC) found that MRSA“Anotherstudythat hospitalizationsin thekillsmore people in the United States than any other infectionUnited States due to MRSA more than doubledtracked by the CDC, including AIDS. [1] The CDC estimatedbetween 1999 and 2005.”that 94,360 invasive MRSA infections occurred in the UnitedStates in 2005; about 20% ‐ 18,650 ‐ of these were associatedwithi h death.d h [1] AnotherA h studyd foundfd thath hhospitalizationsi li iiinthe United States due to MRSA more than doubled between1999 and 2005. [5]”Polling Question Example 4 includes national data.Do you feel this (check all that apply)1. Illustrates a national educational need12. Appropriately draws conclusions about the need foreducation based on national incidence3. Is good start but so much more is needed4. Is just a broad generalization based on populationdata5. Inappropriately draws conclusions about the needfor education based on national incidence14
4/12/2011You make the call .Example 5“The content of this medical education activity wasdetermined by rigorous assessment of educationalneed and includes expert faculty assessment, literaturereview, medical practice and new medical knowledge”Polling Question Example 5 includes a description ofmethodology for their needs assessment.Do you feel this (check all that apply)1. Can stand on its own to support an educationalneed2. Is good start but so much more is needed3. Without supporting dialog, is not an adequatestatement to support an educational need15
4/12/2011You make the call .Example 6Sources LiteratureLittreviewiInterviews with expertsAlzheimer’s Association (alz.org)Evaluations/surveys from half‐day session ongeriatrics and half‐day end‐of‐life sessions at (annualconvention)Narrative provided for each section of outline including data charts andreferences.Polling Question Example 6 includes a list of various sources usedto assess the need as well as a supportingdialog. Without reading the full needs assessment, doyou feel this (check all that apply)1. Can stand on its own to support an educationalneed2. Is good start but so much more is needed16
4/12/2011Defining Terms“Learning needs are personal, specific, andidentified by the individual learner throughpracticeti experience,ireflection,fl tiquestioning,ti ipractice audits, self assessment tests, peerreview, and other sources”.An exploration of the issues that created the gap,in individual cases, would identify the learningneeds.needsSource: Norman G, et al. The need for needs assessment in continuingmedical education. BMJ. 2004;328(7446):999-1001.Defining TermsOther classifications of needs include: felt needs (what people say they need)expressed needs (expressed in action)normative needs (defined by experts)comparative needs (group comparison).Source: Lockyer J. Needs assessment: lessons learned. J Contin EducHealth Prof. 1998;18:190–192.17
4/12/2011Types of Assessment Methods ToIdentify Learning Needs Reflection on action and reflection in action-thinkingback on, or reviewing present performance Self assessment by diaries, journals, log books,weekly reviewsreviews-recordingrecording experiences for review Peer review Facilitated note keeping and reflection around sentinelpatients Critical incident review and significant eventauditing-methods used in quality improvement appliedto identifying learning needs Practice review-routine review of practice using chartsand other measures from practiceSource: Grant J, Learning needs assessment: assessing the need BMJ.2002;324:156.Types of Assessment Methods ToIdentify Educational Needs Gap or discrepancy analysis-comparingperformance to an accepted standard Surveys Focus groups Analysis of regional practice patterns Evaluations of CME programs Epidemiologic dataSource: Norman G, et al. The need for needs assessment in continuingmedical education. BMJ. 2004;328(7446):999-1001.18
4/12/2011ACCME Criteria for Educational PlanningEssential Area 2: Educational PlanningThe provider must:Element 2:1:“Use2:1: Use a planning process(es) that links identifiededucational need with a desired result ”Element 2.2:“use needs assessment data to plan CME activities”Providers are required to describe: How they identify the practice gap(s) of learners How theyy identifyy the educational needs of their learners thatunderlie the professional practice gap(s) that they haveidentified How they incorporate these needs into CME activitiesFactors Considered In Grant Review Qualifications/Experience of Provider and EducationalPartners Needs assessment that identifies practice gaps ofthe target audience. Learning Objectives that are measurable Educational design that incorporates multiple methods and isbased on adult learning principles Evaluation/Outcomes designed to assess changes inknowledge, competence or performance of target audience Quality/Impact‐the likelihood that the intervention willimpact practice and/or improve patient care Importance‐Benefits to patients19
4/12/2011Some Observations Effective and appropriate translation of science to practicerequires rigorous needs assessment Needs assessment is a central element in an integratededucational pplanningg and decision makingg processp Models that combine assessment of learning and educationalneeds are the ideal Learner engagement is a critical success factor The range of techniques available to conduct both learningand educational needs assessments is expanding to includenew technologies like social mediaA Final Thought“ the literature suggests that, at least inrelation to continuing professionaldevelopment, learning is more likely to lead tochange in practice when needs assessmenthas been conducted, the education is linkedto practice, personal incentive drives theeducational effort, and there is somereinforcement of the learning.”Source: Grant J. Learning needs assessment: assessing the need. BMJ2002;324:156.20
4/12/2011Questions and AnswersAdditional Needs Assessment Resources1.2.3.4.5.6.77.Campbell C, Gondocz T, Identifying the needs of the individual learner In The ContinuingProfessional Development of Physicians: From Research to Practice Davis D, Barnes BE, Fox R,Eds 2003 AMA Press pp 81-96Green J, Leist J, Determining needs from the perspective of institutions or organizations providingcare in The Continuing Professional Development of Physicians: From Research to Practice DavisD, Barnes BE, Fox R, Ed’s 2003 AMA Press pp 97-111Moore D, Needs assessment in the new healthcare environment: combining discrepancy analysisand outcomes to create more effective CME. J Contin Educ Health Prof. 1998;18: 133-141Mann K, Not another survey! Using questionnaires effectively in needs assessment J ContinEduc Health Prof. 1998;18: 142-149Tipping J, Focus Groups: a method of needs assessment J Contin Educ Health Prof. 1998;18:150-154Jennett PA, Affleck L, Chart audit and chart stimulated recall as methods of needs assessment incontinuing professional health education J Contin Educ Health Prof. 1998;18: 163-171Rethans J,J Needs assessment in continuingcontin ing medical ededucationcation throthroughgh standardistandardizeded patientsJ Contin Educ Health Prof. 1998;18: 172-17821
4/12/2011Until Next Time Please join us for our next webinar – Sticky Education– Brian McGowan, PhD– Friday, May 6, 2011– 11am11ET We are in an open grant window – please submit yourgrant requests before April 15– Remember to check the revised goals statements See what providers are doing to move education forward– PfizerMedEdGrants Resource Center– Publications– First Friday Webinars22
An Interactive Webinar Robert E. Kristofco, MSW, FACME, and Susan Connelly, PharmD, MBA, CCMEP Pfizer Inc. Objectives At the completion of this session participants should be better able to: 1.Describe the characteristics of quality needs assessment 2.Identify needs assessment data that best inform program planning and
