Transcription
ViewpointLisa Nordan,1 Lorrie Blanchfield,1 Shehzad Niazi,1,2 Juliet Sattar,1,3Courtney Elizabeth Coakes,1 Ryan Uitti,1,4 Michael Vizzini,1,5James M Naessens,1,6 Aaron Spaulding1,71Mayo Clinic Robert D. andPatricia E. Kern Center for theScience of Health Care Delivery,Mayo Clinic, Jacksonville,Florida, USA2Psychiatry and Psychology,Mayo Clinic, Jacksonville,Florida, USA3Operational Support, MayoClinic, Jacksonville, Florida, USA4Neurology, Mayo Clinic,Jacksonville, Florida, USA5Administration, Mayo Clinic,Jacksonville, Florida, USA6Division of Health Care Policyand Research, Mayo Clinic,Rochester, Minnesota, USA7Division of Health Care Policyand Research, Mayo Clinic,Jacksonville, Florida, USACorrespondence toDr Aaron Spaulding, HealthSciences Research, Mayo Clinic,Jacksonville, FL 32224, USA; spaulding. aaron@m ayo. eduReceived 5 June 2018Revised 22 June 2018Accepted 25 June 2018 Author(s) (or theiremployer(s)) 2018. Nocommercial re-use. See rightsand permissions. Published byBMJ.To cite: Nordan L,Blanchfield L, Niazi S, et al.BMJ Qual Saf Epub ahead ofprint: [please include DayMonth Year]. doi:10.1136/bmjqs-2018-008426Determining how to collect and usepatient-reportedoutcomemeasures(PROMs) continues to be an area ofdiscussion, and in some cases frustration.1–5 Gaining a greater depth ofknowledge concerning a patient’s initialhealth status as well as improvementafter a medical or surgical intervention,would provide a clearer understandingof needed care paths and outcomes oftreatments, oftentimes missing from ourcurrent healthcare processes.6 7 WhilePROMs are not a new idea, the ability toelectronically collect, report and use thedata has become more relevant in recentyears. As such, this work focuses on thechallenges and lessons learnt from implementing electronic PROMs (ePROMs)within a destination medical centre whichprovides team-based comprehensive carefor patients.Implementations in multiple departments and disease specific areas of carethroughout the organisation took placebetween January 2016 and March 2018.The International Consortium for HealthOutcomes Measurement (ICHOM) standard sets were used in each departmentas the primary survey tool, as scorescould easily be calculated and the diseasespecific questionnaires administered atthe point of care demonstrated a goodfit for the organisation’s practices.6 8ICHOM Standard Sets are global, standardised and validated questionnairesthat have been combined into a set thatcaptures information for the provider ata level that patients understand. Pre-existing intake assessments were evaluated within each department and serviceline and subsequently were merged orreplaced with the ICHOM sets to createcustomised comprehensive condition-specific survey sets.In all cases, collection of the surveysets at patient intake provides a baselinefor changes to be monitored as well asprovides context for patient concerns,needs and areas in which clinicians mayneed to further inquire. Follow-up dataare collected after operations or procedures and changes in patient responsesare monitored and assessed. In the firstyear of implementation, 2771 ePROMswere completed with a 99.8% capturerate over four different disease conditions. Ten additional condition-specificareas were added in subsequent years,resulting in over 50 000 ePROMs by June2018 (see figure 1).ChallengesNaturally, implementing ePROMs provedmore difficult in some specialties thanothers. This, in some cases, was due tophysicians being reluctant to use theICHOM standard sets.5 Many departments used tools developed and endorsedby specialty societies which promotedhesitation when considering conversionto another data collection tool. In addition, bringing front desk staff on boardprior to implementation also proved challenging due to reluctance to take on additional responsibility as well as perceiveddisruptions of the workflow. In both cases,mapping workflows, eliminating redundant steps as well as defining how the newstandard sets could be either combinedwith or collected in place of previoussets provided evidence on how changescould prove beneficial. Once the net zeroimpact, and in some cases, time savingson work throughputs were demonstrated,the staff was typically receptive to theNordan L, et al. BMJ Qual Saf 2018;0:1–5. doi:10.1136/bmjqs-2018-0084261BMJ Qual Saf: first published as 10.1136/bmjqs-2018-008426 on 18 July 2018. Downloaded from http://qualitysafety.bmj.com/ on 20 July 2018 by guest. Protected by copyright.Implementing electronic patientreported outcomes measurements:challenges and success factors
ViewpointePROM collection over time.new process. This evidence demonstration and workprocess flow communication has been previously indicated as important to managing change, particularlywith respect to ePROM implementation.5 9Furthermore, some staff ’s history with previousattempts to implement and collect ePROMs createdimplementation barriers. Initial trials with homegrowniPad solutions were unstable due to problems withWI-FI internet connection, complex logins designed tocreate secure connections, and the fact that computeradaptive testing and branching logic capabilities werenot possible. The new platform and technology solvedthese issues; however, dedicated communication,demonstration and training were still required to helpovercome these barriers. Again, this is consistent withprevious research focused at change management andtechnology acceptance.10–12Patient acceptance of remote capture of questionnaires also presented a challenge. It was found thatwhen questionnaires were sent to the patient’s email orcell phone for completion rather than being presentedon site, capture rates were lower. This may havebeen partially due to the email appearing as a genericemail coming from the organisation and as such wasdisregarded as ‘junk’ mail. As a result, it was necessary to request clinicians interacting with the patientto provide context and set expectations with patientsfor the remote completion of questionnaires. This2process of communication and expectation settingdramatically increased remote capture rates, which isconsistent with findings of Atherton et al.13 In addition, customising the email to come from the providerversus the organisation was a key driver in acceptanceand capture. The patient received an email from theprovider to complete a questionnaire that was thenused as part of their appointment. That personalisationcreated a connection between the patient and provider,as it was no longer presented like other organisationalquestionnaires that previously may have been deleted.Finally, there were concerns revolving around dataand security. Initial concerns about data storage andsecurity revolved around using a third-party vendorrather than an internal technology solution. Thiswas resolved through use of a HIPAA compliant datacloud owned by the organisation to store and maintain the data. Subsequently, there were concernsaround ensuring the information collected in theePROMs tool were included in the electronic healthrecord (EHR). These concerns were resolved in twodifferent manners. In the absence of an interface withthe current EHR, physicians were required to login toan additional browser window, they were then able toimport narrative reports into their notes in the EHR.Additionally, patient-completed questionnaires wererouted through the organisation’s Health InformationManagement Systems so that appropriate documentsNordan L, et al. BMJ Qual Saf 2018;0:1–5. doi:10.1136/bmjqs-2018-008426BMJ Qual Saf: first published as 10.1136/bmjqs-2018-008426 on 18 July 2018. Downloaded from http://qualitysafety.bmj.com/ on 20 July 2018 by guest. Protected by copyright.Figure 1
ViewpointLessons LearntMinimising workflow disruption is essential increating positive reception of the process changes bystaff.5 14 In order to minimise workflow disruption,we observed and interviewed staff in order to betterunderstand their current processes prior to implementation. In addition, we completed workflow mappingactivities both preintervention and postintervention toshow time and workflow savings.9 Ultimately, the keyto gaining staff members’ participation was enthusiasmand support from leadership, face to face support,collaborative efforts in customising the tool as well asdata that demonstrated tangible benefits to the practice and process of care.15The turning point of the implementation was findinga physician champion to engage his colleagues bydemonstrating the flexibility of the tool, highlightingefficiencies in the overall work process and convincinghis colleagues of the value of the ePROM sets. Consistent with previous inquiry, engagement with physicians, nurses and administrators, that have an interestin using the tools and serve as champions within theirown practices, has proven successful in each of theareas we have implemented the ePROMs.16 In particular, front-line staff perceptions of ePROMs usabilityare critical to implementation, as they are providingthe iPad to the patient, explaining the process and areresponsible for ensuring that patients complete questionnaires. Including front-line staff in the processflow discussions increased acceptance and willingnessto support the ongoing project. In addition, having acoordinator or manager on the implementation projectis extremely beneficial. Peer-to-peer communicationhas proven influential in increasing user acceptance asone’s colleagues are best able to express practice andtime benefits.Flexibility with the ePROM platform was alsocrucial in the successful adaptation of the tool bystaff members. The platform had to be customisablein a number of ways to meet each practice’s specificneeds, provider preferences and workflows; a team’sability to interact in the process and make changes tobetter suit their needs also creates a collaborative environment that allows physicians and staff to becomeinvested in the project.17 The patient questionnaireswere built with a toolkit that allowed multiple-choicequestions, multiselect questions, sliding scales, drawingand diagramming capabilities, free text as well as theability to provide visuals and graphics to better engagepatients. Also appealing to providers was the ability tosend questionnaires remotely via SMS text messagingNordan L, et al. BMJ Qual Saf 2018;0:1–5. doi:10.1136/bmjqs-2018-008426or email. This allows the patient to complete the questionnaires before their visit, and as a backup, patientswere able to complete questionnaires on an iPad in thelobby on arriving for their appointment.18In addition, providers could customise the ePROMsystem to meet their research or practice needs. Forinstance, they are able to define the ‘weight’ of individual questions, enabling them to create customscoring systems tailored to particular conditions orthe needs of specific patient populations. Prompts foradditional follow-up questionnaires or provider interventions can be set based on specified scoring flags andthresholds. By applying scoring algorithms, patientresponses can then be displayed to the provider withcolour coding, allowing the provider to quickly identifythe patient’s top concerns. For example, a red, orange,yellow and green system could be used to illustratesymptom severity to flag patients’ priority concerns inindividual symptoms such as sleep, appetite, energy andconcentration. Furthermore, the ePROM platform hasthe ability to prompt follow-up questionnaires shouldthe patient’s score reach a predefined risk thresholdfor related disease or psychosocial condition. Resultswere immediately available and populated the provider’s inbox, accessible on the platform through any webbrowser. As a result, the provider could easily reviewpatient responses in advance of the care visit to usethat information with the patient to better focus ontheir specific needs. Moreover, triggers set to specificquestion responses or scores can result in an email toclinical staff prompting immediate contact with thepatient when needed. The ability to easily add new andmodify existing questionnaires allows for a smootherimplementation process as specific practice needscan be addressed in real time. This also facilitates theprovision of a scalable solution which can expand toadditional condition specific areas of care.Finally, encouraging individual physicians tocustomise the output summary using natural languageprocessing was also important in the adoption of thenew ePROM system. The natural language outputexamines the way a provider normally dictates theirclinical note and uses that information to build asummary from the patient’s questionnaire that couldbe used as the basis for the clinical note for the patient’svisit. In turn, this decreases the physician work effort,and it is worth noting that there is substantial evidenceshowing that patient-reported symptoms more accurately reflect their health status than physician elicited responses.19 Incorporating ePROM results intothe clinical encounter helps to engage the patient andprovider in more relevant discussion that is focused onthe patient’s needs.4 5 This allows for a more fruitfuland focused conversation concerning patient needs,desires and thoughts about their visit or about thestatus of their recovery. It also increases the depth ofthe conversations allowing for more shared decisionsregarding treatment paths. This opportunity, plus the3BMJ Qual Saf: first published as 10.1136/bmjqs-2018-008426 on 18 July 2018. Downloaded from http://qualitysafety.bmj.com/ on 20 July 2018 by guest. Protected by copyright.could be scanned into the EHR. Each of these barriersis a reality of a complex organisational IT environmentand is an important consideration for cost controls andsecurities. As such, it is necessary to allocate time andenergies to ensure the framework of committees andorganisational policies and procedures are followed.
ViewpointChallenges and lessons learnt summaryChallengesLessons learntUsing scoring algorithms and colour codingto allow providers to quickly focus onpatients' top concerns.Ability to customise and use computeradaptive modelling reduced questionnairefatigue for patients.Current and future state workflows wereProviders and desk staffreluctant to disrupt current completed replacing paper process withelectronic components demonstrating a networkflow and paperzero impact.process.Providers were hesitant due Physicians providing context and settingto previous failed attempts expectation of questionnaire beingat PROM implementation. delivered via email to patient increasedcapture rates.Platform flexibility was crucial for adoptioninto the practice.Reduced ePROM completion Customising heading and email addressrate with remote captureto come from provider versus institutionincreased patient capture rate.Using HIPAA compliant cloud owned by theSecurity concerns of thirdorganisation to resolve concerns on dataparty vendor versus insecurity.house solution.EHR data managementNatural language output decreasedprovider documentation time. Scannablefiles generated from ePROM responses sentto Information Management.EHR, electronic health record; ePROM, electronic PROMs; HIPAA, HealthInsurance Portability and Accountability Act; ICHOM, InternationalConsortium for Health Outcomes Measurement; PROM, patient-reportedoutcome measures.Providers were reluctant touse ICHOM standardisedsets versus familiar specialtysociety tools.ability to track scores for a specific patient, for physicians or for a department in general, allowed forgreater buy-in from patients, physicians and administrators. Ultimately, these data may also prove beneficial when negotiating insurance contracts, physicianrecertification and managing referring providers relationships. A summary of the challenges and lessonslearnt is available in table 1.ConclusionThis ePROM system has helped capture how patientsperceive their health and has proven to be essentialin assessing healthcare quality and outcomes in ourorganisation. As we continue to expand to additionalconditions within our organisation, lessons learntand staff input will continue to be our main drivers.Having key stakeholders at the design session willcontinue to improve our implementation strategyand success. By expanding our collection of ePROMsto other conditions, we hope to transform the practice with the data collected, to improve patient care,patient outcomes and ultimately provide patients withvaluable information to allow them to make decisions on their healthcare supported by shared decision-making models.4Funding The authors have not declared a specific grant for thisresearch from any funding agency in the public, commercial ornot-for-profit sectors.Competing interests None declared.Patient consent Not required.Provenance and peer review Not commissioned; internallypeer reviewed.References1 Deshpande PR, Rajan S, Sudeepthi BL, et al. Patient-reportedoutcomes: A new era in clinical research. Perspect Clin Res2011;2:137.2 Rose M, Bezjak A. Logistics of collecting patient-reportedoutcomes (PROs) in clinical practice: an overview and practicalexamples. Qual Life Res 2009;18:125–36.3 Snyder CF, Aaronson NK, Choucair AK, et al. Implementingpatient-reported outcomes assessment in clinical practice:a review of the options and considerations. Qual Life Res2012;21:1305–14.4 Valderas JM, Kotzeva A, Espallargues M, et al. Theimpact of measuring patient-reported outcomes in clinicalpractice: a systematic review of the literature. Qual Life Res2008;17:179–93.5 Nelson EC, Eftimovska E, Lind C, et al. Patient reportedoutcome measures in practice. BMJ 2015;350:g7818.6 Porter ME, Larsson S, Lee TH. Standardizing PatientOutcomes Measurement. N Engl J Med 2016;374:504–6.7 Rathert C, Wyrwich MD, Boren SA. Patient-centered care andoutcomes: a systematic review of the literature. Med Care ResRev 2013;70:351–79.8 Kelley TA. International Consortium for Health OutcomesMeasurement (ICHOM). Trials 2015;16:O4.9 Kaplan RS, Witkowski M, Abbott M, et al. Usingtime-driven activity-based costing to identify valueimprovement opportunities in healthcare. J Healthc Manag2014;59:399–412.10 Spaulding A, Kash BA, Johnson CE, et al. Organizationalcapacity for change in health care: Developmentand validation of a scale. Health Care Manage Rev2017;42:151–61.11 Kash BA, Spaulding A, Johnson CE, et al. Success factors forstrategic change initiatives: a qualitative study of healthcareadministrators' perspectives. J Healthc Manag 2014;59:65–81.12 Marangunić N, Granić A. Technology acceptance model:a literature review from 1986 to 2013. Univ Access Inf Soc2015;14:81–95.13 Atherton PJ, Burger KN, Pederson LD, et al. Patient-reportedoutcomes questionnaire compliance in Cancer CooperativeGroup Trials (Alliance N0992). Clin Trials 2016;13:612–20.14 Boyce MB, Browne JP, Greenhalgh J. The experiences ofprofessionals with using information from patient-reportedoutcome measures to improve the quality of healthcare:a systematic review of qualitative research. BMJ Qual Saf2014;23:508–18.15 Van Der Wees PJ, Nijhuis-Van Der Sanden MW, Ayanian JZ,et al. Integrating the use of patient-reported outcomes forboth clinical practice and performance measurement: views ofexperts from 3 countries. Milbank Q 2014;92:754–75.16 Rotenstein LS, Agarwal A, O'Neil K, et al. Implementingpatient-reported outcome surveys as part of routine care:lessons from an academic radiation oncology department. J AmMed Inform Assoc 2017;24:964–8.Nordan L, et al. BMJ Qual Saf 2018;0:1–5. doi:10.1136/bmjqs-2018-008426BMJ Qual Saf: first published as 10.1136/bmjqs-2018-008426 on 18 July 2018. Downloaded from http://qualitysafety.bmj.com/ on 20 July 2018 by guest. Protected by copyright.Table 1
ViewpointNordan L, et al. BMJ Qual Saf 2018;0:1–5. doi:10.1136/bmjqs-2018-008426reported outcomes in clinical care. Med Care2015;53:153–9.19 Chung AE, Basch EM. Incorporating the patient's voice intoelectronic health records through patient-reported outcomes asthe "review of systems". J Am Med Inform Assoc 2015;22:914–6.BMJ Qual Saf: first published as 10.1136/bmjqs-2018-008426 on 18 July 2018. Downloaded from http://qualitysafety.bmj.com/ on 20 July 2018 by guest. Protected by copyright.17 Santana MJ, Haverman L, Absolom K, et al. Training cliniciansin how to use patient-reported outcome measures in routineclinical practice. Qual Life Res 2015;24:1707–18.18 Jensen RE, Rothrock NE, DeWitt EM, et al. The role oftechnical advances in the adoption and integration of patient-5
4Neurology, Mayo Clinic, Jacksonville, Florida, USA 5 . Correspondence to Dr Aaron Spaulding, Health Sciences Research, Mayo Clinic, Jacksonville, FL 32224, USA; spaulding. aaron@ mayo. edu Received 5 June 2018 Revised 22 June 2018 . browser. As a result, the provider could easily review patient responses in advance of the care visit to use .
