WIXLIB

Medgen EHR – Usability Test Report170.315.G.3 – SAFETY ENHANCED DESIGNDATE: 02/01/2018Page 2

Medgen EHR Letter of Attestation for Test Criteria: 170.315.G.3:Safety Enhanced DesignDate: 02/01/2018This letter is being submitted in attestation to the accuracy of the information contained in the followingMedgen EHR Safety-Enhanced Design Usability Report. This report is being submitted as part of the EHRcertification requirements outlined in 170.315(g)(3)- Safety Enhanced Design.Regards,David DeBlasioSenior Software DeveloperPage 2

THIS PAGE HAS BEEN INTENTIONALLY LEFT BLANKPage 3

Medgen EHR Version 7.0 Usability Test ReportProduct Name: Medgen EHRVersion: 7.0Date of the testing: 12/21/17Date of Report: 01/01/18Report Prepared by: David DeBlasioSenior Software DeveloperLocation of the Testing:11 Grace Ave. Suite 208 Great Neck, NY 11021Note: The following study was developed using the NISTIR 7742 template as a guide for reporting ourfindings: Customized Common Industry Format Template for Electronic Health Record Usability Testing.Page 4

Contents1 Executive Summary. 6Major findings . 9Areas for improvement . 92 Introduction . 93 Method . 9Participants. 9Study Design. 10Tasks. 11Procedures . 12Test Location . 12Test Environment . 12Test Forms and Documents. 13Participant Instructions . 13Usability Metrics . 134 Results . 14Data Analysis and Reporting . 14Discussion of the Findings . 16Appendix-1: RECRUITING SCREENER . 18Appendix-2: PARTICIPANT DEMOGRAPHICS. 19Appendix-3: NON-DISCLOSURE AGREEMENT AND INFORMED CONSENT FORM . 20Appendix-4: MODERATOR’S GUIDE. 21Appendix-5: SYSTEM USABILITY SCALE QUESTIONNAIRE . 34Appendix-6: INCENTIVE RECEIPT AND ACKNOWLEDGEMENT FORM . 35Appendix-7: TASK DETAIL MEMORY GUIDE. 36Page 5

1 Executive SummaryA usability test of Medgen EHR version7.0, was conducted on several accounts during 12/11/201712/21/2017 at 11 Grace Ave. Ste 208 Great Neck, NY 11021. The purpose of this test was to test andvalidate the usability of the current user interface, and provide evidence of usability in the EHR under Test(EHRUT). During the usability test, four providers and seven clinical personnel (MA’s and RN’s) took partin the testing that matches the target demographic criteria served as participants and used the EHRUT insimulated, but representative tasks.User-Centered Design ProcessNIST 7742 was referenced for Medgen EHR’s user-centered design process.According to the National Institute of Standards and Technology (NIST) Guide to the Processes Approachfor Improving the Usability of Electronic Health Records, EHRs should support a process that provides ahigh level of usability for all users. The goals are for users to interact with the system safely, effectively,efficiently, and with an acceptable level of satisfaction. To this end, design to optimize for safety,effectiveness, efficiency and user satisfaction was utilized throughout the design and development cycle.The following key principles of the NISTIR 7742 were followed:-The design is based on the understanding of specific user group needs, workflows and environments.Users are involved throughout the design and development.The design is driven and refined by user-centered evaluation and feedback.The design addressed the whole user experience.The design is adapted with users until performance objectives are met.The process is iterative.Reference:National Institute of Standards and Technology. (2010). NIST Guide to the Processes Approach forImproving the Usability of Electronic Health Records (NISTIR 7742). Gaithersburg, ub id 907313.http://ws680.nist.gov/publication/get pdf.cfm?pub id 907312This study collected performance data on the following categories typically conducted on Medgen EHR.Description§170.315(a)(1)(2)(3) CPOE – Medications – Laboratory – Diagnostic Imaging§170.315(a)(4) Drug-drug, Drug-allergy interaction checks§170.315 (a)(5)Demographics§170.315 (a)(6) Problem List§170 315(a)(7) Medication List§170 315(a)(8) Medication allergy List§170 315(a)(9) Clinical Decision Support§170 315 (a)(14)Implantable Device List§170 315(b)(2) Clinical Information Reconciliation and Incorporation§170.315(b)(3)Electronic PrescribingPage 6

During the sixty-minute one-on-one usability test, each participant was greeted by the administrator andasked to review and sign an informed consent/release form (included in Appendix 3); they were instructedthat they could withdraw at any time.Participants had prior experience with Medgen EHR but no additional training was provided for thepurposes of the usability tests. The administrator introduced the test, and instructed participants tocomplete a series of tasks (given one at a time) using the EHRUT. During the testing, the administratortimed the test, along with the data logger(s) recorded user performance data on paper and electronically.The administrator did not give the participant assistance in way to complete the task. Participant screens,and audio were recorded for subsequent analysis.The following types of data were collected for each participant:Number of tasks successfully completed within the allotted time without assistanceTime to complete the tasksNumber and types of errorsPath deviationsParticipant’s verbalizationsParticipant’s satisfaction ratings of the systemAll participant data was de-identified – no correspondence could be made from the identity of theparticipant to the data collected. Following the conclusion of the testing, participants were asked tocomplete a post-test questionnaire and were compensated 100 for their participation and time. Variousrecommended metrics, in accordance with the examples set forth in the NIST Guide to the ProcessesApproach for Improving the Usability of Electronic Health Records, were used to evaluate the usability ofthe EHRUT. Following is a summary of the performance and rating data collected on the EHRUT.Meaningful UseCriteria andTaskN#§170 315(a)(6)Medication ListRecord thepatientmedication12§170 315(a)(6)Medication ListAccess andchange thepatientmedication12§170 315(a)(7)Medication allergyListRecord patientallergies/medallergies12§170 315(a)(7)Medication allergyListAccess andupdate patientallergies/medallergies12Page DeviationAverageObserved/OptimalTaskTime (inSeconds)Mean(SD)ErrorsMean(SD).Task Efficiency(5 Easy)Mean(SD)

§170 e clinicalinformation12§170.315(a)(1)CPOERecord ology Order§170.315(a)(1)CPOERecord LabOrder12§170.315(a)(1)CPOEAccess/ChangeLab Order12§170 315(a)(8)Clinical DecisionSupportReview/ConfigureClinical DecisionSupport .315(a)(2)Drug-drug, Drugallergy Drug, DrugAllergyinteraction check12§170.315(a)(1)CPOEAccess andChangeMedication Order12§170 g12§170.315(a)(2)Drug-drug, Drugallergy interactionchecksAdjustment ofseverity level ofdrug-drugintervention12Page 81212

In addition to the performance data, the following qualitative observations were made:Major findingsThe study determined that our intended audience finds the system easy to use. User satisfaction ratingwas high and task time fell into acceptable range for the majority of the features. Participants expressedtheir satisfaction of the system, expressly the system’s feature of creating provider specific medicationfavorite lists.Electronic prescribing did fall into the task time allotted but users felt that the task took longer thanwhat the user felt to be efficient for workflow process. Clinical information reconciliation took longer thanoptimal but our findings show that this was due to the fact that the users were not completely familiarwith recent updates to this feature.Participants of the study expressed their frustration of the system for having to the create a new scriptwhen they simply wanted to change the dosage. Participants were able to point out areas of the systemthat they felt were less intuitive than others such as the clinical decision support.Areas for improvementThe study confirmed the need to train users on the use of the clinical information reconciliation feature.The team will consider how to optimize the layout of the medication entry and e-prescribing layout toreduce the average task time in performing these functionalities.2 IntroductionThe EHRUT(s) tested for this study was Medgen EHR Version 7.0. The testing was conducted remotely.The usability testing environment, patient scenario, workflow and tasks attempted to simulate realisticimplementations, workflows and conditions.The purpose of this study was to test and validate the usability of the current user interface, and provideevidence of usability in the EHR under Test (EHRUT). To this end, measures of effectiveness, efficiency,user satisfaction, and time to complete task, were captured during the usability testing.Intended UsersIntended users of the EHRUT are clinicians working in an ambulatory setting, registered nurses ormedical assistants (RNs and MAs) and physicians (or midlevels; MDs and PAs). Users may have varyinglevels of experience in the clinical setting and may have varying years of experience with the EHRUT.They can be expected to have received training on the EHRUT, and to be frequent users (weekly or dailyuse). The usability test focused on discovering issues of skilled users.3 MethodParticipantsA total of 12 participants were tested on the EHRUT(s). Participants in the test were physicians,physician assistant, nurses and medical asssitants. Participants of the test had to be current usersof the system for a minimum of one year andPage 9

currently work with patients on a daily or weekly basis. They were compensated 100 for their time. Inaddition the participant did not participate in a focus group or usability test in the last 12 months nor dothey or anyone in their home work in marketing research, usability research, or web design. Participantshad no direct connection to the development of/or organization producing Medgen EHR. Participants werenot from the testing or supplier organization. Participants were given the opportunity to have the sameorientation and level of training as the actual end users would have received.For the test purposes, end-user characteristics were identified and translated into a recruitment screenerused to solicit potential participants; an example of a screener is provided in Appendix 1.Recruited participants had a mix of backgrounds and demographic characteristics conforming to therecruitment screener. The following is a table of participants by characteristics, including demographics,professional experience, computing experience and user needs for assistive technology. Participant nameswere replaced with Participant IDs so that an individual’s data cannot be tied back to individualidentities. Our participants match the description of the intended user population.Participant Demographic DataPIDAgeGenderYrs. iveTechnologyNoNeedsMA 1Pulmonary872530MAMARNMAMD 4 2 2 3 3PulmonaryPulmonaryFamily MedFamily MedFamily MedNoNoNoNoNoM3PA 1Family MedNoMFFFF42515316DPMMDMAMAMA/OM 3 3 2 2 4PodiatryOBGYNOBGYNOBGYNOBGYNNoNoNoNoNo100% of all participants recruited for the test showed up to participate in the test.Participants were advised that the test would take 60 minutes. Chart can also be found inAppendix 2.Study DesignOverall, the objective of this test was to uncover areas where the application performed well –that is, effectively, efficiently, and with satisfaction – and areas where the application failed tomeet the needs of the participants. The data from this test may serve as a baseline for futuretests with an updated version of the same EHR and/or comparison with other EHRs provided thesame tasks are used. In short, this testing serves as both a means to record or benchmarkcurrent usability, but also to identify areas where improvements must be made. During theusability test, participants interacted with Medgen EHR. Each participant used the system andwas provided with the same instructions.Page 1010