WIXLIB

application that provides a near real-time and chronological view of healthdata contained in the departments’ separate EHR systems. Further, VAhas undertaken various attempts over the past two decades to modernizeVistA, but its efforts have fallen short of completion at a high cost to thedepartment.In June 2017, the then-Secretary of Veterans Affairs announced that thedepartment would redirect its EHR modernization (EHRM) to deploy thesame system that DOD is in the process of deploying.3 The twodepartments’ respective modernization efforts are intended to deliver asingle, seamlessly integrated (i.e., shared) EHR system, which is basedon a Cerner Government Services, Inc. (Cerner) commercial EHRproduct known as Millennium. Established in June 2018, VA’s Office ofElectronic Health Record Modernization (OEHRM) is responsible foroverseeing and directing the department’s EHRM. VA intends for OEHRMto coordinate with its Office of Information Technology and VHAleadership—specifically, VA’s Chief Information Officer and VHA’s UnderSecretary for Health—under the direction of an Executive Director.4 TheExecutive Director reports directly to the VA Deputy Secretary.Although health data are necessary to provide critical services andbenefits to veterans, VA has historically faced challenges in managingthese data.5 Therefore, it is imperative that the department carefully plandata management activities that will provide comprehensive patientinformation and enable consistent, high-quality care in line with VA’smission and vision for EHRM. This includes activities that support thecontinuity of patient care and VHA operations, such as migrating qualitydata to the new system. It also includes supporting the continuity ofreporting, meaning that stakeholders should continue to have access to3DODrefers to its new EHR system as Military Health System (MHS) GENESIS, whichthe department began deploying in February 2017. Within the Department of HomelandSecurity, the U.S. Coast Guard is also deploying this system and began doing so inAugust 2020.4TheOffice of Information Technology and VHA serve respectively as the technical andfunctional leaders for the department’s health care delivery. Together they have worked todevelop and maintain VistA for decades.5Datamanagement refers to a set of disciplines and techniques used to process, store,and organize data. According VA’s Enterprise Data Strategy, top data managementchallenges include identifying and using authoritative data sources, poor data quality,difficulty linking data across isolated sources, variations in reporting, and immature datagovernance. Department of Veterans Affairs, Enterprise Data Strategy: A Vision for theFuture (Washington, D.C.: January 2021).Page 2GAO-22-103718 VA Electronic Health Record Data Management

reporting capabilities that are necessary to carry out their patient care,operations, and research functions.6You asked us to examine VA’s data management plans for its EHRM.Our specific objectives were to (1) describe the department’s plans formigrating data to the new EHR system and determine the extent to whichVA has implemented those plans, (2) describe the department’s plans forEHRM continuity of reporting and determine the extent to which VA hasimplemented those plans, (3) describe the expected use of JLV defined inthe department’s data management plans, and (4) describe how VAcoordinated its EHRM data management activities with DOD.To describe the plans that VA developed to migrate data to the new EHRsystem and the extent to which the department has implemented thoseplans, we obtained and reviewed EHRM program documentation,including plans prepared by OEHRM and Cerner. We also revieweddocumentation relating to the results of the initial system deployment atthe Mann-Grandstaff VA Medical Center in Spokane, Washington, suchas progress reports, test findings, and analyses.Using this information, we compared VA’s data migration efforts to itsplans for the initial system deployment. In addition, we compared thedepartment’s data migration efforts to guidance for ensuring data qualityoutlined in the Office of Management and Budget’s Federal DataStrategy.7 We also compared these efforts to practices for measuring6VA’sreporting capabilities are varied and support a wide range of stakeholders and usesacross the department. Examples of these reporting capabilities include registries thatcollect information over time about patients with certain health conditions, such astraumatic brain injury; reporting and analytics tools that identify patients at risk for suicide,hospitalization, or illness, and measure quality of care; and research programs, such asthe Million Veteran Program, which uses data about patients to understand how genesaffect health and illness.7TheFederal Data Strategy is a framework of operational principles and best practicesthat are intended to enable government to fully leverage data as a strategic asset. Amongother things, the strategy outlines principles that are intended to guide federal datamanagement activities and inform agencies in developing and executing all aspects of thedata lifecycle. Office of Management and Budget, Federal Data Strategy – A Frameworkfor Consistency, Memorandum M-19-18 (Washington, D.C.: June 4, 2019).Page 3GAO-22-103718 VA Electronic Health Record Data Management

performance,8 such as those identified by GAO,9 as well as thoseidentified in the Project Management Institute’s A Guide to the ProjectManagement Body of Knowledge (PMBOK Guide).10 Further, weinterviewed cognizant officials from OEHRM, including the ChiefTechnology and Integration Officer and Chief Medical Officer, as well as8Performancemeasurement is the ongoing monitoring and reporting of programaccomplishments toward pre-established goals, for any activity, project, function, or policythat has an identifiable purpose or set of objectives. Performance measures include thosethat address the results of program products and services. GAO, PerformanceMeasurement and Evaluation: Definitions and Relationships (SupersedesGAO-05-739SP), GAO-11-646SP (Washington, D.C.: May 2, 2011).9TheGovernment Performance and Results Act of 1993, as amended by the GovernmentPerformance and Results Modernization Act of 2010, established a framework for federalgovernment and agency performance plans, performance measurement, reporting, andfederal government outcome-oriented priority goals. These requirements are applicable atthe department or agency level or to certain agency activities described in the law.However, practices regarding performance measures and goals are importantmanagement tools applicable to all levels of an agency, including the program, project, oractivity level, consistent with leading management practices and internal controls relatedto performance monitoring. We have previously reported that performance measurementallows organizations to track progress in achieving their goals and gives managers crucialinformation to identify gaps in program performance and plan any needed improvements.In addition, according to Standards for Internal Control in the Federal Government,managers should design activities to achieve objectives and respond to risks by, forexample, comparing actual performance to planned or expected results and analyzingsignificant differences. The GPRA Modernization Act of 2010, 5 U.S.C. § 306, 31 U.S.C.§§ 1115-1116 & 1120-1124; GAO, VA Medical Centers: VA Should Establish Goals andMeasures to Enable Improved Oversight of Facilities’ Conditions, GAO-19-21(Washington, D.C.: Nov. 13, 2018); Digital Service Programs: Assessing Results andCoordinating with Chief Information Officers Can Improve Delivery of Federal Projects,GAO-16-602 (Washington, D.C.: Aug. 15, 2016); Standards for Internal Control in theFederal Government, GAO-14-704G (Washington, D.C.: Sept. 10, 2014); DHS Training:Improved Documentation, Resource Tracking, and Performance Measurement CouldStrengthen Efforts, GAO-14-688 (Washington, D.C.: Sept. 10, 2014); Managing ForResults: Enhancing Agency Use of Performance Information for Management DecisionMaking, GAO-05-927 (Washington, D.C.: Sept. 9, 2005); and Executive Guide: EffectivelyImplementing the Government Performance and Results Act, GAO/GGD-96-118(Washington, D.C.: June 1996).10ProjectManagement Institute, Inc., A Guide to the Project Management Body ofKnowledge (PMBOK Guide)—Sixth Edition (2017). PMBOK is a trademark of ProjectManagement Institute, Inc. The Project Management Institute is a not-for-profit associationthat, among other things, provides standards for managing various aspects of projects,programs, and portfolios.Page 4GAO-22-103718 VA Electronic Health Record Data Management

system users from the initial deployment site regarding VA’s datamigration plans and implementation.11To describe the plans that VA developed to support the continuity ofreporting for EHRM and the extent to which the department implementedthose plans, we obtained and reviewed EHRM documentation, includingplans, performance work statements, requirements documents, andCerner contract deliverables that described VA’s approach for continuityof reporting. We also reviewed documentation regarding the results of theinitial system deployment, such as progress reports. Using thisinformation, we compared the department’s activities to support continuityof reporting to the activities identified in its plans.We also compared the department’s reporting continuity efforts toguidance for identifying and engaging stakeholders as outlined in theOffice of Management and Budget’s Federal Data Strategy12 and theProject Management Institute’s PMBOK Guide.13 Further, weinterviewed cognizant officials from OEHRM and VHA.To describe the expected use of JLV defined in the department’s datamanagement plans, we reviewed VA EHRM program plans anddocumentation related to data management, system architecture anddesign, and system capabilities. We also reviewed JLV user manuals tounderstand how VA intended for the application to provide clinicians withaccess to their patients’ health data. Further, we interviewedknowledgeable officials, including the OEHRM Chief Technology andIntegration Officer.To describe how VA coordinated its EHRM data management activitieswith DOD, we reviewed documentation prepared by joint VA-DODdecision-making groups for EHRM, such as the Joint Data, Analytics,Reporting, and Registry Board. These documents included charters,memoranda, and business rules for managing joint risks, issues, and11WithinOEHRM, the Technology and Integration Office is responsible for providingtechnical leadership, management, and oversight of IT. The Chief Medical Office isresponsible for overseeing strategy and planning efforts for change management, usertesting and training, and business process re-engineering. It also leads communicationefforts for the end users and system deployment.12Officeof Management and Budget, Memorandum M-19-18.13ProjectPage 5Management Institute, Inc., PMBOK Guide—Sixth Edition.GAO-22-103718 VA Electronic Health Record Data Management

opportunities. In addition, we interviewed knowledgeable officials fromOEHRM and the joint decision-making groups.We assessed the reliability of computer-processed data from EHRM testfindings, risk and issues registers, and a trouble ticket extract, as well asdata migration and reporting capability progress reports. To do so, wereviewed related documentation and interviewed knowledgeable officialsabout the quality control procedures used to assure accuracy andcompleteness of the data. For reporting capability progress reports, wealso examined the data for obvious outliers, incomplete entries, orunusual entries. We determined that the data used to support the findingsin this report were sufficiently reliable for the purposes of our reportingobjectives. See appendix I for a more detailed discussion of ourobjectives, scope, and methodology.We conducted this performance audit from August 2019 to February 2022in accordance with generally accepted government auditing standards.Those standards require that we plan and perform the audit to obtainsufficient, appropriate evidence to provide a reasonable basis for ourfindings and conclusions based on our audit objectives. We believe thatthe evidence obtained provides a reasonable basis for our findings andconclusions based on our audit objectives.BackgroundVA’s EHRM aims to improve health care delivery for veterans andfacilitate the seamless sharing of health data between VA and DOD asservicemembers transition to veteran status. According to VA’s thenSecretary, implementing a single shared EHR system is intended toenable seamless care between VA and DOD without the exchange andreconciliation of data between two separate systems. As such, the new,shared system is to include records of veterans’ health care at VA sites inaddition to care received during their military service at DOD sites.14Figure 1 depicts how military service and veteran health data areexpected to be available in the shared EHR system that the departmentsare deploying as part of their respective modernization efforts.14VAplans to deploy the new EHR system to sites across its health care system, whichincludes VA medical centers, outpatient clinics, community living centers, and VA VetCenters. DOD’s deployment sites include military hospitals and clinics, known as medicaltreatment facilities.Page 6GAO-22-103718 VA Electronic Health Record Data Management

Figure 1: Military Service and Veteran Health Data in the Department of Veterans Affairs (VA)/Department of Defense (DOD)Shared Electronic Health Record SystemThe departments agreed to coordinate decision-making and oversight forthe new, shared EHR system. In December 2019, they re-chartered theInteragency Program Office as the Federal Electronic Health RecordModernization (FEHRM) Program Office to serve as a single point ofaccountability for the shared system, including the execution of jointfunctions and operations.Page 7GAO-22-103718 VA Electronic Health Record Data Management

VA Has Revised ItsDeployment Schedule forthe New EHR SystemAs part of VA’s EHRM, the department contracted with Cerner in May2018 to provide, host, and deploy its new EHR system.15 VA’s contractincluded Cerner’s commercial EHR product, Millennium, as well asHealtheIntent, a cloud-based software product that aggregates healthdata from multiple sources.16VA’s schedule and approach for deploying the new EHR system hasundergone multiple changes since the department first announced itsdeployment plan in October 2018: Original plan. VA planned to deploy the new system at sites instages based on their geographical location and over a 10-yearperiod. According to the department’s initial schedule, VA expected tobegin deploying the new system in March and April 2020 at siteswithin VHA’s Veterans Integration Service Network 20 (VISN 20).17These initial deployment sites were the Mann-Grandstaff VA MedicalCenter in Spokane, Washington, and the VA Puget Sound HealthCare System in Seattle, Washington.18 Revised initial deployment plan. In August 2019, VA revised itsplans for initial system deployment to include two phases, known ascapability sets 1.0 and 2.0. Capability set 1.0 was to deploy in March2020 at the Mann-Grandstaff VA Medical Center and capability set 2.0was to deploy in November 2020 at the Puget Sound Health CareSystem. 15HostCapability set 1.0 includes key EHR functionalities necessary toimplement the system at a less complex facility.refers to providing a data center for the new EHR system.16Cloudcomputing enables software and data to reside on vast numbers of serversconnected over the internet, rather than on servers at the physical location of anindividual, company, or other organization.17VHAis divided into areas called Veterans Integration Service Networks (VISNs). Thereare currently 18 VISNs throughout VHA based on geographic location. VISNs provideoversight and guidance to the VA Medical Centers and VA Health Care Systems withintheir area and are sometimes called a “network.” VISN 20 includes medical centers andcommunity-based outpatient clinics in the states of Alaska, Washington, Oregon, most ofthe state of Idaho, and one county each in Montana and California.18ThePuget Sound Health Care System includes two divisions, Seattle and AmericanLake. VA manages the two divisions in an integrated manner.Page 8GAO-22-103718 VA Electronic Health Record Data Management

Capability set 2.0 includes capability set 1.0 functionalities andremaining functionalities necessary to implement the system at ahighly complex facility. Initial system deployment delays. In February 2020, VA postponedthe initial system deployment at the Mann-Grandstaff VA MedicalCenter until July 2020 to establish a more complete trainingenvironment and build interfaces between the new EHR system andother VA systems. In March 2020, VA further postponed the initialsystem deployment to focus on the department’s response toCoronavirus Disease 2019. Revised timeline and new functionality. In August 2020, VAannounced a revised timeline for deploying the new EHR system. Forexample, the department revised the date for its initial deployment atthe Mann-Grandstaff VA Medical Center to October 24, 2020. Thisinitial deployment included capability set 1.0 and some additionalveteran-facing functionality (e.g., consolidated mail order pharmacyand online prescription refill), known as capability set 1.1.Following the October 2020 initial system deployment at the MannGrandstaff VA Medical Center, VA paused EHRM implementation toconduct a strategic review of EHRM between March and June 2021.19After the strategic review, in November 2021, the department issued aprogress report that described its revised EHRM deployment schedulethrough fiscal year 2024. According to the report, VA planned to restartEHRM deployment in March 2022, beginning with the Chalmers P. WylieVA Ambulatory Care Center in Columbus, Ohio.20 In addition, the reportstated that VA continues to develop its long-term EHRM deploymentschedule. In January 2022, VA announced that it would delay theColumbus, Ohio deployment from March 2022 to April 2022 due to asurge in Coronavirus Disease 2019 cases affecting the workforce andcommunity.19VAsummarized the results of its strategic review in the Electronic Health RecordComprehensive Lessons Learned report. The report described eight challenge areas forEHRM, as well as plans and progress toward addressing those challenges. In addition,the report proposed a way forward and performance measures that the departmentexpected to ensure the success of EHRM while preventing and reducing issues at futuredeployment sites. We have work under way to assess the results of VA’s strategic review.Department of Veterans Affairs, Electronic Health Record Comprehensive LessonsLearned (Washington, D.C.: July 2021).20Departmentof Veterans Affairs, Electronic Health Record Comprehensive LessonsLearned Progress Update, (Washington, D.C.: November 2021).Page 9GAO-22-103718 VA Electronic Health Record Data Management

VA Has Planned toManage Health DataUsing Cerner ProductsVA relies on data about veterans and their health care to carry out itsmission. In particular, VA clinicians have relied on data stored in VistA toprovide health care services. The department has also relied on healthdata to support reporting capabilities for patient care, operations, andresearch functions. To support these reporting capabilities, historically VAhas transferred health data from VistA into its Corporate DataWarehouse, which resides in VA’s National Data Center.21VA’s EHRM is expected to replace most components of the department’slegacy EHR system, including the management of health data. Accordingto the department’s EHRM Architecture and Design Plan, VA expects tomanage health data using the two key Cerner products—Millennium EHRand HealtheIntent, as explained below:22 Cerner Millennium is expected to provide users, such as clinicians,with access to their patients’ health data. Millennium is also expectedto support clinical capabilities, such as ordering laboratory tests andmedications, documenting notes about the delivery of care, andscheduling appointments. Cerner HealtheIntent is expected to aggregate health data frommultiple sources, including VistA and Millennium, to create a singlerecord for each patient (known as the longitudinal record). Further,data in the longitudinal record are e

coordinate with stakeholders to define objectives and measures of success for EHRM, including a set of key performance indicators related to veteran experience, workforce support, quality and safety, and health care operations. However, VA EHRM key performance indicators did not incl