Audit Of The SEC’s Management Of Its Data Centers

Transcription

U.S. SECURITIES AND EXCHANGE COMMISSIONOFFICE OF INSPECTOR GENERALAudit of the SEC’s Management of Its Data CentersSeptember 29, 2017REPORT NO. XXXiiREDACTED FOR PUBLIC RELEASEDECEMBER XX, 2015Report No. 543

REDACTED FOR PUBLIC RELEASEUNITED STATESSECURITIES AND EXCHANGE COMMISSIONWASHINGTON, D.C. 20549OFFICE OFINSPECTOR GENERALMEMORANDUMSeptember 29, 2017TO:Kenneth Johnson, Acting Chief Operating OfficerFROM:Carl W. Hoecker, Inspector GeneralSUBJECT:Audit of the SEC’s Management of Its Data Centers, Report No. 543Attached is the Office of Inspector General (OIG) final report detailing the results of our audit ofthe U.S. Securities and Exchange Commission’s (SEC or agency) management of its datacenters. The report contains ten recommendations that should help the agency develop a planfor future data center relocations and improve the SEC’s data center contract management.On September 13, 2017, we provided management with a draft of our report for review andcomment. In its September 25, 2017, response, management concurred with ourrecommendations. We have included management’s response as Appendix IV in the finalreport.Within the next 45 days, please provide the OIG with a written corrective action plan thataddresses the recommendations. The corrective action plan should include information suchas the responsible official/point of contact, timeframe for completing required actions, andmilestones identifying how the SEC will address the recommendations.We appreciate the courtesies and cooperation extended to us during the audit. If you havequestions, please contact me or Rebecca L. Sharek, Deputy Inspector General for Audits,Evaluations, and Special Projects.Attachmentcc:Jay Clayton, ChairmanLucas Moskowitz, Chief of Staff, Office of Chairman ClaytonSean Memon, Deputy Chief of Staff, Office of Chairman ClaytonPeter Uhlmann, Managing Executive, Office of Chairman ClaytonMichael S. Piwowar, CommissionerRichard Grant, Counsel, Office of Commissioner PiwowarKara M. Stein, CommissionerRobert Peak, Advisor to the Commissioner, Office of Commissioner SteinRobert B. Stebbins, General CounselRick A. Fleming, Investor AdvocateJohn J. Nester, Director, Office of Public AffairsBryan Wood, Director, Office of Legislative and Intergovernmental AffairsREDACTED FOR PUBLIC RELEASE

REDACTED FOR PUBLIC RELEASEMr. JohnsonSeptember 29, 2017Page 2Vance Cathell, Director, Office of AcquisitionsPamela C. Dyson, Director/Chief Information Officer, Office of Information TechnologyDarlene L. Pryor, Management and Program Analyst, Office of the Chief OperatingOfficerREDACTED FOR PUBLIC RELEASE

REDACTED FOR PUBLIC RELEASEU.S. SECURITIES AND EXCHANGE COMMISSIONExecutive SummaryOFFICE OF INSPECTOR GENERALAudit of the SEC’s Management of ItsData CentersReport No. 543September 29, 2017Why We Did This AuditWhat We FoundThe U.S. Securities and ExchangeCommission’s (SEC or agency) data centershouse critical telecommunications, data, andcomputing resources, including the agency’s(b)(7)(E)andEDGAR—the Electronic Data Gathering,Analysis, and Retrieval system—whichsupports the financial reporting of publiccompanies in the United States. Between2012 and 2013, the SEC completed actionsto relocate its data centers to their presentlocations. The agency awarded new datacenter contracts to (b)(7)(E) (hereinafter D1)and (b)(7)(E)(hereinafter D2) toprovide data center services. The SEC’scontracts with D1 and D2 total about 16 million and 18 million, respectively, ifall contract options are exercised. Weconducted this audit to assess the SEC’smanagement of its data centers, ensure thedata centers have adequate physical andenvironmental controls, and determinewhether SEC personnel properly monitoredthe contractors’ performance.In 2008, the SEC paid 162,000 for a contractor-developed plan to relocate theagency’s data centers. However, the SEC did not follow the plan’srecommended steps or timeline to ensure the 2012-2013 data centerrelocations were properly executed and that the SEC’s data center providers,D1 and D2, could meet the agency’s needs before awarding contracts andmigrating data, thereby exposing SEC data to vulnerabilities. We were unableto determine why the SEC did not follow the recommended data centerrelocation steps or timeline because the current officials responsible for theSEC’s data centers were not aware of the relocation plan, many key officialsresponsible for the data center relocations no longer work at the SEC, and, asdiscussed further below, contract files were incomplete. However, because theagency derived little, if any, benefit from the 2008 data center relocation plan,we believe the 162,000 paid for the plan represents funds that the SEC mayhave wasted. Furthermore, we determined that SEC data and equipment atthe D1 data center have been exposed to certain physical and environmentalcontrol vulnerabilities since the inception of the contract. These vulnerabilitieshave disrupted SEC operations and resulted in increased costs to the agency.Specifically, we estimate that since 2014 the SEC spent about 370,000 inquestioned costs to mitigate the physical and environmental vulnerabilities atthe D1 data center. Finally, based on our observations, we question whetherthe D1 data center meets a key contract requirement—to be a Tier III datacenter or greater—as defined in Telecommunications Industry Associationstandards.What We RecommendedWe made ten recommendations forcorrective action, including that the SECconduct comprehensive reviews of the2012-2013 data center relocations to identifylessons learned. We have previouslyreported that agency staff did not alwaysperform contract management dutiesconsistently and as required. Therefore, inaddition to our recommendations regardingdata center-related contract management,we strongly encourage the Director of theOffice of Acquisitions to conduct acomprehensive review of the SEC’s CORprogram and ensure controls are developedor strengthened to improve the SEC’scontract management activities.Management concurred with therecommendations, which will be closed uponcompletion and verification of correctiveaction. This report contains non-publicinformation about the SEC’s informationsecurity program. We redacted (deleted) thenon-public information to create this publicversion.Additionally, we determined that the SEC did not adequately manage ormonitor its data center contracts. We found that Contracting Officer’sRepresentatives (CORs) did not always validate invoices or maintain completefiles. COR contract files were missing required deliverables, justifications andsupport for critical decisions related to the data centers, and monthly reports.Further, D1’s monthly power consumption reports were unusable and the SECdid not timely or adequately address known vulnerabilities at the D1 datacenter, or effectively assess physical and environmental controls at either datacenter. For example, the agency’s 2016 and 2017 data center assessmentsidentified no findings at either location, despite (b)(7)(E)vulnerabilities at the D1 data center and a report from a contractor wehired that identified 14 physical and environmental control deficiencies at theD2 data center.Because of inadequate contract management, the SEC paid D2 invoicescontaining formula errors resulting in 217,159 in overpayments (which hasbeen refunded). We also identified about 2.8 million in unsupported costspaid to D1. If the SEC does not take corrective action to validate certain costsand if all contract options are exercised, the agency will incur additional costs ofabout 2.7 million in funds that could put to better use over the remaining life ofD1’s contract.For additional information, contact the Office of Inspector General at(202) 551-6061 or https://www.sec.gov/oig.iREDACTED FOR PUBLIC RELEASE

REDACTED FOR PUBLIC RELEASEU.S. SECURITIES AND EXCHANGE COMMISSIONOFFICE OF INSPECTOR GENERALTABLE OF CONTENTSExecutive Summary . iBackground and Objectives .1Background . 1Objectives . 3Results .4Finding 1: The SEC Did Not Adequately Assess Contractors’ Capabilities orAllocate Sufficient Time to Relocate Its Data Centers, Exposing SEC Data toVulnerabilities . 4Recommendations, Management’s Response, and Evaluation of Management’sResponse . 10Finding 2: The SEC Did Not Adequately Manage or Monitor Its Data CenterContracts . 12Recommendations, Management’s Response, and Evaluation of Management’sResponse . 21Tables and FiguresTable 1.Table 2.Table 3.Table 4.Table 5.Table 6.Data Center Tier Levels . 3Summary of Milligan’s D2 Data Center P&E Control Assessment . 30Funds That May Have Been Wasted . 34Questioned Costs . .34Unsupported Costs . 34Funds That Could Be Put to Better Use . 35Figure 1. Recommended Data Center Relocation Best Practices and Timeframes. 7Figure 2. Actual D1 Data Center Relocation Activities and Timeframes . 7Other Matters of Interest.26AppendicesAppendix I. Scope and Methodology . 27Appendix II. Results of D2 Data Center P&E Control Assessment. 30Appendix III. Calculations of Monetary Impacts . 34Appendix IV. Management Comments . 36REPORT NO. 543iiREDACTED FOR PUBLIC RELEASESEPTEMBER 29, 2017

REDACTED FOR PUBLIC RELEASEU.S. SECURITIES AND EXCHANGE COMMISSIONOFFICE OF INSPECTOR GENERALABBREVIATIONSCOContracting OfficerCORContracting Officer’s RepresentativeEDGARElectronic Data Gathering, Analysis, and Retrieval System(b)(7)(E)FARFederal Acquisition RegulationGAOU.S. Government Accountability Office(b)(7)(E)MilliganMilligan and Company, LLC/Samlin ConsultingNISTNational Institute of Standards and TechnologyOAOffice of AcquisitionsOIGOffice of Inspector GeneralOITOffice of Information TechnologyP&Ephysical and environmentalPDUpower distribution unitPOA&Mplan of action and milestonesSEC or agencyU.S. Securities and Exchange CommissionSECRSEC Administrative RegulationSPSpecial PublicationTIATelecommunications Industry AssociationUPSuninterruptable power supplyREPORT NO. 543iiiREDACTED FOR PUBLIC RELEASESEPTEMBER 29, 2017

REDACTED FOR PUBLIC RELEASEU.S. SECURITIES AND EXCHANGE COMMISSIONOFFICE OF INSPECTOR GENERALBackground and ObjectivesBackgroundA data center houses and protects computers and communications equipment thatstore and process data necessary to support business operations. To carry out itsmission, the U.S. Securities and Exchange Commission (SEC or agency) has two datacenters located in commercial facilities. The data centers house the SEC’s criticaltelecommunications, data, and computing resources, including EDGAR—the agency’sElectronic Data Gathering, Analysis, and Retrieval system—which supports the financialreporting of public companies in the United States. The SEC relies on two data center(hereinafter D2).1contractors: (1) (b)(7)(E) (hereinafter D1) and (2) (b)(7)(E)Inside the D1 center data, the SEC maintains a secure cage (that is, a fenced-in areaseparated from other data center customers within a shared space) that houses racks ofSEC equipment. The D1 data center (b)(7)(E)Inside the D2 data center, the SECmaintains modules (that is, secure pods with their own walls, physical securityprotocols, cooling, and power infrastructure) that house racks of SEC equipment.The SEC’s contracts with D1 and D2 total about 16 million and 18 million,respectively, if all contract options are exercised. The agency’s Office of InformationTechnology (OIT) and its Office of Acquisitions (OA) are responsible for overseeing theSEC’s data center operations and monitoring the agency’s data center contracts.According to the National Institute of Standards and Technology (NIST)2 SpecialPublication (SP) 800-12, An Introduction to Information Security, (NIST SP 800-12)3physical and environmental (P&E) controls protect systems, buildings, and relatedsupporting infrastructure against threats associated with their physical environment.1The SEC awarded the D1 data center contract (contract number (b)(7)(E)) on (b)(7)(E). It is a firm-fixed price 1-year contract with nine option years and was last renewed on (b)(7)(E). The SEC awarded the D2 data center contract (contract number (b)(7)(E)) on(b)(7)(E). It is also a firm-fixed price 1-year contract with nine option years and was lastrenewed on (b)(7)(E). Neither contract includes a lease. Rather, D1 and D2 provide servicesonly.2NIST

Table 1. Data Center Tier Levels . 3 . Table 2. Summary of Milligan’s D2 Data Center P&E Control Assessment .30 . Table 3. Funds That May Have Been Wasted . 34 . File Size: 1MBPage Count: 46