Transcription
Data Quality PlaybookThe Leveraging Data as a Strategic Asset Working Group developed and issued this Playbook in collaborationwith federal agencies to assist with developing Data Quality Plans in support of the DATA Act implementation.This playbook is a result of extensive collaboration between agencies, OMB, and the Department of theTreasury. The final document and subsequent versions will be posted to the CFOC website. The use of theplaybook format is a practice to gather, define, and illustrate concepts in federal financial management. TheDATA Quality playbook joins a library of management resources developed for agency use to include the ERMPlaybook and the Program Integrity; Anti-fraud Playbook. As with these playbooks, nothing in this Playbookshould be considered prescriptive. All examples provided should be modified to fit the circumstances,conditions, and structure of each agency (or other government organization).The Leveraging Data as a Strategic Asset Working Group extends its gratitude to our colleagues within thefinancial management, procurement, and financial assistance communities for their contributions. ThisPlaybook is intended for management; it is not audit guidance.November 30, 2018
Data Quality PlaybookExecutive SummarySince 1981, the Office of Management and Budget (OMB) Circular No. A-123, Management’sResponsibility for Internal Control, has been at the center of federal requirements to improvefederal programs and operations. Even before the Federal Managers’ Financial Integrity Act andthe Government Accountability Office Standards for Internal Control in the FederalGovernment, one of A-123’s first requirements was to develop internal control plans across theFederal Government. In 2004, OMB and the Chief Financial Officer (CFO) Council workedtogether to develop internal control plans and implementation guidance to improve internalcontrol over financial reporting (ICOFR). As a result, most CFO Act agencies have strengthenedtheir internal controls over financial reporting In the spirit of executing the President’sManagement Agenda Cross Agency Priority goals to Leveraging Data as a Strategic AssetWorking Group, this working group aims to support the integration of Digital Accountability andTransparency Act of 2014 (DATA Act) with existing Enterprise Risk Management processes andthrough improving reporting controls and data quality by providing agencies with this DataQuality Playbook (The Playbook).The cover memo of (M-18-16) of Appendix A of OMB Circular A-123, Management ofReporting and Data Integrity Risk requires DATA Act reporting agencies to implement a dataquality plan (DQP) effective fiscal year (FY) 2019 through FY 2021 at a minimum. ThisPlaybook represents a compilation of insights gained by multiple federal agencies, each withunique perspectives and lessons learned as DATA Act reporting agencies. Agencies may findthese scenarios useful to consider when developing a DQP.This Playbook is a collection of examples and use-cases agencies have proposed based onagency-specific risk assessments at a given time. This Playbook and accompanying appendicesare designed to provide practical information and helpful scenarios for agencies. It is not an allinclusive catalog of the only acceptable options for developing a DQP and does not address allpotential situations an agency may face with respect to DATA Act implementation. Rather, thePlaybook provides a compendium of potential approaches, solutions, and explanations thatprovide context aligned with policy and business operations to focus the conversation on theutility of the data submitted in accordance with the DATA Act rather than a compliance-focusedexercise. Each agency will need to develop a DQP that is tailored to their unique operatingenvironments challenges and risks to achieving missions. Our goal is to support agencies inmaking their own determinations about how best to meet the DATA Act reporting requirementsand improve the quality of spending data submitted to USAspending.gov.Nothing in this Playbook supersedes or replaces existing regulations, policies, guidance, or thesubmission requirements outlined in the DATA Act Information Model Schema. The Playbookmay be updated periodically, if helpful to federal agencies, as together we strive to accomplishthe underlying objectives of the DATA Act.ii
Data Quality PlaybookTable of ContentsIntroduction . 1A.Applicability . 1B.Accountability for Quality Reporting . 1C.Data Quality Actions . 2Data Quality Plan Illustrative Scenarios . 6Illustrative Scenario 1: Agency XYZ . 7Illustrative Scenario 2: Agency MBT . 19Illustrative Scenario 3: Agency ABC . 23Illustrative Scenario 4: Department of JFK . 28Illustrative Scenario 5: Department of WNB. 31Illustrative Scenario 6: Department of NCR . 45Exceptions Representing Valid Business Processes and Statutory Allowances . 49Award-Specific Considerations . 59Certification Illustrative Examples . 61Glossary of Acronyms . 68Acknowledgments and Considerations . 70iii
Data Quality PlaybookIntroductionA.ApplicabilityThe Data Quality Playbook, or “Playbook,” is intended to assist Senior Accountable Officials(SAOs) with developing Data Quality Plans (DQPs) to achieve reasonable assurance over internalcontrols and processes that support overall data quality for the input and validation of agency datasubmitted to USAspending.gov. The cover memo of Appendix A of Office of Management andBudget (OMB) Circular A-123, Management of Reporting and Data Integrity Risk (Appendix A),is the authoritative source on the Digital Accountability and Transparency Act of 2014 (DATAAct) DQP, and requires DATA Act reporting agencies to implement a DQP. Consideration of thisplan must be included in agencies' existing annual assurance statement over internal controls overreporting (ICOR) beginning in fiscal year (FY) 2019 and continuing through the statementcovering FY 2021 at a minimum, or until agencies determine that they can provide reasonableassurances over the data quality controls that support achievement of the reporting objectives inaccordance with the DATA Act.This Playbook is intended to clarify concepts that can be overly complex and operationalize how toaddress such complexity while balancing burden and costs. The Playbook uses informationgathered in previous DATA Act assessments. It should be understood that this Playbook documenthas no authoritative standing. It is not associated with the OMB Circular A-123 issuance andshould not be considered authoritative guidance pertaining to Circular A-123. This Playbook waswritten by federal agencies to assist preparers by providing scenarios and illustrative examples toachieve the reporting objectives of the DATA Act. Therefore, the Playbook can and should bemodified to fit the particular circumstances, conditions, strategic priorities, and structure of eachagency.B.Accountability for Quality ReportingFederal agencies recognize the importance of confirming that data related to the spending andstewardship of taxpayer money is collected in an efficient and standardized manner by theagencies receiving the appropriations, assessed for overall quality, and published for maximumtransparency and public ease of use. Spending data that is consolidated in an interoperable andconsistent format not only provides visibility to taxpayers, but also enables federal leaders to makeinformed decisions for mission accomplishment and positive performance outcomes. Since theimplementation of the Federal Funding Accountability and Transparency Act (FFATA) in 2006,there have been meaningful and significant improvements towards transparency in federalspending data.The passage of the DATA Act in 2014 and the focus on open data transparency has steeredgovernance bodies, agencies, and other stakeholders toward the common goal of producingquality, published spending data. In accordance with existing DATA Act guidance issued by OMBand the Department of the Treasury (Treasury), agencies are required to report spending data forpublication on USAspending.gov on a recurring schedule based on the submission process outlinedin the DATA Act Information Model Schema (DAIMS). The financial attributes must be generated1
Data Quality Playbookby the agencies’ financial systems of record, the award attributes required by FFATA from theauthoritative sources, and the linkage of these using the award identification (ID) carried in thefinancial system.C.Data Quality ActionsAppendix A to OMB Circular A-123 states agencies’ responsibility for the management ofreporting and data integrity risk. The DATA Act is not the first or only transparency and reportingrequirement for federal agencies’ spending data, as shown in the table below. Memorandum 18-16also states:“Quarterly certifications of data submitted by agency SAO should be based on the consideration of the dataquality plan and the internal controls documented in their plan as well as other existing controls that may be inplace, in the annual assurance statement process.”Per OMB Memorandum M-17-04, the quarterly SAO assurances over the DATA Act submissionare to (1) assure that the alignment among Files A-F is valid and reliable, and (2) the DATA Actfiles submitted for display on USASpending.gov are valid and reliable. M-17-04, AdditionalGuidance for DATA Act Implementation: Further Requirements for Reporting and Assuring DataReliability further provides that existing data quality measures required by regulation and/or OMBguidance will be sufficient for SAO reliance on individual data files. The table below identifiesmany preexisting validations, to support quarterly DATA Act certifications on internal controlsover their DATA Act submissions.Assurances for Each DATA Act File SubmittedDATA Act FileAuthoritativeSourceFile A:AppropriationsAccountThe Report onBudget Executionand BudgetaryResources (SF-133)derived fromGovernment-wideTreasury AccountSymbol AdjustedTrial Balance System(GTAS) dataExisting Assurances or Internal Controlsover Authoritative Source Data(Pre-DATA Act)Pursuant to A-136, the reporting agencymust reconcile 3rd Quarter and year-end SF133 data to their Statement of BudgetaryResources (SBR).Year-end data are audited financial data,which are subject to managementassurances of internal controls overreporting under A-123.Pursuant to the OMB Circular No. A-11 (A11) Section 130.2, for Executive branchagencies, agencies must report all TreasuryAppropriation Fund Symbols (TAFSs) ineach GTAS reporting window.2DATA Act Senior Accountable OfficialAssurance RequiredThe reporting objective is that the data reportedin File A match the authoritative source (i.e.,SF-133) and that all Treasury Account Symbols(TAS) are reported.To increase the likelihood that this objective willbe met the agency SAO will provide assurancethat data integrity processes and controls are inplace and align with A-123. These assurancesmust be made on a quarterly basis and shouldleverage the existing processes and otherassurances listed in the column “ExistingAssurances or Internal Controls overAuthoritative Source Data.”
Data Quality PlaybookDATA Act FileAuthoritativeSourceFile B: Object Classand ProgramActivityExisting Assurances or Internal Controlsover Authoritative Source Data(Pre-DATA Act)Pursuant to A-136, the reporting agencymust reconcile 3rd Quarter and year-end SF133 to their SBR.The SF-133 derivedfrom GTAS dataYear-end data are audited financial data,which are subject to managementassurances of internal controls overreporting under A-123.The Program Activity names and codes areembedded in the President’s Budget. On aquarterly basis, agencies work with OMB tosend updates to Program Activity throughthe Budget Data Request (BDR) process inalignment with the President’s Budget.OMB then passes the Program Activityvalidation list to Treasury as the validationlist for the Broker.DATA Act Senior Accountable OfficialAssurance RequiredThe reporting objective is that the total amountreported in File B matches the authoritativesource (i.e., SF-133) and that Program Activityand OC Codes are reported based on thePresident’s Budget as executed1 and A-11respectively.To increase the likelihood that this objective willbe met the agency SAO will provide assurancethat data integrity processes and controls are inplace and align with A-123. These assurancesmust be made on a quarterly basis and shouldleverage the existing processes and otherassurances listed in the column “ExistingAssurances or Internal Controls overAuthoritative Source Data.”The DATA Act Broker will validate againstthe list of Object Class (OC) Codes in A-11.File C: AwardFinancialFinancial SystemsFile D1:Procurement AwardAttributesFederal ProcurementData System – NextGeneration (FPDSNG)Year-end data are audited financial data. Allfinancial data, including the obligationsreported in this file, are subject tomanagement assurances under A-123.Pursuant to the Federal AcquisitionRegulation (FAR) and OMB memorandabeginning in 20092, agencies are required tosubmit an annual FPDS-NG DataVerification and Validation (V&V) Reportto OMB and the General ServicesAdministration (GSA). The report includesassurances over the timeliness andcompleteness of the data and sampling ofthe core DATA Act required data elements,comparing contract files to FPDS-NG.1The reporting objective is that data reported inFile C match the authoritative source (i.e.,agency financial systems).To increase the likelihood that this objective willbe met the agency SAO shall provide assurancethat data integrity processes and controls are inplace and align with A-123. These assurancesmust be made on a quarterly basis and shouldleverage the existing processes and otherassurances listed in the column “ExistingAssurances or Internal Controls overAuthoritative Source Data.”The reporting objective is that for data reportedpursuant to FFATA (P.L. 109-282) as amendedby the DATA Act (P.L. 113-101, they aresourced from and match FPDS-NG at the timeof quarterly reporting.To increase the likelihood that this objective willbe met, the agency SAO will rely on internalcontrols (A-123) based on FAR required V&Vfor the assurance over Federal procurementawards.A-11 Section 82.5 (a) contains information about how program activities should be created, and provides that agencies obtain OMB approval forany changes in activity structure. If for any reason, such as funding changes made during the appropriations process, an agency needs to change theirprogram activities in the current fiscal year; such changes should be coordinated through their OMB Resource Management Office. In the future,corresponding validations for program activity in the DATA Act Broker may be implemented.2Relevant OMB memoranda can be found at: ederal-procurement-policy/3
Data Quality PlaybookDATA Act FileAuthoritativeSourceFile D2: FinancialAssistance AwardAttributesAgency ManagementSystems/Files (foraward description,award title, etc.)Financial AssistanceAwardee data inSystem for AwardManagement (SAM)at the time of award(for prime financialassistance awardeeinformation)File E: HighlyCompensatedOfficer DataSAM4Existing Assurances or Internal Controlsover Authoritative Source Data(Pre-DATA Act)No current certification or assuranceprocess is performed by agencies. OMB isreviewing opportunities to enhanceassurances over this data.Agencies may also leverage the datavalidations included in the FinancialAssistance Broker Submission (FABS) as acontrol that their data follows the standardformat.For entities required to register in SAM orotherwise registered, a validation of keydata is performed to assure for the purposesof SAM data quality that the entityregistration data are accurate. Agencies mayleverage this existing validation as a controlover the data submitted to SAM.Contracts and financial assistance awards3contain terms and conditions requiringreporting of executive compensation.For procurement OMB guidance of May2011, requires agency Senior ProcurementExecutives (SPE) to certify that agencypolicies, procedures, and internal controlsinclude reviews of contractor data to assesscompliance and completeness. Contractualremedies exist to address any gaps.File F: SubawardAttributesFFATA SubawardReporting System(FSRS)Contracts and financial assistance awards14contain terms and conditions requiringreporting of executive compensation.For procurement, OMB guidance of May2011 requires agency SPEs to certify thatagency policies, procedures, and internalcontrols include reviews of contractor datato assess compliance and completeness.Contractual remedies exist to address anygaps.3DATA Act Senior Accountable OfficialAssurance RequiredThe reporting objective is that data reported inFile D2 match the authoritative source (i.e.,agency award-management systems) for awardlevel data and the authoritative source (i.e.,SAM) at the time of the award for primeawardee information.To increase the likelihood that this objective willbe met the agency SAO will provide assurancethat data integrity processes and controls are inplace and align with A-123. These assurancesmust be made on a quarterly basis and shouldleverage the existing processes and otherassurances listed in the column “ExistingAssurances or Internal Controls overAuthoritative Source Data.”Agencies will be able to leverage assurancesbased on the internal controls of the systemowner, the GSA in accordance with A-123. Inaddition, for procurement-related awards,agencies will leverage the existing OMBguidance on subaward data quality.Agencies will be able to leverage assurancesbased on the internal controls of the systemowner, GSA, in accordance with A-123. OMB isreviewing mechanisms to further enhanceassurances over these data for financialassistance awards. In addition, for procurementrelated awards, agencies may leverage theexisting OMB guidance on subaward dataquality.At the time this Playbook was issued, 2 CFR 170 only required agencies to include this term and condition on grants and cooperative agreements.In the future, this requirement may be expanded to include loans and other forms of financial assistance.4Not all award recipients are required to register in SAM. Controls and assurances associated with SAM data are not expected to be applied torecipients who are exempt from SAM registration or who are not required to register.4
Data Quality PlaybookRegarding File E and File F, 48 Code of Federal Regulations (CFR) states “an agency is notrequired to address data for which the agency would not normally have supporting information.”Appendix A of Circular A-123 enhances the M-17-04 statement that agencies must comply withcurrent regulatory requirements, by adding:“Consistent with terms and conditions of Federal awards, entities receiving federal awards are required by 2C.F.R. Part 25 and the Federal Acquisition Regulation (FAR) to submit accurate data to the System forAward Management (SAM) and the FFATA Subaward Reporting System (FSRS) maintained by the GeneralServices Administration (GSA). The quality of this data is the legal responsibility of the recipient. GSAprovides an assurance statement that the systems are maintained appropriately and can therefore be used forpublic reporting. Agencies are responsible for assuring controls are in place to verify current registration inSAM at the time of the financial assistance award. Pursuant to 2 C.F.R. Part 200.513, agencies areresponsible for resolving audit findings which may indicate if recipients are not complying with theirrequirements to register or report Subawards. Agencies are not responsible for certifying the quality of datareported by awardees to GSA and made available on USAspending.gov.”5
Data Quality PlaybookScenariosData Quality Plan Illustrative ScenariosThe following pages contain representative agency approaches to inform the development ofDQPs by DATA Act reporting agencies using the Playbook. These are intended to assist federalagencies in meeting the requirements of Appendix A of OMB Circular A-123, as well as theagency SAO certification requirements under OMB M-17-04. Each approach was developed bycontributors representing agencies of various sizes, missions, and operating environments. Thescenarios are designed to increase the value of this data in making management decisions andsupport transparency objectives.Readers of this Playbook may consider these scenarios as potential approaches for developingDQPs. DQPs will be unique to each agency, so the duplication in some scenarios shown here andunique solutions in others is representative of this variety. There may be components of each thatare more useful depending on the reporting agency. The strategic priorities and mission of eachagency is unique and these scenarios should not be taken as static options or requirements.Readers may also note that some methodologies and themes repeat in more than one scenario,while some definitions and approaches vary across the scenarios. These variations illustrate theflexibility provided to agencies to develop plans best suited to meet their needs, while meetingthe common SAO assurance requirements and direction on sufficient validations included inOMB M-17-04. A brief description of the agency operating environment is offered as anintroduction for each of the scenarios displayed.OMB’s requirement for implementing the DQP is effective for FY2019. Each agencies’ DQPwill be representative of their existing DATA Act process maturity. Plans will differ vastlydepending on the strength of controls in place over the reporting processes for the agency. Due tothe flexibility of this maturity model, agencies should prioritize and reassess their plans asappropriate. In cases where agencies have significant data quality internal control weaknesses,the interim focus should be on adequately executing corrective actions in areas where controlsare ineffective and then identifying significant reporting areas where assurance can be provided.Agencies have the flexibility to modify their DQP as changes in strategic priorities, new risks, orbusiness needs are identified and deemed appropriate by the agency.6
Data Quality PlaybookScenario 1Illustrative Scenario 1: Agency XYZThe following DQP outline was developed for a small, centralized agency with a robust controland governance structure. The integrated financial systems are modern and data qualitystandards had previously been in place to facilitate reconciliations of information reported ingovernment-wide systems. Readers may be able to leverage some of the processes ormethodologies described in this scenario depending on their DATA Act operating environment.I.IntroductionThe DATA Act requires federal agencies to report financial and award data in accordancewith the established Government-wide financial data standards. In May 2015, the OMBand Treasury published 57 data definition standards, most from existing data standards,and required federal agencies to report financial data in accordance with these standardsfor DATA Act reporting to be displayed on USAspending.gov for taxpayers and policymakers.The DATA Act requires the Office of Inspector General (OIG) of each federal agency toreview a statistically valid sample of the spending data submitted by its federal agencyand to submit to Congress a publicly available report assessing the completeness,timeliness, quality, and accuracy of the data sampled and the implementation and use ofthe Government-wide data standards by the federal agency when reporting the data.XYZ’s objective is to develop a process that assures data is submitted timely, contains allof the required 57 data elements, and ensures data accuracy for information published onUSAspending.gov. The objective will be met through training on the definition andstandardization of data elements and underlying business processes and categories,systemic controls, and periodic reviews that provide feedback that can be leveraged forimproving data quality. XYZ will follow the general concepts presented in OMB CircularA-123, Appendix A and an approach in accordance with the five components in the U.S.Government Accountability Office’s (GAO), Green book.II.Control EnvironmentXYZ will use a control environment as a foundation to provide discipline and structure tothe DATA Act submission process. This will include the oversight body made up ofleadership positions that have the ability to influence objectives and activities beingperformed to ensure the quality of DATA Act submissions.III.Agency-wide Roles and ResponsibilitiesA. Chief Financial Officer – DATA Act SAO Senior official responsible for certifying XYZ DATA Act submissions.7
Data Quality Playbook Scenario 1Required by M-17-04 to provide, on a quarterly basis, reasonable assurance thatthe agency's internal controls support the reliability and validity of the agencyaccount-level and award-level data reported to USAspending.gov. Per OMB M17-04, the SAO is to leverage existing data quality and management controlsestablished in statute, regulation, and Federal-wide policy. Data certifications thatare based in regulation and policy are sufficient for quarterly certifications and isnot the Quarterly SAO assurance that includes three certification levels:o The alignment among Files A-F is valid and reliable, including theinterconnectivity/linkages across all the data in Files A, B, C, D1, D2, E, andF.o The data in each DATA Act file submitted for display on USAspending.govare valid and reliable and existing data quality measures required byregulation and/or OMB guidance will be sufficient for SAO reliance onindividual data files.o Data reported to USAspending.gov matches, or is directly provided by, itsauthoritative sources outlined in M-17-04 (reference to ManagementProcedures Memorandum (MPM) 2016-03).B. Financial Assistance (grants, loans, surety bonds) Awarding Program Offices Responsible for data entry into source systems that feed the DATA Actsubmission.Design and implement control activities over the input of financial and awardspending data into relevant source systems to ensure completeness, timeliness,and accuracy.Heads of Financial Assistance Awarding Program Offices certify the accuracy offinancial assistance award data entered into source systems by the office monthlyto the SAO.o Grants: Executive Director of the Office of Executive Management, Installation &Support Services. Associate Administrator for the Office of Entrepreneurial Development.o Loans and Surety Bonds: Associate Administrator for the Office of Disaster Assistance. Associate Administrator for the Office of Capital Access. Associate Administrator for the Office of Investments and Innovation.C. Senior Procurement Executive and Chief Acquisition Officer 5The SPE, in coordination with the head of the contracting activity, is responsiblefor developing and monitoring a process to ensure timely and accurate reportingof contractual actions to FPDS-NG.Within 120 days after the end of each FY, the SPE reports an annual certificationof whether, and to what degree, agency contract data for the preceding FY iscomplete and accurate to the GSA5.Federal Acquisition Regulation Section 4.604 Responsibilities and OMB Memoranda of 2009, 2010, 20118
Data Quality Playbook Scenario 1Certifies the accuracy of procurement award data entered into the contract writingsystem and FPDS-NG monthly to the SAO provided this is in alignment with theagency’s risk management strategy and until such time that this agency iscomfortable with the quality of data per the DQP.D. Acquisition Division Responsible for data entry into source systems that are used by the DATA Actsubmission.Contracting Officers that award a contract action are responsible for thecompletion and accuracy of the individual contract action report to FPDS-NG6.Designs and implements control activities over the input of contract awardspending data into relevant source systems to ensure completeness, timeliness,and accuracy.E. Chief Data Officer Will maintain and oversee implementation of the agency-wide DQP.Responsible for oversight of data quality of the DATA Act submission.Supports any training necessary to promote compliance with the DATA Act.Oversees the review and resolution, if necessary, of Treasury validation warningand error messages.F. Office of the Chief Financial Officer IV.Compiles and submits DATA Act submission.Completes agency-wide data validations.Monitors and resolves any system
The Leveraging Data as a Strategic Asset Working Group developed and issued this Playbook in collaboration with federal agencies to assist with developing Data Quality Plans in support of the DATA Act implementation. This playbook is a result of extensive collaboration between agencies, OMB, and the Department of the Treasury.
