Transcription
Federal Data StrategyImproving Agency Data SkillsPlaybookMay 2020
Federal Data StrategyImproving Agency Data Skills PlaybookContentsImproving Agency Data Skills Playbook . 1Step 1 – Identify any critical data skills needed for the agency . 3Checklist . 5Example. 5Questions for further thought . 5Step 2 – Assess current staff capacity for needed data skills . 7Checklist . 7Example. 7Questions for further thought . 8Step 3 – Perform a data skills gap analysis . 9Checklist . 9Example. 9Questions for further thought . 10Step 4 – Identify and execute ways to meet those needs . 11Checklist . 12Example. 12Questions for further thought . 13Metrics and Related Practices and Actions . 14Metrics . 14Related strategy practices and actions . 14Appendix A . 16Federal Data Lifecycle Roles. 16Appendix B . 17Roles and Skills . 17Data roles and skills . 18Other roles and skills . 20Appendix C . 22Crosswalk of Federal Data Strategy Practices and the Federal Data Lifecycle . 22STRATEGY.DATA.GOVPage iiRESOURCES.DATA.GOV
Federal Data StrategyImproving Agency Data Skills PlaybookImproving Agency Data Skills PlaybookImproving staff data skills will allow agencies to better harness the power of data. To help agenciesaddress this need, this playbook provides advice and tools to help agencies implement Action 4 of the2020 Federal Data Strategy Action Plan (hereafter “2020 Action Plan”), “Identify Opportunities toIncrease Staff Data Skills.” Executing the activities contained in this document will help agenciesprogress in fulfilling requirements of the Foundations for Evidence-Based Policymaking Act of 2018 inaddition to related guidance 1- 3.This playbook’s structure mirrors and expands upon the four major steps of a data skills capacityassessment:1. identify critical data skills needed for the agency,2. assess the current staff capacity for those data skills,3. perform a data skills gap analysis to prioritize the agency’s needs, and4. identify and execute approaches to meet those needs.The “who, what, and how” of each step is broken out below, followed by practical activities agencies willfind helpful in executing the respective step. These activities are not strict, prescriptive instructions thatrequire exact execution. Rather, agencies should prioritize and adapt the individual approaches in waysthat best apply to their own organizations. While the scope for this assessment is intended to be at theagency level, some agencies may find it more helpful to do a more granular analysis at the operating orbusiness unit level, particularly when skills and mission differ across an agency. If an agency takes such adistributed approach, a centralized body should analyze whether gaps can be filled by another part ofthe organization (e.g., the analytics office can provide advanced statistical analysis to other units).As each agency is part of a larger Federal Government community, agencies should look to one anotherfor inspiration and lessons learned as they identify and meet their data skills needs. Coordination andknowledge sharing will be critical to the success and longevity of efforts to develop data skills. Somecouncils that will be important for this include: Chief Data Officers Council Interagency Council on Statistical Policy Human Capital Planning and Resources councilsAgencies can also leverage centralized data governance bodies for knowledge sharing within theorganization.Foundations for Evidence-Based Policymaking Act of 2018, Pub. L. No. 115-435, 132 Stat. 5529.OMB M-19-23, Phase 1 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Learning Agendas,Personnel, and Planning Guidance, July 10, 2019, found at: pdf.3 The Evidence Act also requires a capacity assessment, which in part determines “the extent to which evaluation and researchcapacity is present within the agency to include personnel” and “the extent to which the agency has the capacity to assistagency staff and program offices to develop the capacity to use evaluation research and analysis approaches and data in theday-to-day operations.” This can help guide agencies in fulfilling Step 2 – “Assessing current staff capacity for needed dataskills” and Step 3 – “Perform a data skills gap analysis” of this Action.12STRATEGY.DATA.GOVPage 1RESOURCES.DATA.GOV
Federal Data StrategyImproving Agency Data Skills PlaybookFigure 1: Process to identify opportunities to increase staff data skillsSTRATEGY.DATA.GOVPage 2RESOURCES.DATA.GOV
Federal Data StrategyImproving Agency Data Skills PlaybookStep 1 – Identify any critical data skills needed for theagencyWhat? The agency formulates a clear idea of how data skills play a role in the execution of its mission(s)and supporting leadership priorities and its functions at the present and continuing into the future.Who? The agency’s Chief Data Officer, Statistical Official, Evaluation Officer, Chief Human CapitalOfficer, Chief Learning Officer, and Data Governance Body.How? Identify agency leadership priorities and mission imperatives around data 4 and then conveneagency staff to outline the ways data flows into and within the agency and is used to support decisionmaking or otherwise create value towards these priorities. It is important to take a holistic, integratedview for assessing data skills – both horizontally across operating units, and vertically within the agencyhierarchy. An agency-level assessment is required; however, this assessment can be performed at thelevel that makes sense for the agency. If a distributed approach is used, a centralized group shouldreview the needs or gaps from each of the operating units of the agency and compile. A commonvocabulary for the federal data lifecycle was generated to support both this playbook and also for use inAction 13 of the Federal Data Strategy 2020 Action Plan, “Develop a Curated Data Skills Catalog,” byadapting the NIST Big Data Reference Architecture 5 and linking subsequent data roles with Federal DataStrategy practices. The framework below lays out a variety of roles for those involved in the federal datalifecycle (Appendix A). These include data-focused roles 6 (given on the next page) and others in theorganization such as leadership and domain experts. The required data skills are given for each of theroles in Appendix B.456The learning agenda development process will be a helpful resource for identifying these priorities. See M-19-23, Phase 1Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Learning Agendas, Personnel, and PlanningGuidance, July 10, 2019, found at: pdf.U.S. Department of Commerce. National Institute of Standards and Technology (NIST). NIST Big Data Program. Retrieved NIST.SP.1500-6r2.pdf.Agency personnel in all data roles need to have a basic understanding of their agency’s cybersecurity and privacy policies thatgovern data and need to work closely with their agency experts in cybersecurity and privacy throughout the data lifecycleMore information on the Federal Information Security Modernization Act of 2014 can be found dfSTRATEGY.DATA.GOVPage 3RESOURCES.DATA.GOV
Federal Data StrategyImproving Agency Data Skills PlaybookFigure 2: Federal data lifecycleData roles Define: Identify agency and stakeholder needs for data of sufficient quality for intended uses Coordinate: Assess the ability of data resources and infrastructure to meet agency andstakeholder needs Collect: Organize, plan, and execute data collections and acquisitions to meet agency andstakeholder needs Curate: Organize, refine, and maintain agency data resources with sufficient quality to meetagency and stakeholder needs Access: Identify and develop multiple data access methods for agency staff and stakeholders Analyze: Optimize the ability of staff and stakeholders to use agency data to generate insights Visualize: Present data insights for consumption by leaders and stakeholders Disseminate: Provide multiple avenues for release of data and insights Implement & Assess: Maximize the use of data for decision-making, accountability, and thepublic good and continuously improving the data processOther roles Leadership: Cultivate and support a culture of enterprise-wide demand and use of data tomaximize outcomes All Others: Each staff member values the use of data in their day-to-day work Domain Expert: Understands the context around the data, the needs of all involvedstakeholders, and often takes on many roles throughout the process in an advisory or leadcapacity to inform collection, data systems, data dictionaries, data set design, and analysis Privacy and Security: Ensure that agencies are consistently adopting and using the most up-todate methods to protect data and comply with all applicable laws and regulationsIn addition, the activities and resulting conclusions from this Step may help inform the agency’s effortsto address other Actions from the Federal Data Strategy 2020 Action Plan, including Action 1 “IdentifyData Needs to Answer Priority Agency Questions,” Action 2 “Constitute a Diverse Data GovernanceBody,” Action 3 “Assess Data and Related Infrastructure Maturity,” and Action 5 “Identify PriorityDatasets for Agency Open Data Plans.”STRATEGY.DATA.GOVPage 4RESOURCES.DATA.GOV
Federal Data StrategyImproving Agency Data Skills PlaybookThe following checklist and questions further break down how an agency might approach Step 1.Checklist Identify agency leadership priorities and mission imperatives around data. Outline, using a shared vocabulary (e.g., the federal data lifecycle presented herein), the dataneeded and the processes or activities agency staff perform to transform raw data into usefulknowledge to address leadership priorities. Discuss and document the data skills needed to execute these processes or activities. The skillsshould be documented regardless of whether agency personnel currently possess them. If adistributed approach is taken, compile the results at an agency level and review with the agencydata governance body.ExampleAgency X is responsible for ensuring facilities are maintained and safely able to support partneragencies’ missions. The table shown on page 6 represents Agency X’s notes from a meeting convened tothink through Step 1 of this Action. (Data roles are not necessarily aligned with a person’s job role as oneperson may perform more than one data role.)Questions for further thought How do an agency’s or operating unit’s public and private peers assess their own data skillsneeds?How well do the agency’s position descriptions match up with the data skills uncovered in theabove exercises?STRATEGY.DATA.GOVPage 5RESOURCES.DATA.GOV
Federal Data StrategyRoleImproving Agency Data Skills PlaybookCritical processes in our contextRelated data skills (technical and non-technical)DefineIdentify questions for data analysis to evaluate strategic andoperational outcomes for Agency XUnderstand the strategic and operational objectives of Agency X;facilitation; communicationCoordinateIdentify data sets internal and external to Agency X requiredto answer key questionsBe familiar with enterprise data architecture sources and systems used byAgency X; problem solving; understand how to assess data fitness for useCollectGather information about facilities from site inspection reportsystems and purchase data from external sources forbenchmarking and work with CFO office for cost dataWork with and query the SQL databases where these reports are stored;pull data from a NoSQL database; knowledge of data acquisitions; facilitatedata use agreements internallyCurateCombine, organize, refine, and maintain Agency X dataresources about facilities and costHandle raw and unstructured data; knowledge of programming tools;ability to develop scalable ETL packages; knowledge of data warehouse andbig data technologies; ability to understand stakeholder needs andrequirementsAccessStore sensitive facility and cost data in a secure way thatensures analysts have the proper accessUnderstand the laws, policies, and supportive processes, e.g., Privacy Act 7,Information Quality Act 8,9, disclosure avoidance techniquesAnalyzeModel real rate of facility degradation in comparison withtheoretical design life and use data analysis techniques toexplore data collected to answer other priority questionsImport data into statistical software; model facility/component condition,and prepare results for visualization using trend or pattern analysis andpredictive modelingVisualizeDesign and communicate data analysis that tells a story and isrelevant to data user needs to depict the past, current, andexpected future health of facilities and equipmentUse robust data storytelling techniques to communicate results with theteam members; programming or tools to create static or interactivereports, graphics, and dashboards; ability to gather/understand audiencerequirementsDisseminateShare data about facilities with the operating unit-levelfacilities office and create an open data set for public releaseDevelop and implement communication products; ability to develop,maintain, and utilize relationships with key stakeholders; familiarity withdifferent data formats and API technologyImplementand AssessCommunicate the implications of performing maintenanceactivities in the current vs. future fiscal yearsPresent insight from data to a range of audiences including seniorleadership; strategic and innovative thinking; process improvementPrivacy Act of 1974.Information Quality Act of 2001.9 OMB Memorandum M-19-15 Improving Implementation of the Information Quality Act issued April 24, 2019.78STRATEGY.DATA.GOVPage 6RESOURCES.DATA.GOV
Federal Data StrategyImproving Agency Data Skills PlaybookStep 2 – Assess current staff capacity for needed dataskillsWhat? Having identified the processes and the associated data skills that the agency performs to turnraw data into actionable knowledge, the agency needs to determine which of those skills staff andmanagers currently possess. Without a clear understanding of how the current workforce’s data skillsmatch up with needed data skills, the agency cannot determine how to invest in filling gaps.The agency should view the skills assessment in terms of a network or ecosystem of staff and skills. Forexample, increasing staff capability and capacity for predictive analytics alone will not result in effectiveuse of predictive analysis products. Managers and leaders, program staff, and mission support staff alsoneed specific applicable data skills in order for these advanced data products to be effectively used.Who? The agency’s Chief Data Officer, Statistical Official, Evaluation Officer, Chief Human CapitalOfficer, Chief Learning Officer, and Data Governance Body.How? Develop and implement a process to determine alignment of staff data skills to critical agencydata skills identified in Step 1.Checklist Decide how best to assess staff capacity to perform the data skills identified in Step 1 across theagency. Possible options include analyzing agency position descriptions and personnel data,querying supervisors or staff through surveys or focus groups, and measuring knowledgethrough pre- and post-training tests. Often a combination of methods will be needed to fullyassess capacity. Creating survey and test questions and focus group protocols are examples ofactivities that require using appropriate data skills to properly develop. Perform the assessment and review the results.ExampleAgency X took a two-pronged approach to Step 2 – Assess the current staff capacity for needed dataskills. Using notes from its Step 1 discussions for inspiration, the agency Evaluation Officer, Chief DataOfficer and Statistical Official created a survey with 10 questions capturing a) to what extent programmanagers consider a given data process is relevant to their work, and b) to what extent, if relevant, theybelieve their staff have the requisite skills to execute that process. In addition to this widely-distributedsurvey, the team coordinated with the Chief Human Capital Officer and Learning Officer, and conductedseveral hour-long focus groups to get a deeper understanding of current data skills capacity. To roundout the analysis, they worked with the Chief Human Capital Officer to analyze personnel data todetermine potential staff turnover using retirement eligibility in key data roles. This approach wasfeasible for Agency X; yet, a different method may be necessary for other agencies.STRATEGY.DATA.GOVPage 7RESOURCES.DATA.GOV p
adapting the NIST Big Data Reference Architecture. 5. and linking subsequent data roles with Federal Data Strategy practices. The framework below lays out a variety of roles for those involved in the f ederal data lifecycle (Appendix A). These include data-focused roles. 6
