U.S. Department Of Homeland Security FY 2022-2026 Learning Agenda - DHS

Transcription

U.S. Department of Homeland SecurityFY 2022-2026 Learning AgendaWith honor and integrity, we will safeguard the American people, our homeland, and our values.

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaAbout this ReportThe Foundations for Evidence‐Based Policymaking Act of 2018 (Evidence Act) requires that theU.S. Department of Homeland Security (DHS) issue an evidence‐building plan (or learning agenda)aligned with the Department’s strategic plan describing priority questions and how they will beaddressed through the Department’s evidence building.The DHS FY 2022‐2026 Learning Agenda describes a subset of the Department’s future evidence‐building activities that align with strategic priorities and how the results of evidence‐buildingactivities will be used. The learning agenda is a plan for the Department’s strategic andcoordinated investment of resources in evidence building to ensure the best evidence is availablefor pressing decisions and high‐priority functions.As required, the DHS FY 2022‐2026 Learning Agenda is published at the DHS public websiteand at Evaluation.gov with the Department’s other Evidence Act plans and reports.DHS invites feedback on the DHS FY 2022‐ 2026 Learning Agenda and continued collaborationfrom relevant communities on potential priority questions, data, methods, and analyticapproaches that could guide these and future DHS evidence‐building activities. Public feedbackand input may be submitted to: dhslearningagenda@hq.dhs.gov.Contact InformationFor more information, contact:Michael Stough, Evaluation OfficerDepartment of Homeland SecurityOffice of the Chief Financial OfficerProgram Analysis and Evaluation Division245 Murray Lane SWMailstop 200Washington, DC 20528i

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaContentsAbout this Report. iOverview.1Introduction . 1Learning Agenda Development . 1Engage External Stakeholders. 2Gather Component Questions . 2Identify Department Priorities . 3Develop Study Plans . 4Publish and Disseminate. 4Learning Agenda Framework . 4Learning Agenda Summary . 6Study Plans.9Strategic Goal 1: Counter Terrorism and Homeland Security Threats . 9G1‐Q1In what ways do service interventions funded through violence and domesticterrorism prevention grants contribute to DHS goals for reducing individuals radicalizing totargeted violence and terrorism? . 9Strategic Goal 2: Secure U.S. Borders and Approaches . 10G2‐Q1What “push” and “pull” factors at the national and local levels predictincreases/decreases in the numbers of noncitizens arriving at the Southwest Border?.10G2‐Q2What effect, if any, have changes in southern land border controls had onmigrant encounters at sea? . 11G2‐Q3How accurate were DHS’s estimates of the benefits and costs in the regulatoryimpact analysis for the regulation, “Standards to Prevent, Detect, and Respond to SexualAbuse and Assault in Confinement Facilities”? . 12G2‐Q4What, if any, person and case characteristics predict higher likelihood of specificimmigration enforcement outcomes and time between border encounter and finaloutcome?. 14G2‐Q5What factors, if any, help predict rates and lengths of nonimmigrantoverstays?.15G2‐Q6What, if any, person and case characteristics predict higher likelihood ofreceiving specific immigration benefits?. 16ii

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaG2‐Q7What are the characteristics of people who naturalize? What demographicvariables among lawful permanent residents are correlated with the likelihood and timingof naturalization?. 18G2‐Q8 What are characteristics of people who online file compared to paper file forimmigration benefits?. 19G2‐Q9 What is the degree and source of variability in USCIS utilization and completionrates, across adjudicative staff, at USCIS field offices? . 20G2‐Q10 To what extent can available data predict adjudications actions sufficiently towarrant automation of steps toward completion or to automate whole processes?.21Strategic Goal 3: Secure Cyberspace and Critical Infrastructure. 23G3‐Q1How are stakeholders using cyber, critical infrastructure (CI), and emergencycommunications (EC) products and services they receive from CISA? To what extent dothese products and services add value, such as increasing awareness and closing gaps incapabilities, plans, and policies? What factors prevent stakeholders from using cyber/CI/ECproducts they receive? . 23G3‐Q2Do all subgroups of the target population participate in and perceive value andutility of national convenings as mechanisms of information exchange about security andrisk resilience? What gaps and unmet needs, if any, exist? . 24G3‐Q3To what extent has the all‐hazards communications unit position‐specific trainingand stakeholder communication unit program enhanced interoperability by (1) creating,maintaining, and deploying a qualified group of people to fulfill Communications Unit(COMU) roles in federal, state, local, tribal, and territorial (SLTT) partner organizations and(2) enhancing integrated, collaborative, communications planning? . 25Strategic Goal 4. Preserve and Uphold the Nation’s Prosperity and Economic Security .27G4‐Q1Which facilities should have Transportation Worker Identification Card (TWIC)readers to appropriately manage risk? . 27G4‐Q2What are the most significant sources of variation in transportation securitysector (TSS) operators’ implementation of cyber security technology and processes? Whatdo TSS operators report as enablers and barriers to implementation, and what gaps/unmetneeds could TSA address?. 28G4‐Q3Where does the majority of federally managed fishing activity (both inside theU.S. Exclusive Economic Zone and nearby) occur, by season and species?.29Strategic Goal 5. Strengthen Preparedness and Resilience . 30G5‐Q1How do individuals’ preparedness behaviors change over time?.30G5‐Q2How do household income and other demographic characteristics influence thedecision to purchase flood insurance?. 31iii

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaG5‐Q3To what extent is FEMA’s training and education enterprise providing equitabledelivery of curricula to state, local, tribal, and territorial (SLTTs) constituents? .32G5‐Q4What indicators, measures, methods, and data can better support the evaluationof Homeland Security Grant Program (HGSP) effectiveness as it pertains to maintaining andimproving state, local, tribal and territorial (SLTT) and national preparedness? .33G5‐Q5What indicators, data, methods, tools, and frameworks can FEMA use to buildevidence and address disparities in achieving equitable outcomes across FEMA programs(e.g., direct federal assistance and grants) across the agency? . 35G5‐Q6To what extent were COVID Public Assistance obligations align with areas ofsocial vulnerability and high case rates in socially vulnerable areas? . 36Goal 6: Champion the DHS Workforce and Strengthen the Department . 37G6‐Q1Does level of Independent Test Agent (ITA) support throughout the acquisitionlifecycle correlate with the adequacy of an acquisition program’s test and evaluationoutputs (operational effectiveness, suitability, and resilience)?. 37G6‐Q2What are the estimated costs and benefits of DHS vehicle fleet electrificationgiven specialized/law enforcement requirements? . 38G6‐Q3What effects has the adoption of maximum telework flexibilities had on the DHSmission and support for the DHS mission? What subgroups of the DHS workforce benefitthe most/least from telework flexibilities? . 39G6‐Q4What effects has adoption of a standard utilization rate (UR) of 150 sq.ft./person had on the Department’s real property footprint? What factors support andhinder implementation of the 150 sq. ft./person standard? . 41G6‐Q5What metrics/indicators can be used to reliably predict cost to maintain materielreadiness requirements for deployed systems in CBP’s stated five Enduring Mission PriorityOutcomes: Combating Transnational Organized Crime, Counter Terrorism, Facilitate LawfulTravel, Secure the Border, and Facilitate Lawful Trade and Protect Revenue?.42DHS Data Priorities for Evidence Building . 43Administrative and Operational Data Systems . 44Statistical Data and Data Compilation Assets . 45DHS Survey and Assessment Data Assets . 46Other Federal and Non‐Federal Data Sets. 47Study Types, Data Collection Methods, Analytic Approaches for Evidence Building.48Study Types . 48Data Collection Methods. 51Analytic Approaches. 53iv

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaChallenges and Mitigating Strategies for Evidence Building. 54Common Challenges. 55Mitigating Strategies . 56Appendix A. List of Abbreviations and Acronyms. 60v

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaOverviewIntroductionThe U.S. Department of Homeland Security (DHS) has a diverse and complex mission to preventattacks and mitigate threats against the United States and our allies, respond to nationalemergencies of all kinds, and advance American prosperity and economic security. Since DHS wasestablished from its predecessor agencies in 2003, the Department has continued to expand andmature capabilities to use data and analysis in shaping strategy and operations.DHS has developed this DHS FY 2022‐2026 Learning Agenda to empower Departmentdecisionmakers to achieve their objectives while fostering organizational learning. The learningagenda supports the Department’s implementation of the Foundations for Evidence BasedPolicymaking Act of 20181 (Evidence Act) by engaging stakeholders in identifying and prioritizingareas for evidence to improve program or policy effectiveness, assess progress toward outcomes,study pilot initiatives and programmatic adjustments, and inform resource management.Evidence may come from program evaluation, performance measurement, policy analysis, andresearch or statistics conducted for foundational fact finding.The DHS FY 2022‐2026 Learning Agenda describes a subset of the Department’s future evidence‐building activities that align with strategic priorities and how the results of evidence‐buildingactivities will be used. Except where limitations to public disclosure are noted, DHS intends todisseminate results on its public website to enable broad use in DHS management activities andby external stakeholders. DHS will update the learning agenda annually to reflect progress towardthe original priority questions, shifting learning priorities, changing contexts within which theDepartment operates, and emerging evidence needs for which new questions and plans will beadded. DHS invites feedback on the DHS FY 2022‐2026 Learning Agenda and continuedcollaboration from relevant communities to inform these and other evidence‐building activitiesthat may be included in annual updates.Learning Agenda DevelopmentDHS engaged internal and external stakeholders throughout fiscal year (FY) 2021 to develop thelearning agenda. Collaboration with stakeholders ensures that the Department’s learning agendaaddresses questions that are relevant, salient, and meaningful to stakeholders inside and outsidethe Department, and that the learning that results from evidence building will be timely anduseful for their decision making. Stakeholder engagement is described below in the context ofeach stage of the learning agenda development process, and is ongoing. Exhibit 1 and thesections that follow outline the key stages in the DHS FY 2022‐2026 Learning Agendadevelopment process.1Pub. L. No. 115‐435, 132 Stat. 5529 (2019)1

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaExhibit 1. DHS Learning Agenda Development estionsIdentifyDepartmentPrioritiesDevelopStudy PlansPublish andDisseminateEngage External StakeholdersDHS conducted external stakeholder engagement for the learning agenda using well‐establishedmechanisms and relationships. DHS solicited public input regarding questions or studies thatcould guide DHS evidence‐building activities through two Requests For Information (RFI).2 3Social media campaigns and email outreach to 27,000 private sector partners and committeemembers representing industry and trade groups, professional associations, nongovernmentalorganizations, advocacy groups, community groups, and the public invited participation in theRFIs.Members of the public, industry, research, and state and local government communitiessubmitted 17 responses through the Federal Registry and email. In addition, DHS directlyengaged other federal agencies, including the Office of Management and Budget, and reviewedpriorities of the Administration and Congress in executive orders and memoranda, Congressionalhearings, and Government Accountability Office (GAO) audits.Gather Component QuestionsDHS gathered employee input for the learning agenda through a variety of activities. AnnualStrategic Reviews, an internal management process conducted to improve program outcomes,provided the first opportunity to engage senior officials and mission program staff in surfacinglearning priorities. Briefings with Component senior leaders on the learning agenda developmentactivities sought their priority questions and invited Components’ participation.DHS Components, led by Component members of the DHS Evaluation Officer Council, plannedand conducted Component‐specific internal stakeholder engagement activities, using meetings,workshops, working groups, executive taskers, and employee engagement forums to identify andprioritize Components’ learning agenda questions and proposals. Their activities consideredinput from external stakeholder engagement, as well as Administration and Departmentleadership priorities. The DHS Evaluation Officer provided flexible tools and guidance to supportComponent efforts and hosted weekly webinars and office hours to promote awareness andworkforce engagement in Component‐led activities.The participating Components listed below advanced 122 priority questions and proposals thatwere considered for this learning agenda. More organizations within DHS will participate insubsequent annual activities to update and amend the learning agenda.23See Request for Information: Evidence Building Activities, 85 FR 71353 (DHS, 2019)See Request for Information: Evidence Building Activities, 86 FR 8921 (DHS, 2020)2

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaOperational Components U.S. Customs and Border Protection (CBP)Cybersecurity and Infrastructure Security Agency (CISA)Federal Emergency Management Agency (FEMA)U.S. Immigration and Customs Enforcement (ICE)Transportation Security Administration (TSA)U.S. Citizenship and Immigration Services (USCIS)U.S. Coast Guard (USCG)U.S. Secret Service (USSS)Support Components and Executive Leadership Offices Countering Weapons of Mass Destruction Office (CWMD)Federal Law Enforcement Training Center (FLETC)Intelligence and Analysis (I&A)Management Directorate (MGMT)Office of Strategy, Policy, and Plans (PLCY)Science & Technology Directorate (S&T)Identify Department PrioritiesDHS systematically prioritized a set of relevant questions and scientifically rigorous evidencebuilding from Components’ proposals. A working group of 13 senior Department officials andtechnical experts,4 with responsibilities for enterprise‐wide coordination and governance ofevidence building and data, conducted multiple rounds of systematic review, deliberation, andprioritization of the Components’ priority questions and proposals. The results of a criterion‐based review of questions and accompanying proposals submitted by Components served as thebasis for the working group’s prioritization of questions and proposals with high relevance toAdministration priorities and high potential to provide scientifically rigorous, timely evidence fordecision making.The working group recommended 30 priority questions and proposals to the DHS Deputy’sManagement Action Group for inclusion in the learning agenda. Those 30 priority questionsaddressed 47 of the 122 Component priority questions when accounting for similar or relatedquestions. The working group has been reconstituted as a coordination body to continuallypromote collaboration and unity of effort across evidence building activities.4The working group includes the DHS Evaluation Officer (who is also the Performance Improvement Officer),the DHS Statistical Official, the DHS Chief Data Officer, the DHS Chief Economist, senior scientific officials fromS&T, senior operations research experts, and their deputies.3

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaDevelop Study PlansComponent members of the DHS Evaluation Officer Council and question champions identifiedteams to assist the DHS Evaluation Officer in developing study plans. The teams provided subjectmatter, evidence‐building, and data expertise needed to draft plans for inclusion in the DHS FY2022‐2026 Learning Agenda. Teams considered the Learning Agenda Working Group’s feedbackin their planning and engaged additional stakeholders as needed.Twenty‐seven (27) study plans were fully developed during this stage. DHS will establishadditional teams as needed to plan evidence building that may be proposed in future annualupdates of this learning agenda.Publish and DisseminateThe DHS FY 2022‐2026 Learning Agenda is published at the DHS public website andevaluation.gov. The DHS public website will host future updates on and findings of the evidencebuilding undertaken from the learning agenda as well as amendments to the learning agendathat reflect evolving needs. DHS intends to release reports, summaries, and key findings that areappropriate and accessible for external stakeholders and the public, except in the few cases forwhich disclosure limitations are noted in the study plans.As DHS envisions continual stakeholder engagement on the learning agenda, we invite feedbackon the published DHS FY 2022‐2026 Learning Agenda and continued collaboration from relevantexternal communities on questions in which data and expertise reside outside DHS, externalresearchers have interest in supporting DHS evidence building, and questions reflect sharedpriorities or have broad implications beyond DHS.Learning Agenda FrameworkThe DHS FY 2022‐2026 Learning Agenda is organized according to the strategic goals in theDepartment’s strategic plan.5 For each goal, the learning agenda identifies priority questions thatbuild evidence and foster organizational learning. The strategic goals are: Goal 1: Counter Terrorism and Homeland Security ThreatsGoal 2: Secure U.S. Borders and ApproachesGoal 3: Secure Cyberspace and Critical InfrastructureGoal 4: Preserve and Uphold the Nation’s Prosperity and Economic SecurityGoal 5: Strengthen Preparedness and ResilienceGoal 6: Champion the DHS Workforce and Strengthen the DepartmentAs required by the Evidence Act, the DHS FY 2022‐2026 Learning Agenda questions consider abroad range of Administration and Department priorities such as domestic terrorism,immigration, cybersecurity, infrastructure, climate, equity, COVID‐19, and the federal workforce.5The DHS Strategic Plan Fiscal Years 2020‐2024 (DHS, 2020)4

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaThe organization of the DHS FY 2022‐2026 Learning Agenda is as follows:The Learning Agenda Summary section outlines summary information for each priority question: the primary DHS strategic goal and objective number addressed by the question, orcrosscutting if multiple goals are addressed;the proposed evidence‐building activity(s) for each priority question, includingfoundational fact finding (FFF), policy analysis (PA), performance measurement (PM), andprogram evaluation (PE);the fiscal year in which evidence building will start; andthe timeframe for expected results, indicated short term (ST) if results are expected FY2022‐2024 and long term (LT) if results are expected in FY 2025‐2026, or if the activity isrecurring.The Study Plans section is organized by strategic goal and provides plans for each priorityquestion. The study plans describes: the strategic or operational background for the priority question and the primary strategicobjective addressed;the proposed evidence building, including the evidence‐building activity(s), data,methods, analysis, and tools needed; andthe anticipated use and dissemination of findings from evidence building.Three sections provide additional information about DHS evidence building. The Data Priorities section describes general data types and named data sets that areproposed for DHS evidence building.The Study Types, Data collection Methods, and Analytic Approaches section providesmore explanation of commonly used and aspirational aspects of DHS evidence‐buildingactivities.The Challenges and Mitigating Strategies section describes barriers to evidence buildingthat are common across DHS organizations and collective solutions DHS is deploying toovercome such barriers.The information included in this plan has been collaboratively developed with the Componentsthat are the principal producers and users of the evidence and meets Evidence Act and OMBrequirements. The details provided ensure that DHS stakeholders have a clear and collectiveunderstanding of what priority evidence building is intended and how it will be used to informdecision making in the next four years.5

U.S. Department of Homeland SecurityFY 2022‐2026 Learning AgendaLearning Agenda SummaryExhibit 2 provides a summary of the learning agenda’s evidence‐building questions and plans.Exhibit 2. Summary of DHS Evidence Building Questions and PlansQuestionIDPriority tartExpectedResults1.2PM, PEFY22LTStrategic Goal 1: Counter Terrorism and Homeland Security ThreatsG1-Q1In what ways do service interventions funded throughviolence and domestic terrorism prevention grantscontribute to DHS goals for reducing individualsradicalizing to targeted violence and terrorism?Strategic Goal 2: Secure U.S. Borders and ApproachesG2-Q1What “push” and “pull” factors at the national and locallevels predict increases/decreases in the numbers ofnoncitizens arriving at the Southwest Border?2.1FFFFY22LTG2-Q2What effect, if any, have changes in southern land bordercontrols had on migrant encounters at sea?2.1FFFFY22LTG2-Q3How accurate were DHS’s estimates of the benefits andcosts in the regulatory impact analysis for the regulation,“Standards to Prevent, Detect, and Respond to SexualAbuse and Assault in Confinement Facilities”?2.3PEFY22STG2-Q4What, if any, person and case characteristics predicthigher likelihood of specific immigration enforcementoutcomes and time between border encounter and finaloutcome?2.3FFFFY22LTG2-Q5What factors, if any, help predict rates and lengths ofnonimmigrant overstays?2.3FFFFY22LTG2-Q6What, if any, person and case characteristics predicthigher likelihood of receiving specific immigration benefits?2.4FFFFY22LTG2-Q7What are the characteristics of people who naturalize?What demographic variables among lawful permanentresidents are correlated with the likelihood and timing ofnaturalization?2.4FFFFY22LTG2-Q8What are the characteristics of people who online filecompared to paper file for immigration benefits?2.4FFFFY22STG2-Q9What is the degree and source of variability in USCISutilization and completion rates, across adjudicative staff,at USCIS field offices?2.4FFFFY22STG2-Q10To what extent can available data predict adjudicationsactions sufficiently to warrant automation of steps towardcompletion or to automate whole processes?2.4FFFFY22STNote: Evidence building includes the following: FFF is foundational fact finding, PA is policy analysis, PM isperformance measurement, and PE is program evaluation. Expected results includes the following: short term (ST) ifresults are expected FY 2022-2024 and long term (LT) if results are expected in FY 2025-2026, or if the activity isrecurring.6

U.S. Department of Homeland SecurityFY 2022‐2026 Learning rtExpectedResultsG3-Q1How are stakeholders using cyber, critical infrastructure(CI), and emergency communications (EC) products andservices they receive from CISA? To what extent do theseproducts and services add value, such as increasingawareness and closing gaps in capabilities, plans, andpolicies? What factors prevent stakeholders from usingcyber/CI/EC products they receive?3.2PM, PEFY22STG3-Q2Do all subgroups of the target population participate in andperceive value and utility of national convenings asmechanisms of information exchange about security andrisk resilience? What gaps and unmet needs, if any, exist?3.2PEFY23STG3-Q3To what extent has the all-hazards communications unitposition-specific training and stakeholder communicationunit program enhanced interoperability by (1) creating,maintaining, and deploying a qualified group of people tofulfill Communications Unit (COMU) roles in federal, state,local, tribal, and territorial (SLTT) partner organizationsand (2) enhancing integrated, collaborative,communications planning?3.2PM, PEFY23STQuestionIDPriority QuestionStrategic Goal 3: Secure Cyberspace and Critical InfrastructureStrategic Goal 4: Preserve and Uphold the Nation’s Prosperity and Economic SecurityG4-Q1Which facilities should have Transportation WorkerIdentification Card (TWIC) readers to appropriatelymanage risk?4.2FFF, PAFY22STG4-Q2What are the most significant sources of variation intransportation security sector (TSS) operators’implementation of cyber security technology andprocesses? What do TS

2026 Learning Agenda. is published at the DHS public website and evaluation.gov. The DHS public website will host future updates on and findings of the evidence building undertaken from the learning agenda as well as amendments to the learning agenda s,summaries,andkeyfindingsthatare