Transcription
DoD Cybersecurity Test & Evaluation:Where We Were, Where We Are &Where We Are Going!Prepared for 2014 NDIA T&E ConferenceMr. Pete Christensenpchris@mitre.org703-983-2516With support fromMs. Jean Pettyjpetty@mitre.org703-983-9269Special Thanks toDASD DT&E and OSD DOT&E22 July 20141300-1400 2014 The MITRE Corporation. All rights reserved.Some portions previously Approved for Public Release; Distribution Unlimited. 14-1912
2What, Why and How? What do we want to accomplish? – Provide an overview of DOD Cybersecurity T&EActivitiesWhy is this important?– Existing processes have been ineffective!– Cybersecurity T&E, Systems Security Engineering(SSE), and RMF processes must be aligned andmutually supportive– DT&E should provide feedback as early as possible!– OT&E outcomes will be better!How will we do it?– Overview DOD Cybersecurity T&E Phases– Overview TRMC and National Cyber Range– Discuss Cyber Evaluation Framework– Walk through a simple example and have fun!Cybersecurity WORDLECyber Threats WORDLEDefense Acquisition WORDLECyber GothsGraphic Source: WIKIPEDIA Commons 2014 The MITRE Corporation. All rights reserved.Pending Public Release
(Former) DASD (DT&E) Principal DeputyDr. Steve Hutchison: Interoperability, Security T&ETechnologyMaturation &RiskReductionSecurity T&EDIACAPLate to Need!3 2014 The MITRE Corporation. All rights reserved.Pending Public Release
Where We Are Now:Ongoing Policy and Guidance Activities Interim DoDI 5000.02: Issued 26 Nov 2013– New/better guidance for both developmental and operational testing of IT DoD 8500.01, Cybersecurity: Issued 14 Mar 2014– “Cybersecurity” adopted for DoD: replaced “information assurance”– Policy: Risk Management, Resilience, Integration and Interoperability – Applied early, integrated across lifecycle DoDI 8510.01 – Risk Management Framework (RMF) for DoD IT:Issued 14 Mar 2014– Implements RMF (replaced DIACAP)– Policy, Responsibilities, Visibility, Reciprocity Cybersecurity T&E Process– DASD DT&E internal guidelines developed until DAG promulgated– DASD DT&E and OSD DOT&E are collaborating Defense Acquisition Guidebook Chapter 9– DASD SE, DT&E and OSD DOT&E are collaborating Cybersecurity Implementation Guidebook for PMs– Will address Cybersecurity T&E Cybersecurity T&E Guidebook– Work in progress to provide more detailed Cybersecurity T&E guidance 2014 The MITRE Corporation. All rights reserved.Pending Public ReleaseFollowing DoDI8500 series 4
CybersecurityImportant New Revisions to DoD 8500 5 Adopts the term: “Cybersecurity” Implements Risk Management Framework (RMF)– New guidance from the National Institute of Standards and Technology (NIST)and Committee on National Security Systems Instruction (CNSSI) documentson cybersecurity– Mission Assurance Category/Confidentiality Level (MAC/CL) replaced withCybersecurity Attributes (confidentiality, integrity, and availability) and impactlevels (high, moderate, low) Other terminology changes– Certifying Authority Security Control Assessor– Certification and Accreditation Assessment and Authorization– Designated Approving Authority (DAA) Authorizing Official (AO)Coordinating Security Controls Assessments and T&E can makeCybersecurity A&A more efficient! 2014 The MITRE Corporation. All rights reserved.Pending Public Release
Risk Management Framework (RMF) for InformationSystems and Platform Information Technology (PIT)SystemsGraphics Source: DoDI 8510.01 – Risk Management Framework (RMF) for DoD IT: Issued 14 Mar 2014 2014 The MITRE Corporation. All rights reserved.Pending Public Release 6
7 RMF Steps 4 and 5 Necessary But Not Sufficient ToUnderstand Systems Real Cybersecurity Posture!T&E verifies implementation andidentifies and closes residualvulnerabilities overlooked in designand implementation!Graphics Source: DoDI 8510.01 – Risk Management Framework (RMF) for DoD IT: Issued 14 Mar 2014 2014 The MITRE Corporation. All rights reserved.Pending Public Release
8 Cybersecurity T&E PhasesMS AReqDecisionMS BMS CIATTPreEMDTechnologyMaterielDRAFT Maturation &Solution AOA CDDRisk ReductionAnalysisMDDASR SRRSFRCDDPDREngineering yRequirementsCharacterizeCyber AttackSurfaceCPDDT&ETRR Event ll RateProductionDecision ReviewProduction andDeploymentO&SOTRR IOT&EVulnerability andPenetrationAssessmentAdversarialAssessment Phases as depicted are notionally mapped to milestones and designreviews Phases are incremental and iterative as system matures Phases 3/5 DT&E and 4/6 OT&E analogous with different objectives! DT&E Shifts “vulnerability discovery” earlier in acquisition life cycle to help PMachieve acquisition goals! 2014 The MITRE Corporation. All rights reserved.Pending Public Release
Cybersecurity T&E Complements SSE and RMF toPositively Impact Cost Schedule and Performance! Cybersecurity T&E should be “Multi Purposed”– Collaborative activity involving all “responsible”stakeholders– Started as early as possible in Acquisition– Verify requirements and baseline capabilities– Evaluate exposed “Attack Surface”– Identify and help close exposed vulnerabilities– Evaluate system resilience in operationalcontext– Provide early feedback to “responsible”stakeholders– Reduce Cost, improve schedule and informLRIP– Improve OT&E Outcomes 2014 The MITRE Corporation. All rights reserved.Pending Public ReleaseGraphic Source: WIKIPEDIA Commons9
Phase 1 - Understand CybersecurityRequirementsT&E WIPT develop Cybersecurity T&E Strategy Understand Program Protection Plan and Cybersecurity Strategy– Critical Components, Software, RMF Security Categorization, etc. Identify cybersecurity requirements for Cybersecurity T&E– Critical Operational Missions and supporting systems– Critical data exchanges and interfaces– Additional implied (derived) and essential requirements Identify cybersecurity test organization(s)– Security Controls Assessor, Vulnerability Identification/Assessment Teams Identify Cybersecurity T&E Resources– Cyber range resources(e.g., National Cyber Range (NCR), DoDCybersecurity Range, Joint Information Operations Range (JIOR))– Cybersecurity Test Tools, M&S needs Plan to integrate Cybersecurity into overarching T&E StrategyT&E WIPT should engage SMEs in a Core Team to execute! 2014 The MITRE Corporation. All rights reserved.Pending Public Release 10
11 Essential Cybersecurity Requirements “Distilled” from BothSpecified and Derived RequirementsSpecifiedRequirements Clearly identified inprogramdocumentation ICDs/CDDs,CONOPs, ProductSpecifications andPPP Requirementsmandated by Lawand DoD Policy andRegulations Risk ManagementFrameworkEssentialRequirements Must be achieved supportmission accomplishment inresponse to cyber attack Kill Chain analysis helps identifyessential cybersecurityrequirements Goal is to ensure resilience ofthe operational system despitecyber attack.DerivedRequirements Driven by operationalcapabilitiesDriven by acquisitionapproach and/ortechnology choices, e.g.COTS/GOTSTechnical requirementsthat enable thecapabilities defined inCONOPS, etc.Includes the Cyber ThreatenvironmentEvaluate Cyber attacksurface to identify theadditional impliedcybersecurityrequirements.T&E WIPT collaborates to confirm requirements, testability, identify test resourcesand plan T&E events! 2014 The MITRE Corporation. All rights reserved.Pending Public Release
12 Phase 2 - Characterize the Cyber Attack SurfaceIdentify the seams and gaps between theCybersecurity Strategy/RMF Artifacts and “Verify”the system as planned/built- Utilize cybersecurity SMEs to assist- Review Technical Requirements, SecurityArchitectures, Preliminary/Critical Designs- Examine system Capabilities Documents,CONOPS and Operational Architectures- OV-3 Operational Information Exchanges,OV-6 Critical Missions- Examine ISP and system architectureproducts- SV-1, SV-6 viewpoints identify interfacingsystems, services, and data exchangesE-2C HawkeyeU.S. Navy Photo (RELEASED)USMC Tactical VehicleU.S. Navy Photo (RELEASED) 2014 The MITRE Corporation. All rights reserved.Pending Public Release
13 Working Definition: Attack SurfaceCAN-BusLAN-WANEthernet LANSCADA Network1553 Data BusSCADA NetworkMulti-hop Wireless SensorNetworkAttack Surface: A system’s exposure to reachable and exploitable cybervulnerabilitiesSource: SANS Attack Surface Problem: rticle/did-attack-surface 2014 The MITRE Corporation. All rights reserved.Pending Public ReleaseGraphics Sources: WIKIPEDIA Commons
14 Phase 3 – Vulnerability Identification Evaluate Baseline Performance and Identify and Close exposed vulnerabilities in a SOS ContextConfirm “Baseline Performance”– Functional test data– Evaluate SW/HW Cybersecurity test data– RMF security controls assessment data– Enumerate and close vulnerabilitiesTeam has full knowledge/access to system– Works collaboratively to perform assessmentConduct cybersecurity testing in SOS context– Include or emulate the CNDSP in test infrastructure– Exercise Mission Threads– Use Kill Chain Model to portray cyber threats– Enumerate residual vulnerabilities and evaluate mission impact– Provide results to SE Team for remediationT&E WIPT must engage vulnerability assessment team to plan and execute Phase 3! 2014 The MITRE Corporation. All rights reserved.Pending Public Release
155 Vulnerability Identification and Adversarial T&E Verifies RMF Controls and validates them as implemented Identifies exposed vulnerabilities Technical Vulnerabilities require resources to mitigate Operational and Administrative Vulnerabilities impact CONOPS, TTPs and Training Threat Portrayals are developed by Vulnerability Assessment Teams– Teams have “Full Knowledge” of the System and Mission Threat Agents exploit weaknesses/vulnerabilities in controls to capabilities– Cyber Attacks are portrayed by Vulnerability Assessment Teams Exploits ultimately impact system resiliency and operational lGraphic Sources: WIKIPEDIA Commons 2014 The MITRE Corporation. All rights nistrativeTechnicalPending Public ReleaseSystemCapabilitiesMissionImpacts
16 Cybersecurity Testing Resources(SCA)Security ControlsAssessors(Blue Team)Cooperative VulnerabilityIdentification(Red Team)Adversarial VulnerabilityExploitationFocus is compliance with RMFcontrolsCooperative and comprehensiveassessment with full knowledge andaccess to systemNon cooperative and adversarialassessment to exploit known orsuspected weaknessesExecutes the Security AssessmentPlan (SAP)Exposes known/discovers newvulnerabilities present in systemsAttention on specific problem or attackvectorLinked to the Certification andAccreditation systemReveals systemic weaknesses in securityprogramDevelops an understanding of inherentweaknesses of systemBased on Security TechnicalImplementation Guides (STIGs) orsimilar documentationFocused beyond adequacy &implementation of technical securitycontrols and attributesBoth internal and external threatsCan be determined by multiplemethods: hands-on testing,interviewing key personal, etc.Multiple methods used: hands-ontesting, interviewing key personal, orexamination of relevant artifactsModel actions of a defined internal orexternal hostile entityIncludes a review of operational andmanagement security controlsFeedback to developers, systemengineers and administrators for systemremediation and mitigationReport at the end of the testingConducted with full knowledge andassistance of systemsadministrators, owner and developerConducted with full knowledge andcooperation of systems administratorsConducted covertly with minimal staffknowledgeNo harm to systemsMay harm systems and components andrequire clean upMay harm systems, may not harm people 2014 The MITRE Corporation. All rights reserved.Pending Public Release
17 Working Definition: Cyber Attack LifecycleMITRE: Cyber Attack LifecycleCyber Attack Lifecycle:Framework to understandand anticipate the moves ofcyber adversaries at eachstage of an attack.Typical adversary attack stages include:Reconnaissance, weaponization, delivery, exploitation, control, execution,and persistence.Source: Mandiant APT 1 Attack Cycle 2014 The MITRE Corporation. All rights reserved.Pending Public Release
Vulnerability Assessment Teams “Portray” CyberAttack LifecycleVulnerability Assessment TeamPortrays Advanced PersistentThreat (APT)Operators ExerciseSystem Under Test,Mission Threads Recon Detect Weaponize Deny Deliver Exploit ControlAPT attemptsmultipleattacks whileadjusting forsuccess or failureDefendersattempt toanalyze attacksand determinecourses of action Disrupt Degrade Deceive Execute Destroy Maintain RecoverAPT Objectives Exfiltrate data Violate data availability Corrupt data integrity 18 Data Collection Attacker actions Defender detections Defender actions Mission activitySource: Institute for Defense Analysis (IDA), February 2013Defender Objectives Protect Against IntrusionsDetect IntrusionsReact to IntrusionsMitigate IntrusionsDetermine ResponsesRestore After intrusion
19 Phase 4 – Adversarial Cybersecurity DT&E“Adversarial” Assessment to evaluate “Cyber Resiliency” in mission context! Assessment Team identifies and evaluates remaining and or residualvulnerabilities Include or emulate the CNDSP in test infrastructureInclude typical users if available and exercise Mission ThreadsPortray threats in a contested cyber domainTeam emulates the threat adversary TTPs to exercise Cyber AttackLifecycle– Analyze results to determine impact to mission– Recommend corrective actions to improve resilience“Cyber Resiliency” ability of a nation, organization, or mission or business process (andsupporting systems) to anticipate, withstand, recover from, and evolve to improve capabilitiesin the face of, adverse conditions, stresses, or attacks on the supporting cyber resources itneeds to function.T&E WIPT must engage Vulnerability Assessment Team to plan/execute Phase 4! 2014 The MITRE Corporation. All rights reserved.Pending Public Release
Developmental Evaluation FrameworkDecisions lity #2Decision #1DSQ #1Functional evaluationareasTechnicalReqmtsSystem x.x.5PerformanceCapability #13.x.x.6DSQ #2Decision #2DSQ #3DSQ #4Decision #4Decision #3DSQ #5DSQ #6DSQ #7DSQ #8DescriptionIdentify major decision points for which testing and evaluation phases, activity and events will provide decision supporting information.Cells contain description of data source to be used for evaluation information, for example:1) Test event or phase (e.g. CDT1.)2) M&S event or scenario3) Description of data needed to support decision4) Other logical data source descriptionTechnical Measure #1DT#1M&S#2DT#4M&S#2Technical Measure #2M&S#1DT#3DT#4M&S#23.x.x.7Technical Measure #33.x.x.8Technical Measure #43.x.x.1Technical Measure #13.x.x.2Technical Measure #23.x.x.3Technical Measure #33.x.x.4Technical Measure yCapability #3Test / M&SSystem Requirements and T&EMeasuresInteroperabilityCapability eDefineCybersecuritySW/System Assurance PPP 3.x.xSW Assurance Measure #1RMFRMF Contol Measure #1Vulnerability AssessVul Assess Measure #1Interop/Exploitable Vuln.Vul Assess Measure #2SW Dev AssessCont AssessSW Dev Asses SW Dev AssessCont Assess Cont Assess Cont AssessBlue TeamBlue TeamRed TeamRed TeamReliabilityResourcesSchedule4.x.x.1Technical Measure #11M-demo#1IT#5IT#2T&E WIPTshouldto help4.x.x.3TechnicalMeasure #13 engage SMEs in a core teamIT#2M-demo#24.x.x.4 developTechnical Measure #14Cyber Evaluation Framework!Reliability Cap #2M-demo#2IT#2Reliability Cap #1 2014 The MITRE Corporation. All rights reserved.4.x.x.2Technical Measure #12M-demo#1Pending Public ReleaseIT#520
Cyber Evaluation Framework Expands on DEF’s“Security” Decision Support QuestionsCyberTechnical Capability/Evaluation ActivityCategoriesDT Objectives - Cyber TechnicalCapabilitiesIs the system and software developedsecurely?Systems and SoftwareAssuranceSoftware Vulnerabilities Mitigated in criticalcomponentsProgram Protection Plan (PPP) Table 5.3.3. ExampleSoftware Metrics include:Number/Category outstanding SDRs% Code Static Analysis Planned/Inspected% Code Planned/Inspected%SW LOC Planned/Inspected CVE%SW LOC Planned/Inspected CAPEC%SW LOC Planned/Inspected CWE%SW LOC Planned/Pen Tested%SW LOC Tested (Coverage)PPP Table 5.3.3 Example Operational System Metricsfor CPI, Critical Functions, Developmental SW andCOTS/NDI include:Fault Isolation Planned/ImplementedLeast Privilege Planned/ImplementedSystem Element Isolation Planned/ImplementedInput Checking/Validation Planned/ImplementedSW Load Key (Signed) Planned/ImplementedPPP Table 5.3.3 Example Development EnvironmentMetrics based upon SW Products selected includingCompiler, Automated Testing Tools, ConfigurationManagement System, Test Results Database, etc.Software Vulnerabilities Mitigated inOperational SystemSoftware Vulnerabilities Mitigated inDevelopment EnvironmentRMF Controls and AttackSurface StandardsVerificationAnti-Tamper Vulnerbailties MitigatedPPP Table 5.3.3, PPP Section 5.3.1 and/or Appendix D:Anti-tamper Plan.Metrics derived for appropriate CPI, CriticalComponentsSupply Chain Risks MitigatedPPP Section 5.3.4 Supply Chain Risk Management(SCRM)Metrics derived from SCRM V&V Plan for appropriateCPI, Critical Components etc.RMF Control Categories include:Access ControlAwareness and TrainingAudit and AccountabilityConfiguration ManagementContingency PlanningIdentification and AuthenticationIncident ResponseMedia ProtectionMaintenancePhysical and Environmental ProtectionPlanningSecurity Assessment and AuthorizationPersonnel SecurityRisk AssessmentSystem and Services AcquisitionSystem and Communications ProtectionSystem and Information IntegrityProgram ManagementAttack surfaces to be evaluated based onStep 2 analysis. Potential Attack Surfacesinclude:Connecting systems explicitly identified inCybersecurity StrategyRF Interfaces (Data Links, Wi-Fi, Bluetooth)SCADA Interfaces (Control Net, Device Net,Fieldbus, Zig Bee, etc.)Cyber Kill Chain Vulnerability Cyber Kill Chain Vulnerability and SystemAssessmentInteroperability and functionality inresponse to exploited cyber vulnerabilitiesshall be evaluated in operational scenarios:Operational scenarios and critical missionsshould be based on authoritative sourcesincluding CONOPS and capabilitiesdocuments. Representative cyber threatsshould be developed based upon STARsand cyber attack scenarios developed byvulnerability assessment teams andapproved by appropriate authoritativesource. Cyber kill chain as exercised by theadversary includes the following steps:Reconnaissance, Weaponization, Delivery,Exploit, Control, Execute, Maintain. CyberDefense in response to adversarial actionsinclude actions to redirect, obviate, Impede,detect, limit, and expose adversarialactions. The lexicon reference is IntendedSystem interoperability andEffects of Cyber Resiliency Techniques onfunctionality in response toAdversary Activitiesexploited cybervulnerabilities 2014 The MITRE Corporation. All rights reserved.Pending Public ReleaseDoes the system satisfybaseline Cybersecurity/IAtechnical standards?Do exposedvulnerabilitiesadversely effectsystem resiliency?Is the systemm ission capableand interoperableand able to sustaincritical m issions inresponse toexploited cybervulnerabilities?Test Activity / DataSourceContractor T&E/FunctionalQualification Testing(FQT)/ GovernmentST&EPPP, CDRLs from CTRand government.Contractor T&E/FunctionalQualification Testing(FQT)/ GovernmentST&EPPP, CDRLs from CTRand government.Contractor T&E/FunctionalQualification Testing(FQT)/ GovernmentST&EPPP, CDRLs from CTRand , CDRLs from CTRand government.Supply Chain RiskManagement/ReportsPPP, CDRLs from CTRand government.RMF Metrics and measures canbe derived from several sourcedocuments includingCapabilities Documents, PPP,Cybersecurity Strategy, SecurityControls Assessment Plan,Performance Specifications etc.Example metrics by controlcategory may include:% of controls verified# and Category Deficiencies% of inherited controls verified# and Category InheritedDeficienciesST&E/ SecurityControls Assessor/Step 3/4 VulnerabilityAssessmentMetrics and measures can bedeveloped from DIACAP/RMF andtechnical standards appropriatefor the exposed Attack Surface.ST&E/ SecurityControls Assessor/Step3 VulnerabilityAssessment,Contractor ST&E andGovernment TechnicalStandards Testing asappropriateITT will develop measures in collaboration withother program stakeholders. Critical Missionsmay be derived from CONOPS, CapabilitiesDocuments, PPP, etc. Interoperability metricsand measures should be derived from the NRKPP. Metrics include:- Support to military operations- Enter and be managed in the network- Exchange information- Support net-centric military operations.Sources for cyber security metrics and measuresmay be derived from program technicaldocumentation, or other authoritative sourcesincluding the DoD Strategy for Operating inCyberspace and Resilient Military Systems CyberThreat Defense Science Board Task Force. Thebelow measures are derived from MP 120053,Rev 1, Cyber Resiliency Metrics, dated Apr 2012.Additional metrics will be selected by the ITT incollaboration with other Stakeholders. Initialplanned metrics include:- % cyber resources properly configured(Configuration varies by resource)- % attempted intrusions stopped at networkperimeter/ deflected- % mission-essential capabilities for whichmultiple instantiations available- Avg Length of time between initial disruptionand restoration- Quality of restored data- Quality of choices made during design andengineering that affect resiliency- % mission-essential datasets for which allitems effectively have two or more independentexternal data feeds- % mission-essential data stores for which amaster copy exists- % data value assertions in a mission-essentialdata store for which a master copy existsStep 3 VulnerabilityAssessment: Team hasfull knowledge andaccess to the Systemand all supportingcomponents (BlueTeam)Step 4 VulnerabilityAssessment: Teamfunctions as anadversary withoutknowledge or access tothe system (Red Team)21
22 Example Cyber Evaluation Framework Decision SupportQuestions and Evaluated Cyber Capabilities Is the system and software developed securely?– Software Vulnerabilities Mitigated in critical components– Software Vulnerabilities Mitigated in Operational System– Software Vulnerabilities Mitigated in Dev. Environment– Anti-Tamper Vulnerabilities MitigatedProgram Protection Plan– Supply Chain Risks Mitigated Does the system and associated Attack Surfaces & Interfaces satisfybaseline Cybersecurity technical standards?– RMF Controls Verification– RMF Interfaces Verified– Other Attack Surfaces Verified (Based on Phase 2 analysis)RMF– Examples: GPS, Data Links, Wi-Fi, Bluetooth, ICS, SCADA Interfaces 2014 The MITRE Corporation. All rights reserved.Pending Public Release
Cyber Evaluation Framework Decision SupportQuestions and Cyber Capabilities Evaluated Does Baseline Performance support Critical Missions and are exposedvulnerabilities identified and closed?– Exercise Critical Missions Derived from CONOPS, Capabilities Documents, PPP, etc.– Identify Number and Severity of Exposed Vulnerabilities Is the system mission capable, interoperable and resilient in response toexploited cyber vulnerabilities?– Evaluate mission performance in context of Cyber Attack LifecycleMITRE: Cyber Attack Lifecycle 2014 The MITRE Corporation. All rights reserved.Public Release Pending 23
24“Simple” Example: Comprehensive ExperimentalAnalyses of Automotive Attack Surfaces Modern automobiles pervasivelycomputerized– Engine, Transmission, Body,Airbag, Antilock Brakes, HVAC,Keyless Entry Control, etc. Attack surface extensive– Telematics: Blue Tooth, Cellular,Wi-Fi, Keyless Entry Attack Surface easily exploited– OBD Diagnostics, CD players,Bluetooth Example:– Cellular radio/ Wi-Fi exploitspermit .– Long distance vehicle control,location tracking, in-cabin audioexfiltration 2014 The MITRE Corporation. All rights reserved.Pending Public ReleaseAug 2011: Comprehensive Experimental Analyses of AutomotiveAttack SurfacesSource: University of California, San Diego, University of Washington
Example Phase 1:Understanding Cybersecurity Requirements/DevelopT&E ApproachUrban Assault Vehicle Graphic Sources: WIKIPEDIA Commons 25 Example Requirements ResourcesCONOPSCapabilities DocumentsInformation Support PlanSystems Requirements DocumentsProgram Protection PlanCybersecurity StrategyRMF PackagesContract Specs/TechnicalRequirements DocumentsPlan Cybersecurity T&E to Engage with SE Team EarlyEngage with SE/SSE Activities/ProcessesRequirements Reviews, Contracting, SETRs etc.Plan Verification DT&E to close Attack SurfaceConduct “Kill Chain Vulnerability Assessments” (BlueTeam and Red Team) to evaluate missionperformanceVerify Production Readiness at MS COT&E post MS C 2014 The MITRE Corporation. All rights reserved.Pending Public ReleaseMITRE: Cyber Attack Lifecycle
26 Example Phase 2:Characterize the Attack SurfaceStakeholders Identify VehicleAttack Surface1. Vehicle to Vehicle Comms2. Telematics3. Keyless Entry4. OBD II5. Radio6. Anti TheftUrban Assault Vehicle Attack SurfaceRefine T&E Strategy to Understand All systems interfaces Likelihood of attack? What happens if/when exploited? Approach to close/mitigatevulnerabilities Adequacy of Cybersecurity T&EApproachAug 2011: Comprehensive Experimental Analyses of Automotive Attack SurfacesSource: University of California, San Diego, University of WashingtonPPP Criticality Analysis 2014 The MITRE Corporation. All rights reserved.Pending Public Release
27 Example Phase 3:Vulnerability IdentificationUrban Assault Vehicle Attack SurfaceVehicle Attack Surface1. Deny Vehicle/Vehicle Comms2. Intercept Telematics3. Clone Keyless Entry4. Corrupt OBD-II5. Monitor Radio6. Disable Anti-TheftT&E Activities Verify/Exercise Critical Missions Cooperative “Kill Chain VulnerabilityAssessments” (Blue Team) ID potential exploits, exposedvulnerabilities/mission impactAug 2011: Comprehensive Experimental Analyses of AutomotiveAttack SurfacesSource: University of California, San Diego, University of WashingtonCyber Attack LifecycleVehicle SV-6 Systems Data Exchange Requirements 2014 The MITRE Corporation. All rights reserved.Threat Based TestingPending Public Release
28 Example Phase 4:Adversarial Cybersecurity DT&EExercise Critical Missions1. Tx/RX Vehicle/Vehicle Comms2. Cellular Phone Calls3. Use Keyless Entry4. Upload/Download OBD II Data5. Tune Radio6. Anti TheftUrban Assault Vehicle Autobahn MissionT&E Actions Verify/Exercise Critical Missions Adversarial “Kill Chain VulnerabilityAssessments” (Red Team) ID exposed vulnerabilities/missionimpact Develop DT&E AssessmentCyber Attack Lifecycle 2014 The MITRE Corporation. All rights reserved.Graphic Sources: WIKIPEDIA CommonsGraphic Sources: WIKIPEDIA CommonsPending Public Release
29 Where are we going? DASD DT&E and DASD SE – High level engagement ongoing between principals– DT&E Staff Specialist are reviewing PPPs as they surface forreviewDASD DT&E and OSD DOT&E– Working to update DAG, DAU Course Material etc.DASD DT&E direct “Program Engagement”– DT&E Staff Specialists are leading core teams to assist PMs– Significant insight gainedDASD DT&E Cybersecurity “Process Improvement”– Cybersecurity Pilot being executed in collaboration with NAVAIRTRMC JMETC provides distributed Cyber T&E capabilities– National Cyber Range 2014 The MITRE Corporation. All rights reserved.Pending Public Release
30 Closing DASD DT&E, SE, and OSD OT&E are collaborating to improve acquisition outcomes– Current policy and procedures are being updatedSystems Security Engineering (SSE), RMF and Cybersecurity T&Eprocesses must be aligned and mutually supportive– T&E Community must engage early to influence SSE process– T&E must provide feedback in a timely manner to key stakeholders for“assessment and mitigation”– Early feedback will positively impact cost schedule and performance!Cybersecurity T&E is not “Controls Compliance”– Evaluates planned and implemented Cybersecurity Measures T&E can help verify baseline security requirements– Evaluates exposed “Attack Surface” Identify exposed Vulnerabilities– Focuses on critical operational missions Evaluate system resilience 2014 The MITRE Corporation. All rights reserved.Pending Public Release
Prepared for 2014 NDIA T&E Conference . Mr. Pete Christensen . pchris@mitre.org. 703-983-2516 . With support from . Ms. Jean Petty . jpetty@mitre.org 703-983-9269 . Special Thanks to . . Cybersecurity Strategy/RMF Artifacts and "Verify" the system as planned/built - Utilize cybersecurity SMEs to assist - Review Technical Requirements .
