Transcription
CYBER TEST ANDEVALUATION IN THEUS AIR FORCEJoseph Nichols, PhDTechnical Advisor for Flight Test and EvaluationAir Force Test CenterEdwards AFB CAjoseph.nichols.13@us.af.milDistribution A: Approved for public release; distribution unlimited. 412TW-PA-17133
WHAT IS CYBER TEST AND EVALUATION?2
DIACAP Compliance CheckAre you compliant with thesecontrols?YesNoRisk Management FrameworkAre you compliant with these controls?YesNoWhat is the Risk?Vulnerability level (includes STIG findings)What is the vulnerability level(Severity Category/code) ?CAT I FindingAssociated ThreatsLikelihood of ExploitationImpact level (CIA)Compensating Controls and MitigationsProcessCompleteWhat is the Residual Risk? What is my organization’srisk tolerance? What is my risk tolerance?Risk AcceptedCERTIFICATION & ACCREDITATIONEVOLUTION3
Risk ManagementFrameworkCyber TestRequirements Compliance withcybersecuritypolicy (DoDD8510.01) Risk mitigationand acceptance Issuance of IATTor ATO SpecificationCompliance Mission suitability SurvivabilityCERTIFICATION AND ACCREDITATIONVERSUS TEST & EVALUATION4
AirworthinessRequirementsFlight TestRequirements Compliance withdesign criteria(MIL-HDBK-516) Risk mitigationand acceptance Issuance of TypeCertificate or MFR SpecificationCompliance Mission suitability SurvivabilityAIRWORTHINESS ANALOGY5
WHAT IS A CYBER SYSTEM?6
MILITARY AND INDUSTRIAL SYSTEMS AREDIFFERENT FROM STANDARD COMPUTERNETWORKS7
Traditional IndustrialITInfrastructure PlatformsCYBERSPACE TAXONOMY8
FAC-A/ISRC2ISRDCGSUHFLink 16MissionPlanningAOCMDLOFPLoaderJTACAF CYBER BOUNDARY FRAMEWORK9
RFWHEEL OF ACCESS10
WHAT IS THE CYBER TEST PROCESS?11
DOD CYBERSECURITY TEST ANDEVALUATION GUIDEBOOK12
Likelihood of 5L-1L-2L-3L-4L-5L-1L-2L-3L-4L-4L-1L-1L-2L-3L-3M-1 M-2 M-3 M-4 M-5Likelihood of Attack SuccessThreatAssessmentsLikelihood ofLossRisk AssessmentVulnerabilityAssessmentsWithin risktolerance?Overall Risk Factor MatrixL-5L-4L-3L-2XL-1I-1Consequenceof LossSystem SeverityImpactTEST54321Likelihood L-1Impact I-5Risk ModerateLikelihood5 - Near Certainty4 - Probable3 - Occasional2 - Remote1 - ImprobableLIKELIHOODRisk -4Mission CriticalityI-5I-5I-5I-4I-3C-554321Impact5 - Catastrophic4 - Major3 - Moderate2 - Minor1 - NegligibleI-2I-3I-4I-5IMPACTDerived from:NIST 800-30 Risk AssessmentDOD Risk Management GuideCriticalityAnalysisCYBERSECURITY ASSESSMENTMETHODOLOGY13
Mission Critical VulnerabilitiesDetermine MissionThreadExecute Cyber TestTechniques / ProceduresDetermine RelatedAttack SurfacesProvide Survivability / Mission Impact AssessmentMISSION THREAD ANALYSISMETHODOLOGY14
WHAT IS THE STATUS OF CYBERTEST & EVALUATION IN THE USAF?15
Continuingto execute DT/OT on traditional IT and weapons platformsfor developmental systems and legacy platforms ExecutingFY16 NDAA 1647 Cyber Assessments Standup of Cyber Resiliency Office for Weapon Systems in 2016 Standup Cyber Test Group/Squadrons in 2017 Beginning Growingdevelopment of the Avionics Cyber RangeUSAF cyber test manpower AFOTEC sponsoring cooperative/adversarial assessment teams Major government push to hire/retain certified/experienced cyber testersMAJOR LINES OF EFFORT16
DODdirection to conduct a cybersecurity evaluationof all major US weapon systems Testingmust be completed by Dec 2019 Combined Planningvulnerability identification phasecombined DT/OT testingFY16 NDAA SECTION 164717
CYBER RESILIENCYOFFICE FOR WEAPONSYSTEMS (CROWS)18
Boeing FieldJB Lewis-McChordHanscom AFBBeale AFBHill AFBWright-Patterson AFBWhite Oak, MDCrystal City, VAMoffett FieldWright-Patterson AFBDenver COCreech AFBNellis AFBLangley AFBAEDCPeterson AFBHQ AFTCHQ AFOTECNAS Patuxent RiverArnold AFBCharleston AFB412 TWEdwards AFBWhite SandsMissile RangeKirtland AFBHolloman AFBTucson, AZTinker AFBGunter AnnexGreenville, TXWarner-Robins AFBHurlburt FieldAF Plant 04Melbourne, FLWaco, TX96 TWLackland AFBUSAF TEST INFRASTRUCTUREEglin AFB19
DoD test facility capable of conductingcyber testing compatible with the uniquefeatures of aircraft avionics and airbornemunitions Center of Excellence for avionics cyber T&Eand developer of cyber test techniquesand test tools Connected with the NCR and other aircraftand weapons cyber test facilitiesAVIONICS CYBER TESTINFRASTRUCTURE20
AFOTEC has three cyber missions for DoD and Air ForcePrimary is testing systems in a realistic operational environment Mission assurance and resiliency of systems are test objectives Providing cyber mission assessments of PACOM, NORTHCOM/NORAD and asexecutive agent for Air Force for DOT&E Authorizing official for operations test infrastructure Teamingwith AFTC to provide support for vulnerability analysis, “bluebooks”, and integrated cyber testing Workingoperational cyber range requirements through AFTC IncreasingNSA certified Adversary Assessment Team capacity withMOUs with Kansas ANG, Air Warfare Center, and New Mexico ANG Totalforce initiatives—with both AFTC and ANG Readiness CenterOPERATIONAL TEST INITIATIVES21
CyberT&E expertise for aircraft and weapons requiresa merge of traditional avionics test expertise andcomputer network penetration expertise Sendingavionics test engineers to cyber training DevelopingDOD cyber training courses Standingup new test organization dedicated to allaspects of cyber test and evaluation – networks,aircraft, weaponsMANPOWER REQUIREMENTS22
23
Cyber T&E expertise for aircraft and weapons requires a merge of traditional avionics test expertise and computer network penetration expertise Sending avionics test engineers to cyber training Developing DOD cyber training courses Standing up new test organization dedicated to all aspects of cyber test and evaluation -networks,
