WIXLIB

IntroductionCyber Intelligence Definition and Analytic FrameworkCyber intelligence grew from the halls of government into a burgeoningbusiness providing tools and services to industry and academia. As moreorganizations focus on this topic, varying methodologies, technologies,processes, and training complicate the operating environment.Recognizing a need to understand and improve this situation, the SEIInnovation Center began to study the state of the practice in cyberintelligence in June 2012. This report discusses the CITP’s process andkey findings.The SEI Innovation Center developed a definition of cyber intelligenceto standardize the scope of the CITP with participants. Drawn fromgovernment and industry descriptions, the SEI Innovation Center definescyber intelligence as:ParticipantsThe CITP involved 26 organizations from government, industry, andacademia. They included six government agencies with dedicated cyberintelligence missions and 20 entities representing multiple economicsectors, such as academia, defense contracting, energy, financialservices, healthcare, information technology, intelligence serviceproviders, legal, and retail. These organizations range in size from oneemployee to global organizations with hundreds of thousands of networkusers. Their cyber intelligence workforces have diverse backgrounds inintelligence, information security, and the military, and hold a multitude oftitles, such as chief technology officer, chief information security officer,vice president of threat management, information architect, intelligenceanalyst, and network analyst.The acquisition and analysis of information to identify, track, and predict cybercapabilities, intentions, and activities that offer courses of action to enhancedecision making.An analytic framework also was created to guide the CITP’s baselineand benchmark processes, the foundation of which is based on the U.S.government’s traditional intelligence rocessingAnalysisandProductionFigure 1 – Traditional Intelligence Cycle 2The CITP’s analytic framework promptly deviates from the precedingbecause the utility of the traditional intelligence cycle is limited whenapplied to cyber. This traditional intelligence cycle is depicted as alinear process and does not emphasize the inter-related nature of its fivefunctions or their relevance to related functions, namely cyber security.The SEI Innovation Center captured these unique cyber intelligenceanalysis characteristics by creating an approach that more accuratelyshows the inter-dependencies and outside influences in the cyberintelligence process. This approach incorporates how technologyinfluences the way analysis is done, and uniquely identifies the functionsthat integrate technology. In particular, the CITP’s analytic frameworkseparates analysis into two distinct functions: specialized technicalanalysis (i.e. functional) and strategic analysis.22 he Traditional Intelligence Cycle was reproduced from a paper authored byTJudith Meister Johnston and Rob Johnston, hosted on the Central IntelligenceAgency’s public website: y/page 46.pdf. Last accessed January, 2013.

This analytic framework utilizes five functions to capture interdependencies of and external influences on cyber DataGatheringFigure 2 – CITP Analytic Framework Environment: Establishes the scope of the cyber intelligence effortand influences what data is needed to accomplish it. Data Gathering: Through automated and labor-intensive means, analystsexplore data sources, collect information, and aggregate it to performanalysis. Functional Analysis: Analysts use gathered data to perform technicaland tailored analysis, typically in support of a cyber security mission. Strategic Analysis: Analysts apply a strategic lens to functional dataand report this intelligence to a stakeholder or use it to influence theenvironment. If functional analysis attempts to answer the “what”and “how” of cyber threats, strategic analysis aims to answer“who” and “why.” Stakeholder Reporting and Feedback: After intelligence is disseminatedto stakeholders, they provide feedback and/or use the intelligence toinfluence the environment.It is important to note that the analytic framework does not solely existto address cyber security. Cyber intelligence is a critical component ofcyber security, and the two functions are inter-related; however, the CITPfocuses on cyber intelligence. Cyber intelligence supports a variety ofmissions in government, industry, and academia; to include national policy,military applications, strategic communications, international negotiations,acquisitions, risk management, and physical security. Throughout theanalytic framework, cyber security professionals receive data andintelligence, but the cyber intelligence process operates independentlyand does not necessarily need to support a cyber security mission.Baseline and Benchmarking ApproachThe SEI Innovation Center employed an iterative process to create adiscussion guide that served as a starting point to baseline organizations.It reduced biases and was specifically designed to capture entities’ corecyber intelligence functions, regardless of if they were representing thegovernment, industry, or academia. Using the discussion guide, the SEIInnovation Center typically sent a cross-functional team of intelligence andsoftware engineering professionals to engage with organizations duringface-to-face interview sessions. The team interacted with representativesfrom their cyber intelligence and cyber security leadership as well asfunctional and strategic analysts. During the interview sessions, theseentities provided information on the methodologies, technologies,processes, and training enabling them to perform cyber intelligence.The data gathered during these interviews established the baseline thatthe SEI Innovation Center used to benchmark against its cyber intelligenceanalytic framework. For benchmarking, the SEI Innovation Center compiledand reviewed the baseline to ensure it captured the pertinent data. Theinformation then was ranked against 35 assessment factors distributedamongst the analytic framework’s five functions using an ordinal scale of , , 0, -, --, with 0 representing average performance. Due to the varietyin the organizations’ backgrounds and sizes, the ordinal scale offeredthe necessary flexibility for benchmarking, despite its limitations withnumerical and interval analysis. Peer and group reviews also ensuredconsistency throughout the rankings.The SEI Innovation Center derived the 35 assessment factors from theinterview sessions and its cyber intelligence and software engineeringexpertise: Environment: Top-sight on cyber footprint; cyber intelligence distinctionwith cyber security; role alignment; personnel to support cyberintelligence; organizational structure; workflow utilization; prioritizationof threats; organizational situational awareness; cyber intelligencefunctional and strategic analysis; scope of past, present, and futureanalysis; insider threat and cyber intelligence relationship. Data Gathering: Requirements and sources relationship; informationsharing; meeting analytical needs; technology facilitating datagathering; indexing and archiving of data; validation of sources. Functional Analysis: Workflow exists; timeliness in producinganalysis; diversity with incorporating multiple technical disciplines;skills, knowledge, and abilities; tools utilized. Strategic Analysis: Distinguished from functional analysis; workflowexists; diversity with incorporating multiple technical disciplines;skills, knowledge, and abilities; tools utilized. Stakeholder Reporting and Feedback: Report types generated;reporting mechanism for actionable and predictive analysis;leadership influences format and production timelines; cyberintelligence influences decision making; feedback mechanismsexist; feedback influences data gathering and analysis; satisfyingintelligence consumers; capturing return on investment.3

Key FindingsThe following highlights the common challenges and best practicesidentified during the CITP by describing them within the context of theanalytic framework’s five functions. A stacked bar chart accompanies eachfunction to summarize the baseline of organizations’ ratings in these areas.Each bar within the charts represents one of the benchmark’s 35 factors(X-axis). The height of each color within the bars shows the percentage oforganizations (Y-axis) receiving that particular rating and the red-coloreddiamond symbol displays the median. The ratings range between --, -, 0, ,and , with 0 being average performance for that assessment factor.Figure 3 – CITP BaselineFigure 4 – CITP Baseline Variances4Figure 3 divides a stacked bar chart by the five functions of the analyticframework to visually show the CITP’s baseline. Figure 4 removes themedian (the red-colored diamond symbol) and the yellow-colored barsections depicting the percentage of organizations receiving an averagerating in Figure 3 to highlight the variances among entities with ratings of--, -, , and . Figures 6, 9, and 11-13 display a stacked bar chart for thefactors within each of the five functions.

State of the Practice in Cyber IntelligenceMost organizations identified cyber intelligence and cyber security astwo distinct and capable work functions that interact when necessary tobest support their needs. They performed cyber intelligence by trying tounderstand the internal and external environment, gathering data, andanalyzing technical threats, ranging from malware to email phishing.However, their intelligence reporting generally did not include strategicanalysis or adequately inform stakeholders—especially decision makers—limiting its impact beyond the realm of cyber security. This exhibits anendemic problem of functional analysts not effectively communicating withnon-technical audiences. It also demonstrates organizations’ reluctanceto share information within their own entities, industries, and acrosseconomic nalAnalysisDataGatheringChallenge: Applying a strategic lens to cyber intelligenceanalysisFigure 5 – Cyber Security-Centric Analytic FrameworkDespite having a wealth of data available, many organizations struggle withmoving beyond the functional analysis of low-level network data to incorporatestrategic analysis of threats and threat indicators.Challenge: Information sharing isn’t bad; it’s brokenCurrent state: Most organizations had difficulty incorporating strategicintelligence analysis into existing security-focused processes.Correspondingly, entities with poor or no strategic analysisfunctions struggled with communicating security requirementsto leadership, had a more reactionary network security posture,and were less likely to anticipate or be prepared for emergingcyber threats. This can be attributed to an organization-wide lackof support for strategic analysis, demonstrated by organizationsnot having the resources to index and store data for strategicanalysis, perform trend analysis, or look at individual networkevents in a more strategic context. Some organizations cannotobtain resources to address these issues because of an inabilityto effectively communicate the complexities of cyber intelligenceto non-technical decision makers and relate its importanceto the organization’s overarching goals and objectives. Thus,decision makers do not grasp the benefits of investing in tools andpersonnel, and cyber intelligence efforts suffer. Organizations generally had a mature cyber intelligence workflowthat incorporated functional analysis, but only as a means tosupport cyber security (see Figure 5). The challenge withinthis version of the analytic framework is communicating theimportance and relevance of technical issues to stakeholdersin compelling terms they understand. Although cyber securitybenefits from functional analysis, the CITP’s findings indicate thatthe addition of strategic analysis to the analytic framework is themost effective means of bridging the communication gap betweencyber security and non-technical decision makers.The highest performing organizations actively share—not just consume—data informal and informal information sharing arrangements.Current state: Government organizations in the CITP demonstrated excellentinternal information sharing practices. Many codified processes thatrequire internally distributing artifacts to other departments, suchas draft analytical products, network security data, and indicationsand warnings information. However, they consistently cited accessto data from external organizations as a challenge. Organizationalculture is the largest road block to success in this space, as maturetechnology solutions are available to overcome classification andneed-to-know restrictions on information sharing. Information sharing for the organizations in industry and academiavaried significantly. They generally failed to share data in ameaningful way, resulting in a reactive, patch-and-remediate cybersecurity posture. Similar to those in government, the most significantbarrier to external information sharing in industry and academia iscultural; organizations remain reluctant to share “sensitive” networkdata and intelligence indicators with competitors. Conversely,entities that overcome this reluctance and routinely provide andconsume data identified these practices as a major reason for theirability to stay ahead of cyber threats. Examples of data being sharedinclude indicators of malicious activity, draft analytical reports, andcontextual data surrounding malware and bad IP addresses.5

State of the Practice in Cyber Intelligence, continuedBest Practice #2: Information sharing inthe financial sector Initiatives sponsored by government and industry attempt to facilitateinformation sharing, but with limited success. Industry-sponsoredinformation sharing generally is open only for select audiences andrequires a financial commitment to join. Many of the governmentsponsored arrangements tend to be redundant; they report the samedata, but in different formats (one agency reports in .PDF, anotherin XML, another through RSS feeds), and with a range in timeliness.Information sharing relationships with the government also have theperception of being a “reporting” mechanism, which has dissuadedorganizations from being more engaged.Financial sector organizations exhibit the strongest information sharingculture, processes, and mechanisms. Internally, they have formalcommunication channels between cyber security experts, analysts, andthe various business divisions within their organizations. Analysts producea range of intelligence products, each one designed to meet the needs ofinternal stakeholders; from strategic summaries for executive leadershipto organization-wide products educating the workforce on pertinentcyber threats. Strategic cyber intelligence analysts also work closely withfunctional analysts to understand the scope and nature of cyber threats,which better allows them to communicate risks and impacts to internalbusiness operations.Best Practice #1: Aligning functional and strategic cyberintelligence resourcesHigh performing cyber intelligence programs employ a mix of functionaland strategic analysts. For three organizations in the CITP, one governmentand two commercial, functional analysts were physically co-located withstrategic analysts. Cyber intelligence is too big a topic for any one personto cover adequately. The nuances of technology, the intricacies of networkdefense, and the complexity of adversary intentions and capabilities makesit difficult for any one person to fully understand the cyber landscape. Forthis reason, successful cyber intelligence programs adopt a collaborativeculture, so that experts can interact and share ideas.Organizations that adopt this best practice are able to generate timelyintelligence products, better communicate technical issues to seniorleadership, and adjust data gathering tools to meet analysts’ needs moreefficiently. The close interaction between functional and strategic analystsallows them to more effectively understand complex technical details. This,in turn, provides analysts a better understanding of the threats and risks,benefitting their ability to communicate these concepts to leadership. TheSEI Innovation Center observed that organizations not employing this bestpractice incurred delays in reporting due to lags in collaboration eitherby email or phone calls. Other alternatives included paying to collaboratewith third-party intelligence providers that offered technical expertise, orengaging in an online collaboration portal where participant expertise wasdifficult to verify.Analysts also benefit from being co-located with their counterpartsbecause it enables them to seamlessly communicate data gatheringrequirements to the people that have access to the collection tools.Functional analysts typically have the ability to adjust data gatheringtools or resources so that others can receive the data that they need.One organization in the CITP had strategic analysts sitting next to theirfunctional counterparts responsible for a unique data-gathering tool. Asthe strategic analysts received new requirements, or wanted to pursueinteresting data, they asked the functional analysts to collect this data, andreceived it almost instantly.6Externally, these organizations are very active, benefitting from theirinvolvement with the Financial Sector Information Sharing and AnalysisCenter (FS-ISAC). The financial services organizations in the CITPunanimously agreed that FS-ISAC indications and warnings directlyenhance their network security. Additionally, the FS-ISAC facilitatesnumerous analytical exchanges, allowing participants to better understandthe capabilities and techniques of cyber actors targeting the financialsector. It also fosters informal collaboration among members, despite thesector’s overarching competitive environment.

EnvironmentFigure 6 – Environment – CITP BaselineUnderstanding internal and external environments allows organizationsto establish the scope of their cyber intelligence effort. The internalenvironment usually consists of determining where the cyber intelligenceprogram should exist and how to allocate resources. In some instances,aligning functional and strategic analysis efforts according to threatprioritization models aided resource allocation. The internal environmentalso includes studying participant’s global cyber presence, whatinfrastructure is accessible through the Internet, and how to identify whatdata needs to be collected to maintain network situational awareness.Externally, the environment involves knowing the entities capable ofaffecting organizations’ networks by focusing on system vulnerabilities,intrusion or network attack vectors, and the tactics, techniques,procedures, and tools used by relevant threat actors. It tends not to gaugethe threat emanating from software supply chains, but in certain casesdoes track external factors affecting organizations’ different businessunits using open source monitoring. By investing the time and energyto define the environment, organizations can significantly improve theirdata gathering efforts, resulting in more efficient and effective cyberintelligence programs.Challenge: Understanding threats to the softwaresupply chainThe unknown provenance of software complicates the ability todefine the cyber environment.Current state: Software development is a critical component of the networked world.Businesses, government, and individuals completely rely on software toperform daily tasks. Error-free and reliable software is a necessity forsoftware found in commercial enterprises, industrial control systems,and military technology. When buying software, or having it codedfor a specific purpose, these customers gener

from their cyber intelligence and cyber security leadership as well as functional and strategic analysts. During the interview sessions, these entities provided information on the methodologies, technologies, processes, and training enabling them to perform cyber intelligence. The data gathered during these interviews established the baseline that