WIXLIB

Logistic Regression – ClassificationIdea: Classification using a discriminative model Predict future behavior based on existing labeled data Draws a line to assign labelsMainly used for binary classification: either “red” or “blue”Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited38

Logistic Regression – ClassificationLook at distribution, what’s likely based on current dataProbably blueProbably redData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited39

Logistic RegressionThree dimensions,same conceptHUNDREDS ofdimensions,same conceptData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited40

Classification: Support Vector MachineBarometric PressureIdea: The optimal classifier is the one that is the farthest from bothclassesDew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited41

Classification: Support Vector MachineBarometric PressureIdea: The optimal classifier is the one that is the farthest from bothclassesDew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited42

Classification: Support Vector MachineAlgorithm: Find lines like beforeBarometric Pressure Assign a cost to misclassified data points based on distancefrom the classification line𝝃𝟐𝝃𝟏Dew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited43

Classification: Decision TreesIdea: Instead of drawing a single complicated line through the data,draw many simpler lines.Algorithm: Scan through all values of all features to find the one that “helpsthe most” to determine what data gets what label.Barometric Pressure Divide the data based on that value, and then repeat recursivelyon each part.Dew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited44

Classification: Decision TreesIdea: Instead of drawing a single complicated line through the data,draw many simpler lines.Algorithm: Scan through all values of all features to find the one that “helpsthe most” to determine what data gets what label. Divide the data based on that value, and then repeat recursivelyon each part.Barometric Pressure𝐷𝑃 60Dew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited45

Classification: Decision TreesIdea: Instead of drawing a single complicated line through the data,draw many simpler lines.Algorithm: Scan through all values of all features to find the one that “helpsthe most” to determine what data gets what label (“informationgain”).Barometric Pressure Divide the data based on that value, and then repeat recursively𝐷𝑃 60on each part.𝐵 760Dew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited46

Classification: Decision TreesBenefits: Works well when small. Very easy to understand!Challenges: Trees overfit easily Very sensitive to data; Random ForestsBarometric Pressure𝐷𝑃 60𝐵 760𝐷𝑃 55Dew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited47

Classification: K-Nearest NeighborsIdea: A new point is likely to share the same label as points aroundit.Algorithm: Pick constant k as number of neighbors to look at.Barometric Pressure For each new point, vote on new label using the k neighborlabels.Dew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited48

Classification: K-Nearest NeighborsIdea: A new point is likely to share the same label as points aroundit.Algorithm: Pick constant k as number of neighbors to look at.Barometric Pressure For each new point, vote on new label using the k neighborlabels.Dew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited49

Classification: K-Nearest NeighborsIdea: A new point is likely to share the same label as points aroundit.Algorithm: Pick constant k as number of neighbors to look at.Barometric Pressure For each new point, vote on new label using the k neighborlabels.Dew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited50

Classification: K-Nearest NeighborsWorks well when there is a good distance metric and weighting function to voteon classificationChallenges: Not a smooth classifier; points near each other may getclassified differently Must search all your data every time you want to classify a newpoint When k is small (1,2,3,4), essentially it is overfitting to the datapointsData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited51

Clustering Unsupervised learning Structure of un-labled data Organize records into groups based on some similarity measure Cluster is the collection of records which are similarData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited52

Clustering: K-meansIdea: Find the clusters by minimizing distances of cluster centers to data.Algorithm: Instantiate k distinct random guesses 𝜇- of the cluster centers Each data point classifies itself as the 𝜇- it is closest to it Each 𝜇- finds the centroid of the points that were closest to it andjumps thereBarometric Pressure Repeat until centroids don’t moveDew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited53

Clustering: K-meansIdea: Find the clusters by minimizing distances of cluster centers to data.Algorithm: Instantiate k distinct random guesses 𝜇- of the cluster centers Each data point classifies itself as the 𝜇- it is closest to it Each 𝜇- finds the centroid of the points that were closest to it andjumps thereBarometric Pressure Repeat until centroids don’t moveDew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited54

Clustering: K-meansIdea: Find the clusters by minimizing distances of cluster centers to data.Algorithm: Instantiate k distinct random guesses 𝜇- of the cluster centers Each data point classifies itself as the 𝜇- it is closest to it Each 𝜇- finds the centroid of the points that were closest to it andjumps thereBarometric Pressure Repeat until centroids don’t moveDew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited55

Clustering: K-meansIdea: Find the clusters by minimizing distances of cluster centers to data.Algorithm: Instantiate k distinct random guesses 𝜇- of the cluster centers Each data point classifies itself as the 𝜇- it is closest to it Each 𝜇- finds the centroid of the points that were closest to it andjumps thereBarometric Pressure Repeat until centroids don’t moveDew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited56

Clustering: K-meansIdea: Find the clusters by minimizing distances of cluster centers to data.Algorithm: Instantiate k random guesses 𝜇- of the clusters Each data point classifies itself as the 𝜇- it is closest to it Each 𝜇- finds the centroid of the points that were closest to it andjumps thereBarometric Pressure Repeat until centroids don’t moveDew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited57

Clustering: K-meansWorks well when there is a good distance metric between the points the number of clusters is known in advanceChallenges:Barometric Pressure Clusters that overlap or are not separable are difficult to clustercorrectly.Dew PointData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited58

InfluencersGoal: Detect the people who control or distribute informationthrough a network.Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited59

Influencers: Degree CentralityIdea: Influential people have a lot of people watching them.Equation Degree centrality number of directed edges to the node- High degree centrality people are those with large numbers offollowers. If undirected graph, transform to bi-directional and computeData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited60

Influencers: Degree CentralityIdea: Influential people have a lot of people watching them.Equation Degree centrality number of directed edges to the node- High degree centrality people are those with large numbers offollowers. If undirected graph, transform to bi-directional and compute3111213Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University12017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited61

Influencers: Betweenness CentralityIdea: Influential people are “information brokers” who connectdifferent groups of people.Algorithm Find all shortest paths from all nodes to all other nodes in thegraph. Betweenness centrality for a node sum over all start and endnodes of the number of shortest paths in the graph that includethe nodeData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited62

Influencers: Betweenness CentralityIdea: Influential people are “information brokers” who connectdifferent groups of people.Algorithm Find all shortest paths from all nodes to all other nodes in thegraph. Betweenness centrality for a node sum over all start and endnodes of the number of shortest paths in the graph that includethe node33314721211Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University52017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited63

Indicator communitiesBut what if we aren’t startingwith a reference indicator?We assume that indicatorsgenerated by a coherent realworld process will be more likelyto co-occur in tickets thanarbitrary pairs of indicators.Find groups of highly similarindicators in complete indicatorticket graph.Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited64

Indicator-ticket graphA subset of the ticket-indicator graph(for a small set of selected indicators) Tickets are grey triangles Indicators are black circles Edges connect tickets to the indicators theycontainData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited65

Machine Learning Is GrowingPreferred approach for many problems Speech recognition Natural language processing Medical diagnosis Robot control Sensor networks Computer vision Weather prediction Social network analysis AlphaGO, Watson Jeopardy!Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited66

This slide also intentionally left blank, just like the earlier oneData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited67

What we did todayNameMgrDirBeth JonesDan Thomas Anne KimBeth JonesDan Thomas Anne KimJones, BethDan Thomas Anne Kim❶Tom KeaneMark RyanTim PikeLength LineSolved?Comment1:301Y❺ 1:523Y 902Y 882N ❶Average (5:08)Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited68

What we did todayData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited69

Data Science helps youuse data to get results.Data Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited70

Eliezer KanalTechnical Manager(412) 268-5204ekanal@sei.cmu.eduDaniel DeCapriaData Scientist(412) 268-2457djdecapria@sei.cmu.eduData Science TutorialAugust 10, 2017 2017 Carnegie Mellon University2017 SEI Data Science in Cybersecurity SymposiumApproved for Public Release; Distribution is Unlimited71

2017 SEI Data Science in Cybersecurity Symposium Approved for Public Release; Distribution is Unlimited About us Eliezer Kanal Technical Manager, CERT Recent projects: ML-based Malware Classifier Network traffic analysis Cybersecurity questionnaire optimization Daniel DeCapria Data