WIXLIB

What is Zeek? "Network Security Monitor" Inspects all network traffic Generates forensically sound logs See “Network Forensics With Zeek” by Matthias Vallentin Has a scripting language, making it very extensible automation, custom functionality, etc. Crucial difference: IDS is logging "bad" things NSM is logging everything What if you didn't know that something was bad?Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Zeek Logs Goal: Determine what happened on the networkLogs are smallLogs are plain-text, tab-delimited CSVDesigned for grep, sed, awk, sort, head, etc.Poll: Are you currently using Zeek (Bro) on yournetwork?Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Zeek Log Limitations Logs are an abstraction 100s of Gbps of input, a few GB/hour of output requiresloss of dataOnly some protocols can be parsedRising use of encryptionLimited by network visibilitySecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Example 1: IRC141.142.11.2:4766 - 164.32.77.23:6667JOIN #foobarirc.logValueid.orig h141.142.11.2id.orig p4766id.resp h164.32.77.23id.resp p6667nickUSA 74634user[urX]-700159commandvvalueSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019JOIN#foobar

Example 2: HTTP GET /phpBB HTTP/1.1 HTTP/1.1 404 Not Foundhttp.logValueid.orig h192.168.1.20id.orig p7182id.resp h141.142.192.147id.resp p80methodGEThostwww.ncsa.eduurivstatus codeSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019/phpBB404

Types of Zeek Logs: Network Protocols conndhcpdnsftphttpirckerberosmysqlradiusSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019 rdpsipsmtpsnmpsockssshsslsyslogtunnel

Types of Zeek Logs: Files files.logpe.log (Portable Executable)x509.log (Certificate information)Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Types of Zeek Logs: Detection intel.log (Intelligence data hits)notice.log (Alerts from Zeek scripts)signatures.log (Traffic signature hits)Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Types of Zeek Logs:Network Observations known certs.log (SSL certs observed)known devices.log (MAC addresses)known hosts.log (IPs w/ established TCP)known services.log (Open ports)software.log (Software w/ version info)Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Zeek - Viewing Logs lessgrepbro-cutbawk (https://github.com/deltaray/bawk)try.zeek.orgELK Stack / Elastic StackSplunkSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Zeek - Viewing Logs (more commands) cat and zcatawksortuniqwchead and tailwhile read -r MyVar ;do something MyVar ;doneImage source: forever/

Zeek - Viewing Logs (and more) grepcidrClickHouse column store databaseEL(Kibana)SplunkWhat others?Poll: Do you use the command line regularly?

Command syntax (grep)Pattern:grep [options] search pattern [file1] [file2] [.]Real examples:grep -i administrator sysloggrep -i administrator syslog grep -v 172\.16\.0\.5Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Command syntax (awk)Pattern:awk [options] 'program' [file1] [file2] [.]Starter program keywords: {print 0} (action statements) 1, 2, ., NF 2 "foo", 2! "foo" 3 / [Bb]etty /true false, true && true()variable valueSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

How a command pipeline works Read in data, send output to next commandExample (search for IP and count instances) grep 10\.100\.52\.5 conn.log wc -l 507 Example (show list of IPs ordered by count) zcat conn.log.gz awk -F\\t '{print 3}' sort uniq -c sort -rn 155489 6.0.2172.16.0.7Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019Brief regular expression primer A regex can be used to match patterns of text data.Use " " or ' ' to protect expression from shell interpretation. - matches any single character\. - Matches a literal . (use a \ before any special character).* - matches any character zero or more times. - matches any character 1 or more times - Matches the beginning of the line. - Matches the end of the line[a-z]- matches any letter between a and z in 1 position[a-zA-Z0-9] - Matches any alphanumeric in ASCII[ 0-9] - Matches any character that is not 0 through 9.[0-9]{1,3}- Matches any character 0 - 9 between 1 and 3 times(apple banana cherry) - Matches any of these words

Regex precision is importantUse 2\.4\.150\.1 to search for the IP 2.4.150.1Why shouldn't I just run this?grep "2.4.150.1" access logBecause it will also match these IPs:22.4.150.15204.150.100.10and these values:2E49150A1/script.php?id 12948150218Poll: How specific areyour searches?

Apache's Common Log Format22.64.26.43 - - [19/Sep/2017:00:16:21 -0500] "GET/getting-started HTTP/1.1" 200 11820"https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0;Win64; x64; Trident/7.0; rv:11.0) like Gecko"Requestor’s IPServer response codeDate/time of requestSize of responseRequest methodRefererPath requestedUser-agent

Web server log analysis exercise - ( 45 min)Exercise Information Sheet:http://bit.ly/2018logs

Finding Compromised SystemsSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Phases of forcementConsolidationPillageSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Phases of forcementConsolidationPillageSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Recon: Searching for exploitable codeWhich IP had the most HTTP 404 Not Found errors? What is a 404 not found error? HTTP status return code to the clientWhat logs track this information? Zeek's http.logWhat field is it in the bro log? status codeHow can we match a number in a log? * awk, grep, sedHow can we generate a top list? * Collect like groups (sort) Count the number of items in each group (uniq -c) Order the counts. (sort -n)Security Log Analysis

Recon detection command (404s) cat http.log bro-cut id.orig h status code awk -F\\t ' 2 "404"' sort uniq -c sort -n tail -n 1 165 64.39.106.131 404 dig short -x 64.39.106.131 sn031.s01.sea01.qualys.comSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Using scripts for complex commands Recommend using shell scripts to save complex and often reused analysiscommands.Make the scripts adaptable through use of arguments.Run them regularly from cronSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Recon: Can web app read filesystem?Do any successful queries to Wordpress code contain filesystem paths in the querystring? Where do wordpress requests get logged? access log (on Apache)What should I search for? Filesystem path indicators like '/', '.', '/etc' or Specific filenames like my.cnf, passwd, .htaccessHow can I figure out if the exploit attempt worked? HTTP return status (if 404, then probably not; 200 only means potentially) Does the file referenced exist?Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Recon detection command (web app) grep -E "wp-admin.*\.\./.*\" 200 " access log 208.83.1.168 - - [23/May/2017:16:47:21 0000] "GET/wp-admin/admin-ajax.php?action revslider show image&img ././.my.cnf HTTP/1.1" 200 410 "-" "Mozilla/5.0 (X11; U;Linux i686; en-US; rv:0.9.3) Gecko/20010801"Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Recon: Did a new exploit hit us in past?Given that the recent Intel AMT vulnerability has been hidden in chips since 2010, canwe find any indication of previous attacks against our network? What are we looking for? meta data about traffic to tcp ports 16992 and 16993Where can we find this? Zeek's 'conn.log'How can we be sure the connections were successful? Check that the conn status column in conn.log is not "S0".Make a list of potential attackers first, save it to a file.Then investigate the overall activity of the potentials.Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Recon detection command (Intel AMT) zcat 201[0-7]-*/conn.*.log.gz cat - current/conn.log awk -F\\t '( 6 16992 6 16993) && 12! "S0"{print}' less -SSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019Image source: ker inside.jpg

Recon detection command (Intel AMT) zcat 201[0-7]-*/conn.*.log.gz cat - current/conn.log awk -F\\t '( 6 16992 6 16993) && 12! "S0"{print 3}' potential-attackers.txt zgrep -F -f potential-attackers.txt201[0-7]-*/conn.*.log.gz current/conn.logSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Phases of forcementConsolidationPillageSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Compromise 1: http.logid.resp hid.resp pmethodhosturireferreruser agentstatus coderesp mime typesSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 201991.134.161.4280GETtop4download.org/Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0)like Gecko200text/html

Compromise 1: http.logid.resp hid.resp pmethodhosturireferreruser agentstatus coderesp mime typesSecurity Log AnalysisGPN AHM 2019 -- May 22nd, lf06 24&4e0flef.http://top4download.org/(IE 11)200text/html

Compromise 1: http.logid.resp hid.resp pmethodhosturireferreruser agentstatus coderesp mime typesSecurity Log AnalysisGPN AHM 2019 -- May 22nd, lf06 24&4e0flef.http://top4download.org/(IE 11)200text/html

Referrer Chain1. GET top4download.org/ (IE 11) text/html2. GETroseindia.vip/?644v0o1fsfarflf06 24&4e0flef186v43re1bv 1280&37fj4d7g94969r 720 (IE 11) text/html3. GET 18a43zad864bo96.armlay.gdn/ (IE 11) text/html4. GET 18a43zad864bo96.armlay.gdn/712g40rdf7k06 (IE 11) application/x-shockwave-flashSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Overview for curity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Overview for IP1. Redirect chain with random-looking domain names,and suspicious TLDs (.vip, .gdn)2. Shockwave Flash followed by Windows executabledownload3. Queried for IP address informationSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Compromise 2: http.logJun 17 23:00:10 CcMeer3amA5aZ9nrx/version0145200 52375Jun 18 02:10:21 CQMaBW2KP1XCGMVNlb 107.160.46.226 4533/version0145200 OK- - - (empty) - - -2375Jun 18 02:34:35 CqA2Xg3qh9Lrpi6IEj 107.160.46.226 2516/version0145200 OK- - - (empty) - - -141.142.234.27 - Fr5LXVyNQ3lRrs2tg1GET141.142.234.27 - python-requests/2.10.0 Fay4vxEzVjage6cy1text/json1GET141.142.234.27 - Python-urllib/2.7 FUpmSO27PvsmkOk5n4 text/json1GET141.142.234.27 - Python-urllib/2.7 FHqbUe1aylw9O5YFP8 text/json-141.142.234.27GET-Jun 18 02:10:21 CFVSv31q8HACwAJSOc 107.160.46.226 4534141.142.234.27/v1.23/containers/json?all 0&limit -1&trunc cmd 0&size 0036000200 OK- - - (empty) - - - 141.142.234.2712375-- text/jsonJun 18 02:34:35 CTAMVF3Rv4jhcgBRAc 107.160.46.226 2517141.142.234.27 2375 1 POST 141.142.234.27 d01c9dfb847faa12286c9e3bcd5d745c/exec - python-requests/2.10.0 216 74201 Created- - - (empty) - - - Fds3MstwaFnM6XAw8text/json FpxUE944g6vBSuAfkh text/jsonJun 18 02:34:35 CTAMVF3Rv4jhcgBRAc 107.160.46.226 2517141.142.234.27 2375 2 b814518bde903c957cfdc272066d01/start31119200 OK- - - (empty) - - - FWK4NW22KWWiB462p1 text/json141.142.234.27 - python-requests/2.10.0FzCk3uWDE3YjVKkb-Jun 18 02:35:02 CaBfuW2tjnMVk7FnIl 107.160.46.226 3747/version0145200 OK- - - (empty) - - -141.142.234.27 - Python-urllib/2.7 FISSYk4kMVOJ8A9wv1 text/json141.142.234.27-23751-GET Jun 18 02:35:02 CSI7QrHUkubbD8nU1107.160.46.226 3750141.142.234.27 2375 1 POST 141.142.234.27 d01c9dfb847faa12286c9e3bcd5d745c/exec - python-requests/2.10.0 246 74201 Created- - - (empty) - - - FLzVNf1jnhEtYjki2j text/json FfkBeY1jz0SEpgK0Ktext/json

Exploit: Logins vs. non-work timeCan we analyze a log to show entries of login activity outside of normal workinghours? What service do we want to check against? sshWhat logs provide this information? /var/log/secure, /var/log/messages or /var/log/syslog Zeek's ssh.logHow to compare time of day? Use bro-cut to convert ts column to parsable local time. Use awk's substr() function for hour of the day.Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Break it downcat ssh.log bro-cut -C -d ts id.orig h id.resp h auth success direction[snipped]#open2017-06-24-00-00-38#fields ts#typesid.orig hstringaddrid.resp hauth 4T04:05:14-040010.0.1.10200.112.224.16 .172.107 73.0OUTBOUNDSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019TT

Break it down (getting a sub string)substr( string , starting index* , length of substring )(*starting index is from 1, not 0.)substr("this is easy", 9, 4);easy( 1 2017-01-24T04:03:58-0400)substr( 1,12,2)04Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Break it down (Doing comparisons) 0! / #/(Don't print lines starting with comment characters) 4 "T" && 5 "INBOUND"(Successful inbound logins)if (true) { do something } else { do something else }if (hour 9 hour 17) { print }true && true false { print }Security Log AnalysisGPN AHM 2019 -- May 22nd, 2019( Not Workin' 9 to 5 )

Exploit: Logins vs. non-work time(Check for inbound successful logs not between 9am and 5pm) cat ssh.log bro-cut -C -d ts id.orig h id.resp h auth success direction awk -F\\t ' 0! / #/ && 4 "T" && 5 "INBOUND"{ hour int(substr( 1,12,2)); if (hour 9 hour 17){print}}' less 2017-04-01T19:05:44-0400Security Log AnalysisGPN AHM 2019 -- May 22nd, 0.0.1.510.0.1.5TTTINBOUNDINBOUNDINBOUND

Exploit: Logins vs. non-work time Alternate way using modulus of epoch time. % is modulus operator, gives you the remainder afterdivision. Unix epoch time modulo 86400 will give you the same time ofday no matter what the day cat ssh.log awk -F\\t ' 8 "T" && 9 "INBOUND" &&( 1 % 86400 43200 1 % 86400 75600) {print}' less -SSecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Exploit: Two factor auth analysisCan we determine if our first level authentication hasbeen compromised by analyzing our second levelauthentication?Duo log format:Timestamp,User,Integration,Factor,IP tate,City2016-09-05 16:48:22 UTC,jskeens,ACME VPN,Duo ecurity Log AnalysisGPN AHM 2019 -- May 22nd, 2019

Exploit: Two factor auth analysis( Show all failures to auth to Duo ) awk -F, ' 6 "FAILURE" ' duolog.csv( Show failures from non-work geo locations ) awk -F, ' 6 "FAILURE" && 10 "US" && 12 ! /(Houston New York City)/ ' duolog.csv(Or you may want to look for successes too for comparison)Security Log AnalysisGPN AHM 2019 --

Security Log Analysis Tutorial Goals Today you can expect us to: Take you thru the Log Analysis Lifecycle Provide log analysis examples of real attacks with logs from Zeek,