WIXLIB

BIG-IP Secure Web Gateway and Splunk templatesSummaryBIG-IP Secure Web Gateway (SWG) provides 26 specific reports that were created toease the integration of F5 BIG-IP SWG logs and the Splunk reporting system. Eleven arein advanced view report format and fifteen are in a saved search report format. Customerscan use these reports as-is or as templates to create their own customized reports.Fourteen of the reports can be displayed in graphical form on the BIG-IP SWG Dashboardwith graphical representation. The reports are grouped into four search categories: URLsand Categories, Users, IP, and Security.PrerequisitesBy default, a Splunk server must be installed and configured to receive syslog entries onUDP port 514. BIG-IP SWG-specific logs are automatically grouped into sourcetype –“swg log.” BIG-IP SWG Splunk templates are specifically looking for syslog entries thatcontain sourcetype “swg log.” Instructions on how to set up BIG-IP SWG logging may befound at the following link: http://support.f5.com/kb/en-us/products/bigip ntations-11-5-0/7.htmlTo view Combined Reports in Splunk, you need to enable logging of the session.user.* andsession.client.* session variables in the access policy. Refer to 00/200/sol11253.html for details.Note: You can set up the BIG-IP system to send log entries to different ports or to use adifferent protocol. Then, you need to change the [source::udp:514] line in the SPLUNK nf and inputs.conf filesRefer to Data/Monitornetworkports foradditional information.Note: To distinguish from multiple BIG-IP SWG syslog sources, you can add a qualifier tothe search command. For example, host “192.168.1.15” sourcetype “swg log” and so on.These reports were developed and tested using BIG-IP version 11.5.0.

CustomizationF5 Networks SWG dashboard and saved search reports are placed in your Splunkinstallation server’s SPLUNK DIR/etc/apps/SplunkforF5AccessSWG/default directory inXML format. You can add or remove search groups in the SPLUNK nav/default.xml file. Youcan add or remove graphical reports in the SPLUNK w/SWG dashboard.xmlfile. You can add or remove saved search reports in the SPLUNK rch.conf file. Pleaserefer to http://docs.splunk.com/Documentation/Splunk for detailed customizationinstructions.Advanced SearchEleven advanced view search reports, URLs requested from category, URLs requestedfrom hostname, URLs requested by user, URLs requested by user by category, URLsrequested by user by hostname, User’s IP addresses, URLs requested by IP, URLsrequested by IP by category, URLs requested by IP by hostname, URLs requested by userfrom Security categories and URLs requested by IP from Security categories can be foundunder the SPLUNK viewsdirectory. The files are named category url.xml, host url.xml, user url.xml,user category.xml, user hostname, user ip.xml, ip url.xml, ip category.xml,ip hostname, security user.xml, and security ip.xml. You can find instructions about howto build advanced form searches on a Splunk server st/AdvancedDev/AdvancedIntro.All of these reports use subsearches to retrieve entries from logs and place them into themain request. The subsearches have time ranges that are hardcoded in the report files.You can change this time range or even remove it. You can do it by changing or removing param name “earliest” /param from “MultiSelect”.

Security CategoriesSome of the reports show the count of blocked requests for URLs and are categorized asSecurity. The Security category includes sub-categories such as “Malicious Web Sites,”“Spyware,” “Advanced Malware Payloads,” and so on. To avoid hardcoding all thesecategories in the reports, the SPLUNK es/catlist file wascreated. The SplunkforF5 application created a monitor for this file, so if you want to add anew security category you can just add a new line at beginning of the file. Or you canreplace this file, specify the categories that you are interested in, and create anotherreport. For example, you can include a counter that shows how many requests were madeto Entertainment, Facebook, and Twitter URLs in the last 24 hours. Actually, you can view“Security stats” and “Security blocks” reports from the SPLUNK rches.conf file andmonitor definitions in the SPLUNK nf file to create your ownsimilar reports.AlertsSplunk has the ability to generate alerts based on collected statistics. Alerts occur as aresult of reports that are run regularly. When the alerts trigger, different actions can takeplace, such as sending of an email with the results of the triggering search to a predefinedlist of people. Four examples of possible alerts can be found in the SPLUNK rches.conf file. Forthem to work, you need to change “action.emai.to” to the recipient email address and set“enableSched” to 1.Please refer to Alert/Aboutalerts fordetailed information.

BIG-IP SWG DashboardThe BIG-IP SWG Dashboard contains 13 graphical reports and 1 raw report:

Top 25 URLs by request count – pie chart presentation of the top 25 requestedURLs. Top 25 Blocked URLs by request count – pie chart presentation of the top 25blocked URLs.

Top 10 hostname by request count – pie chart presentation of the top 10 requestedhostnames. Top users by allowed request count – pie chart presentation of the top 20 users byallowed request count.

Top user by blocked requests count – pie chart presentation of the top 20 users byblocked request count. Allowed requests per IP address – bar chart presentation of allowed request countper IP address.

Top 20 Categories by request count – pie chart presentation of the 20 mostrequested categories. Blocked requests per IP address – bar chart presentation of the blocked requestcount per IP address.

Top Categories by blocked request count – pie chart presentation of the top 20categories by blocked request count. Last 5 SWG Events – raw syslog entry presentation of the last 5 BIG-IP SWGevents.

HTTP/HTTPS request count – column chart presentation of the HTTP and HTTPSrequest count. Recent 5 active sessions – table that shows information about 5 recent activesessions.

Unique client IP-addresses count – counter that shows how many unique clientsource IP addresses appear in logs. Security blocked requests count last 24 hours – counter that shows how manyrequests were blocked because of security categories.Note:Most widgets on the dashboard have own TimeRangePicker. One of the options it providesis All time. By default, this option searches through all events on the Splunk server. Thisbehavior may be unacceptable if there is a large amount of data on the server. So, you canchange the search range by deleting comments in savedsearches.conf on all lines thatcontain “dispatch.* time” and setting a more limited time period. Or, you can disable the Alltime option by creating a file called times.conf that contains the following lines:[all time]disabled 1You may want to switch one type of chart presentation to another. You can do it changing param name “chart” /param of “HiddenChartFormatter” related to report you wantto change in SWG dashboard.xml file located in the SPLUNK views directory.

Details on Splunk BIG-IP SWG ReportsTop URL requested – ReportThis report searches for swg log entries and chart count by destination url. Then it sortsby count and takes the first 25.Actual search command:search sourcetype ”swg log” chart count by destination url sort limit 25 – count.

Top URL blocked– ReportThis report searches for “Blocked” swg log entries. Then it charts the count bydestination url. It then sorts by count and takes the first 25.Actual search command:search sourcetype “swg log” action Blocked chart count by destination url sort limit 25– count

Top 10 hostname requested – ReportThis report searches for swg log entries. It extracts hostname from the destination urlfield, then charts the count by hostname. It sorts by count and displays the first 10.Actual search command:search sourcetype “swg log” rex field destination url“[?:http https]://(?hostname[ /]*)” chart count by hostname sort limit 10 – count

Note: You can use the following search to make this report show the number of requests by Second LevelDomains:sourcetype “swg log” rex field destination url “[?:http https]://([ /.]*\. )*(? hostname ([ /]*\.[a-z]* [0-9]*\.[09]*\.[0-9]*\.[0-9]*))(:[0-9]* )/.*” chart count by hostname

Top categories by blocked requests – ReportThis report searches for “Blocked” swg log entries. It then extracts the category namefrom the url category field. It charts the blocked requests by category. It then sorts bycount and takes the first 20.Actual search command:search sourcetype “swg log” action Blocked rex field url category “(.*/ )(? Category .*)” chartcount by Category sort limit 20 – count

Top users by allowed requests – ReportThis report searches for “Allowed” swg log entries. It charts the count by username. It thensorts by count and takes the first 20.Actual search command:search sourcetype “swg log” action Allowed chart count by username sort limit 20 – count

Top users by blocked requests – ReportThis report searches for “Blocked” swg log entries. It charts the count by username. It sortsby the count and takes the first 20.Actual search command:search sourcetype “swg log” action Blocked chart count by username sort limit 20 - count

Allowed requests per IP address – ReportThis report searches for “Allowed” swg log entries. It charts the count by source IPaddress.Actual search command:search sourcetype “swg log” action Allowed chart count by source ip

Top categories requested – ReportThis report searches for swg log entries. It extracts the category name from theurl category field. It charts the count by category name. It sorts by the count and takes thefirst 20.Actual search command:search sourcetype “swg log” rex field url category “(.*/ )(? Category .*)” chart count byCategory sort limit 20 – count

Blocked requests per IP addressThis report searches for “Blocked” swg log entries. It charts the count by source IPaddress.Actual search command:search sourcertype “swg log” chart count by source ip

HTTP/HTTPS request countThis report searches for swg log entries. It charts destination URLs that match “https://.*”or “http://.*” and displays them by host.Actual search command:search sourcetype “swg log” chart count(eval(match(destination url, “https://.*”))) as “https”,count(eval(match(destination url, “http://.*”))) by host

Recent 5 active sessionThis report searches for swg log entries. It removes entries that contain the sameusername, session id, and source ip. It sorts by time and takes the first 5 entries. Ittabulates the username, session id, and source ip.Actual search command:search sourcetype “swg log” dedup username, session id, source ip sort limit 5 - time tableusername, session id, source ipNote: You can also add time column to this table replaced “table username, session id, source ip”by “table time, username, session id, source ip”

Security blocked requests count for last 24 hoursThis report searches for security categories. It renames the cat name field to url categoryand tabulates url category. Includes a subsearch that returns category request statistics forthe last 24 hours. It replaces null values with zeros. It then summarizes the count column inthe resulting table and returns it named “blocks”.Actual search command:search sourcetype “securityCategories” rename cat name as url category table url category jointype outer [search sourcetype swg log earliest -24h stats count by url category] fillnull value 0 count stats sum(count) as “blocks”Resulting table (last step removed):