AlienVault Plugin Documentation - Blueliv

Transcription

AlienVault Plugin DocumentationAUTHOR: BluelivDATE: April 16, 2015

Restricted access documentContents1 Installation21.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21.2 Related files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21.3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21.4 Activate/Deactivate plugin . . . . . . . . . . . . . . . . . . . . . . . .42 Getting started52.1 CrimeServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52.2 Bot Ips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .62.3 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7AnnexesAFiles9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Plugin code10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10Plugin Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . .16AlienVault Configuration File . . . . . . . . . . . . . . . . . . . . . .17AlienVault SQL Script18. . . . . . . . . . . . . . . . . . . . . . . . . .Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 1

Restricted access documentInstallation1.1RequirementsThis plugin has been tested on a 4.6.1 version of AlienVault OSSIM. Forthe full functionality of this plugin, a standalone install of python 2.7 must bepresent.An internet connection is required in order to download the data feed fromblueliv’s API. Also note that the machine from where you are going to executethe plugin must see the AlienVault machine through the port 515.1.2Related files blueliv.cfg [3]: Basic configuration for AlienVault to be able to enable thecustom plugin and parse the logs. blueliv.sql [4]: SQL script to add the required entries to enable the plugin. BluelivAlienVaultPlugin.py [1]: Python script to feed AlienVault with Blueliv’s data. config.cfg [2]: Configuration file to set-up the access to Blueliv’s data feeds.1.3Installation1. Add your api-key at ’config.cfg’ (Listing 2) under section [blueliv] propertytoken.[ b lueliv ]token host https : / / api . blueliv .com2. Add your OSSIM host at ’config.cfg’ (Listing 2) under section [alienvault]property host.Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 2

Restricted access document[ alienvault ]host port 5143. Edit ’blueliv.cfg’ (Listing 3) to set your desired plugin id (defaul 9002) undersection [DEFAULT] property plugin id.[DEFAULT]plugin id 90024. Make sure that the plugin id on ’blueliv.cfg’ is the same as the plugin id in’blueliv.sql’ (Listing 4).5. Create a file called ’blueliv.conf’ at /etc/rsyslog.d with the following content:i f fromhost ip ’your plugin ip here ’ then / var / log / b l u e l i v . log& Replace your-plugin-ip-here with the IP from where you are going to execute your plugin. In order to not over populate this log file, configurea rotation rule for this file at /etc/logrotate.d/rsyslog by including the file/var/log/blueliv.log. This will force all logs coming from the specified IP towrite at /var/log/blueliv.log.6. Save ’blueliv.cfg’ at /etc/ossim/agent/plugins7. Execute cat blueliv.sql ossim-dband /etc/init.d/ossim-server restart8. Enable blueliv plugin, either by console or web UI.9. Execute python BluelivAlienVaultPlugin.py -feed FEED NAMEfrom where you saved the plugin. Replace FEED NAME with the feed thatyou want to activate (’botips’, ’crimeservers’ or ’all’).Once everything is set, check if /var/log/blueliv.log at your specified locationis filled with lines like the one below.Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 3

Restricted access documentApr 14 03:55:27 192.168.3.72 crimeservers http : / /m stone . co . jp / i n s t a l l /FlashFXP5 3822 Setup . exe MALWARE 211.125.81.112 m stone . co . jp JP GMO Internet, Inc ONLINE UNCLASSIFIED 2015 03 24T18:50:57 0000 2015 03 25T01:17:08 00001.4Activate/Deactivate pluginIn order to activate/deactivate this plugin. Go to Configuration Deployment.Go to ’Sensor Configuration’.Search for ’blueliv’ in Plugins available list, add it to Plugins enabled andapply changes.Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 4

Restricted access documentGetting started2.1CrimeServerA CrimeServer is an entity that stores relevant data (hosts, IPs, geolocationdata, etc) about a given server that is used to perform some kind of maliciousactivity. An example of a Crime Server is shown below:{" id " : 7c5d792fbd4f2b6" ," url " : " http : / / 0 rrkut2012 . j e . ro " ," type " : "C AND C" ,"subType" : "ZEUS" ," country " : "KR" ,"countryName" : " Republic of Korea" ," status " : "ONLINE" ,"domain" : " je . ro " ," host " : "0rrkut2012 . je . ro " ," latitude " : 37 . 57 ," longitude " : 126 . 98 ," ip " : "66.232.140.226" ," createdAt " : "2011 01 19T21:34:59 0100" ,"updatedAt" : "2014 11 25T13:53:25 0100" ,"asnDesc" : " Enterprise Networks" ," firstSeenAt " : "2011 01 19T21:34:59 0100" ," lastSeenAt " : "2011 01 19T21:34:59 0100"}This data provide Indicators of Compromise (IoC) keys such as domain, ipaddress, url, etc. That allow you to correlate with your logs to identify and blockany potential risk.All this parameters are mapped into a syslog event that will be parsed by aregexp from the OSSIM. This mapping is described below. src ip: CrimeServer IPBlueliv is a registered trademark of Leap In Value S.L., all rights reserved 5

Restricted access document hostname: CrimeServer hostname userdata1: CrimeServer Type userdata2: CrimeServer url userdata3: CrimeServer Country userdata4: CrimeServer ASN userdata5: CrimeServer Status [ONLINE/OFFLINE] userdata6: CrimeServer SubType date: CrimeServer last seen date.2.2Bot IpsA Bot IP is an entity that stores relevant data about a given IP infected bymalicious software that is part of a botnet. An example of a Bot Ip is shownbelow:{" botnetFamily " : [" Trojan Banker"]," ip " : " 220.89.127.42 " ," country " : "AU" ,"countryName" : " Australia " ," latitude " : 37 . 7833 ," longitude " : 144 . 9833 ,"seenAt" : "2015 06 01T04:35:16 0000" ," botnetUrl " : " http ://89.110.147.222 " ," botnetIp " : "89.110.147.127 " ,"botnetType" : "DRIDEX" ,"operatingSystem" : "Windows 7 , x64 , SP 1" ," botId " : 5af8fb9746a80ca" ," botVersion " : "131151" ," c i t y " : " Fitzroy " ," createdAt " : "2015 06 01T12:41:40 0000" ," botnet : "245"}Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 6

Restricted access documentThis data allows you to detect infected hosts inside your networks and analyze the level of compromise thanks to the context provided such as the Operating System, BotNet, portal domain, etc.All this parameters are mapped into a syslog event that will be parsed by aregexp from the OSSIM. This mapping is described below. src ip: Bot IP hostname: BotNet Type userdata1: Portal Url userdata2: Portal Domain userdata3: Bot Country userdata4: Bot City userdata5: Bot ASN userdata6: Bot Operating System dst ip: BotNet IP userdata8: BotNet Port userdata9: BotNet url date: Seen at date.2.3EventsThis is a sample of a CrimeServer event sent it to OSSIM. Each event is agroup of parameters separated by pipes ordered as follow.Apr 14 03:55:27 192.168.3.72 crimeservers http : / /m stone . co . jp / i n s t a l l /FlashFXP5 3822 Setup . exe MALWARE 211.125.81.112 m stone . co . jp JP GMO Internet, Inc ONLINE UNCLASSIFIED 2015 03 24T18:50:57 0000 2015 03 25T01:17:08 0000Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 7

Restricted access documentProcessed events can be shown at ’ANALYSIS SECURITY EVENTS(SIEM)’ filtering by plugin ’blueliv’.Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 8

Restricted access documentAnnexesBlueliv is a registered trademark of Leap In Value S.L., all rights reserved 9

Restricted access documentAFilesPlugin code1#! / usr / bin / python2# * coding : utf 8 * 34import argparse5import u r l l i b6import sys7import json8import ConfigParser9import socket10import datetime11import logging12import logging . handlers13import time14import threading15from BotIps import BotIps16from CrimeServers import CrimeServers17from l i b . dateutil import parser18from SyslogConverter import SyslogConverter1920class AlienVaultPlugin ( ) :2122RESERVED CHARACTERS [ " " , " \ \ " , " " ]232425def i n i t ( self ) :#LOGGER26self . logger logging . getLogger ( "MainApp" )27self . logger . setLevel ( logging .INFO)28# Create the logging f i l e handler29l o g f i l e " blueliv . log "30fh logging . handlers . RotatingFileHandler ( l o g f i l e , maxBytes 2097152,backupCount 5)31formatter logging . Formatter ( ’%(process )d %(asctime ) s %(name) s %(levelname ) s %(message) s ’ )32333435fh . setFormatter ( formatter )# Add handler to logger objectself . logger . addHandler ( fh )#CONFIG FILE36self . config ConfigParser . SafeConfigParser ( )37self . config . read ( ’ config . cfg ’ )38Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 10

Restricted access document39try :40AlienVaultHost self . config . get ( ’ alienvault ’ , ’ host ’ )41AlienVaultPort self . config . getint ( ’ alienvault ’ , ’ port ’ )42i f AlienVaultHost ’ ’ :raise4344except Exception as e :45self . logger . error ( " AlienVault Host and Port must be set in b l u e l iv . cfg " )46print " AlienVault Host and Port must be set in b l u e l iv . cfg "47sys . e x i t ( 1)4849#SYSLOG50self . loggerlt logging . getLogger ( " alienvault " )51self . loggerlt . setLevel ( logging .DEBUG)52ce logging . handlers . SysLogHandler ( address (AlienVaultHost , AlienVaultPort ) )53formatter logging . Formatter ( ’ %(message) s ’ )54ce . setFormatter ( formatter )55self . loggerlt . addHandler ( ce )5657try :self . config . get ( ’ b l u el i v ’ , ’ token ’ )58token 59apiHost self . config . get ( ’ b l u e li v ’ , ’ host ’ )60i f token ’ ’ or apiHost ’ ’ :raise6162except Exception as e :63self . logger . error ( " Blueliv Host and token must be set in b l u e l iv . cfg " )64print " Blueliv Host and token must be set in b l u e l iv . cfg "65sys . e x i t ( 1)6667try :self . SCHEDULER TIME CS SECONDS self . config . getint ( ’ crimeservers ’ , ’ t t l ’ )*6068except :69self . logger . warning ( "No scheduler configured , using default 15 minutes" )70self . SCHEDULER TIME CS SECONDS 15*60717273try :self . SCHEDULER TIME BOTIPS SECONDS self . config . getint ( ’ botips ’ , ’ t t l ’ )*6074except :75self . logger . warning ( "No scheduler configured , using default 10 minutes" )76self . SCHEDULER TIME BOTIPS SECONDS 10*6077787980try :i f self . config . getboolean ( ’ proxy ’ , ’ enable ’ ) is True :i f not self . config . get ( ’ proxy ’ , ’ host ’ ) :Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 11

Restricted access documentself . logger . error ( "Proxy host must be set in b l u e l iv . cfg or disable81proxy" )sys . e x i t ( "Proxy host must be set in b l u el i v . cfg or disable proxy" )82else :83host self . config . get ( ’ proxy ’ , ’ host ’ )84i f not self . config . get ( ’ proxy ’ , ’ port ’ ) :85self . logger . error ( "Proxy host must be set in b l u e l iv . cfg or disable86proxy" )sys . e x i t ( "Proxy port must be set in b l u el i v . cfg or disable proxy" )87else :88port self . config . get ( ’ proxy ’ , ’ port ’ )89i f self . config . get ( ’ proxy ’ , ’ user ’ ) and self . config . get ( ’ proxy ’ , ’password ’90):self . logger . info ( "Using proxy with credentials " )9192user self . config . get ( ’ proxy ’ , ’ user ’ )93password self . config . get ( ’ proxy ’ , ’password ’ )94proxies {" http " : " http ://{0}:{1}2:3".format(user, password, host,port),"https" : "http://0:1{2}:{3}" . format( user , password , host , port )}else :9596self . logger . info ( "Using proxy without credentials " )97proxies {" http " : " http ://{0}:{1} " . format( host , port ) , " https " : " http://{0}:{1} " . format( host , port )}else :98proxies None99100except :101self . logger . error ( " Error getting proxy settings . Check b l u e l iv . cfg " )102sys . e x i t ( " Error getting proxy settings . Check b l u el i v . cfg " )103104cs mapping [105’ url ’ ,106’ type ’ ,107’ ip ’ ,108’ host ’ ,109’ country ’ ,110’asnDesc ’ ,111’ status ’ ,112’subType ’ ,113’ firstSeenAt ’ ,114’ lastSeenAt ’115]116117bots mapping [118’ ip ’ ,119’ botnetType ’ ,120’ portalUrl ’ ,Blueliv is a registered trademark of Leap In Value S.L., all rights reserved 12

Restricted access document121’ portalDomain ’ ,122’countryName ’ ,123’ city ’ ,124’asnDesc ’ ,125’ operatingSystem ’ ,126’ botnetIp ’ ,127’ destinationPort ’ ,128’ botnetUrl ’ ,129’ seenAt ’ ,130]131132self . cs converter SyslogConverter (cs mapping133134)135self . bots converter SyslogConverter (bots mapping136137)138self . BOTIPS OUT OF DATE TIME self . config . getint ( ’ botips ’ , ’ nupdates ’ )139self . CRIMESERVERS OUT OF DATE TIME self . config . getint ( ’ crimeservers ’ , ’nupdates ’ )140self . BOTIPS SCHEDULER TIME SECONDS self . SCHEDULER TIME BOTIPS SECONDS*1000141self . CRIMESERVERS SCHEDULER TIME SECONDS self . SCHEDULER TIME CS SECONDS*1000142self . botsIpsFeed BotIps ( ’ https : / / api . b l u e l iv .com ’ , token , self .BOTIPS SCHEDULER TIME SECONDS, self . BOTIPS OUT OF DATE TIME, 60, proxies )143self . crimeServersFeed CrimeServers ( ’ https : / / api . b l u e l iv .com ’ , token , self .CRIMESERVERS SCHEDULER TIME SECONDS, self . CRIMESERVERS OUT OF DATE TIME,60, proxies )144145def cs to event ( self , apiDict ) :146severity 3147try :148149150151152153i f apiDict [ " type " ] "C AND C" :severity 5

1.1 Requirements This plugin has been tested on a 4.6.1 version of AlienVault OSSIM. For the full functionality of this plugin, a standalone install of python 2.7 must be present. An internet connection is required in order to download the data feed from blueliv’s API. Also note that the machine from where you are going to execute