Health Check Framework For Ibm Security Qradar Siem

Transcription

www.scnsoft.comHEALTH CHECK FRAMEWORKFOR IBM SECURITY QRADAR SIEMADMIN GUIDEUS, McKinney, TX 1 214 306 68 37Finland, Vantaa 358 (45) 178 48 80www.scnsoft.comcontact@scnsoft.com 2017 ScienceSoft

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideTable of ContentsOverview . 3Installation . 4Download HCF Manager . 4Install HCF Manager . 4Download HCF . 5Prepare HCF server . 5Install HCF . 6Connect HCF Manager to HCF . 6Request license key . 7High Availability license . 8Install license key without HCF Manager. 8Install license key with HCF Manager. 8Execution Parameters. 9Configuration Usage . 11Manual Execution . 12Using HCF Manager . 12Using command line . 12Scheduling for Periodical Monitoring . 13Using HCF Manager . 13Using command line . 13Email Reporting . 14Using HCF Manager . 14Without HCF Manager. 14Health Markers . 16Custom Logo. 18Adding Custom logo with HCF Manager . 18Adding custom logo without HCF Manager . 18Disabling HCF Listener . 19Troubleshooting . 20Appendix A: Monitoring metrics . 21Appendix B: Release notes . 25Appendix C: Installing HCF on QRadar Console . 31 2017 ScienceSoft Page 2 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideOverviewHealth Check Framework (HCF) for IBM Security QRadar SIEM is a tool that allows QRadar users,administrators and security officers to perform periodical and on-demand monitoring of a range ofstatistical, performance and behavioral parameters of QRadar deployment including All-in-One anddistributed environments.Supported QRadar versions: 7.2.87.3.0Versions 7.2.4 to 7.2.7 were not tested.HCF gathers and analyzes more than 60 different parameters (metrics) and produces an Excel report thatcan be delivered to one or more recipients via email. This report reflects system health statistics in atabular and graphical representation. For complete list of supported metrics please refer to AppendixA: Monitoring metrics section.NOTE: HCF is a commercial software and requires a license key to run. Free demo mode with limitedfunctionality is also available. No license key required for running HCF in this mode.NOTE: HCF is developed by ScienceSoft Inc. and not supported by IBM.NOTE: HCF does not change any settings of QRadar deployment. 2017 ScienceSoft Page 3 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideInstallationFully functional HCF deployment includes the following components:Health Check FrameworkMain executable, libraries and configuration filesHCF ManagerApp Exchange extension (HCF application tab in QRadar UI)Resident tool providing interaction between HCF and HCFHCF ListenerManagerIn order to prepare HCF deployment the following steps should be taken:1.2.3.4.5.6.7.Download HCF ManagerInstall HCF ManagerDownload HCFPrepare HCF serverInstall HCFConnect HCF Manager to HCFInstall License keyRefer to corresponding sections in this document to complete all steps.NOTE: HCF Manager and HCF Listener are optional components. If you are not planning to use them,skip steps #1, #2, #6 from the list above and refer to Disabling HCF Listener section.Download HCF Manager Go to https://exchange.xforce.ibmcloud.com/hubLogin using your IBMidFilter by Type: ApplicationSelect HCF Manager extensionClick Download button at the top right cornerSave the extension zip fileInstall HCF Manager Login to QRadar UIGo to Admin tabOpen Extensions ManagementClick Add buttonClick Browse button, locate the extension file downloaded from IBM App Exchange and clickAdd buttonConfirm on all steps and wait for installation to finishClose Extensions Management window, press Ctrl F5 to fully reload QRadar UI. New HCF tabwill be addedNOTE: it takes several minutes for HCF Manager to become active after installation is completed.NOTE: For more details on using IBM App Exchange and Extension Management tool, please refer toofficial IBM documentation: 2017 ScienceSoft Page 4 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin n/SS42VS 7.3.0/com.ibm.apps.doc/c Qapps MngExts.htmlDownload HCF In QRadar UI, navigate to HCF tab and click download link. Fill out the form and click Getdownload link buttonNOTE: Internet connection is required on your QRadar Console in order to send the form. radar-siem instead and click Start your free trial button Download link will be sent to your email address defined in the form. Follow the link and savethe zip filePrepare HCF serverHCF requires a physical or virtual server with the following minimum specifications: RAM: 2 GbHDD: 5 GbCPU: 2 coresNetwork adapter: 1OS: CentOS 7Below are the steps to prepare HCF server: Allocate resources for virtual server or prepare a physical server according to systemrequirements listed above 2017 ScienceSoft Page 5 from 31at

Health Check Framework for IBMSecurity QRadar SIEM: Admin Guide Download CentOS-7-x86 64-Minimal-1611.iso from official CentOS mirrorInstall CentOSSet up default network interface:o Assign IP address from the same subnet/VLAN as your QRadar Console/AiO (this IP addressis referred below as HCF Server IP)o Make sure the interface is enabled on booto Verify SSH connection to QRadar Console/AiOInstall additional packages using the following command:o yum install zip openssh openssh-clientsAllow HTTPS communication to HCF server:o firewall-cmd --zone public --add-port 443/tcp --permanento firewall-cmd --reloadNOTE: Refer to CentOS official documentation to complete all OS installation and configuration steps.NOTE: On all involved firewalls, a two-way communication must be allowed between QRadar Consoleand HCF server via ports 22 (SSH) and 443 (HTTPS).Install HCF Extract HCF- version .x86 64.rpm file from the archive obtained in Download HCF sectionUpload the RPM file to HCF server using your preferred SCP clientLogin as root user to HCF server and change directory to the one containing the RPM packageInstall using the following command:rpm –Uvh RPM file name HCF will be installed to /opt/scnsoft/hcf folder.Linux man page is available for HCF: man healthcheckConnect HCF Manager to HCFIn order for HCF Manager to interact with HCF server the following steps should be taken: Make sure QRadar Console/AiO appliance is accessible via SSH (port 22):ssh QRadar Console IP In your web browser go to https:// HCF Server IP where HCF Server IP is an IP addressdefined in Prepare HCF server section. Add security exception when required. 2017 ScienceSoft Page 6 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin Guide Click Generate SSH keys buttonEnter QRadar Console IP address and QRadar Console root passwordClick Generate HCF keys buttonCopy the generated connection key to clipboard Login to QRadar UI as Admin userGo to HCF tab Enter HCF Server IP address and paste the connection key into corresponding fieldsClick Connect buttonHCF Manager interface will be shown in HCF tab, containing four sections: HCF deployment, Executionparameters, Reports and Health Markers.Request license keyIn order to generate Excel reports, HCF requires a license key. You can request a commercial license bysending request to contact@scnsoft.com.The following information required for the license key to be created: Company nameContact personContact person positionContact emailContact phone 2017 ScienceSoft Page 7 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin Guide QRadar versionUUID code of QRadar Console/AiOTotal licensed EPS capacityIn order to obtain UUID code of your QRadar Console/AiO follow the steps below: Login as root user to QRadar Console/AiO via SSHExecute command: dmidecode -s system-uuidCopy the generated alpha-numeric code to use it in your license request.You will be sent an email with a license key once you complete your purchase.High Availability licenseIf you have a High Availability option for QRadar Console/AiO, additional license key is required to getHCF reports when the secondary instance is active.In this case, make sure to include in your request both primary and secondary UUIDs. Note that astandby instance is only accessible via SSH from an active one.Install license key without HCF ManagerExtract license.key file from the received archive and upload it to the following folder on HCF server:/opt/scnsoft/hcfInstall license key with HCF Manager In QRadar UI, navigate to HCF tabOpen HCF deployment sectionClick Select license file button, locate the ZIP file received from ScienceSoft and click Uploadlicense button 2017 ScienceSoft Page 8 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideExecution ParametersCommand lineHCF ManagerDescription-h, --helpN/AShow HCF help.-v, --versionN/AShow HCF version.Enable debugDebug mode. Enhanced execution log will be hcf/reports/HCF-YYYY-mm-DD-HH-MM-d, mation about your deployment and connectedLog Sources will be extracted from configurationdatabase.-rsm, --remoteN/AEnable HCF-QRadar connection. Required.-i, --host [Console IP]N/ADefine QRadar Console IP address. Required.-p, --port [PORT]N/ADefine custom SSH port (default is 22).-k, --ssh-keyN/ASSH keys storage. Default is /root/.ssh/id rsa.-ndq, --no-data-Disable Data QualityDisable Data Quality analysis in order to reduce HCFqualityanalysisexecution time.-nam, --no-advanced-Disable AdvancedDisable Advanced Metrics in order to reduce HCFmetricsJMX metricsexecution time.-r, --rules-Rules performanceRules performance check duration (in seconds). Ifperformance-intervalintervalnot defined, will run with default 600 seconds interval[FILENAME][X]-ad, --ariel-delta [X]-mf, --mail-from[EMAIL](10 minutes).Time range for ArielTime range for Ariel queries (in hours). If not defined,querieswill run with default 24 hours range.Define customDefine custom FROM address for email reports.FROM address foremail reports-cl, --custom-url [URL]URL for custom logoDefine custom URL for logo in XLS report.-to, --top-offences-Top Offenses CountNumber of offenses to display for Top Uniquecount [X]Offenses metric. Default is 10. 2017 ScienceSoft Page 9 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin Guide-ta, --top-assets-countTop Assets Count[X]-bn, --backups-Number of assets to display for Top Risky Assetsmetric. Default is 10.Backups numbernumber [X]Number of backups to display for Last backupsmetric. Default is 5.-tlt, --log-source-Log Source TypesNumber of Log Source Types to display for EPS pertypes-count [X]CountLog Source Type metric. Default is 10.Log Source ActionsNumber of Log Sources to display for Last inactive,Countdisabled, added, deleted, modified Log sources-lsa, --log-sourceactions-count [X]and Protocol Configuration Errors metrics. Defaultis 10.-rp, --rules-Rules PerformanceNumber of rules to display for Rules Performanceperformance-countCountmetrics. Default is 10.Top ReportsNumber of reports to display for Top Heavy Reports[X]-tr, --top-reports [X]-sn, --systemnotification-count [X]metric. Default is 10.Sys NotificationNumber of entries to display for Last Warnings andCountErrors from System Notification metrics. Default is10.-ae, --autoupdate-Autoupdate ErrorsNumber of entries to display for Top Autoupdateerrors-count [X]CountErrors metric. Default is 10.For example: executing /opt/scnsoft/hcf/hcfmain -rsm -i 10.11.6.192 -d -ndq -r 60 will run HCF for QRadarConsole at 10.11.6.192 in debug mode, skipping Data Quality metrics, and rules performance monitoringwill hold on for 1 minute. 2017 ScienceSoft Page 10 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideConfiguration UsageIn order to override default parameters, /opt/scnsoft/hcf/config.json file is used. It can be edited eithermanually or via HCF Manager.Set desired values in Execution parameters fields and click Save configuration button. All further ondemand HCF reports will be generated using this configuration, unless overridden via parameters set incommand line or HCF Manager.To revert to out-of-the-box settings, click Reset to default button. 2017 ScienceSoft Page 11 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideManual ExecutionUsing HCF Manager Login to QRadar UIGo to HCF tab, Execution parameters sectionDefine execution parameters as described in the previous sectionClick Run HCF buttonExecution status will be displayed at the top of the window. Once finished, reports list will be updated.Using command line Login as root via SSH to your HCF ServerRun /opt/scnsoft/hcf/hcfmain [-rsm -i XXX.XXX.XXX.XXX] [ ]with parameters defined according to the previous sectionExecution log will be displayed in the console. 2017 ScienceSoft Page 12 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideScheduling for Periodical MonitoringUsing HCF Manager In QRadar UI, go to HCF tab, Execution parameters sectionClick Add to schedule buttonDefine the schedule using drop-down lists or enter manually in the text fieldClick Schedule buttonClick Edit HCF tasks button to review and/or change existing crontab entriesUsing command lineCron tasks can be edited via on HCF Server using crontab -e command. Refer to vi editor user’s guide forinstructions.NOTE: refer to crontab scheduling format to create desired HCF tasks. 2017 ScienceSoft Page 13 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideEmail ReportingAfter each run HCF can send reports via email. By default, the report will be sent to the email addressspecified under System Settings – Administrative Email Address in QRadar Admin tab:If you want to send HCF reports via email to other addresses, refer to steps below.Using HCF Manager Login to QRadar UINavigate to HCF tabOpen HCF deployment sectionClick Create report recipients list button. QRadar Reference Set will be created and thebutton will change to Report recipients which is intended to manage email addresses viaQRadar standard Reference Set editor.Without HCF Manager Create new Reference Seto Login to QRadar UIo Navigate to Admin tabo Press Reference Set Management buttono Press Add buttono Type HCF Report Emails as Name 2017 ScienceSoft Page 14 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin Guideoo Enable Lives Forever checkboxPress Create buttonUpdate Reference Set contento Double click on HCF Report Emails reference seto Press Add buttono Add one or several email addresses to the listNOTE: In order to temporary disable email reports without removing the existing recipients, add nomailitem to the Reference Set. Delete this item once you need email reports again. 2017 ScienceSoft Page 15 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideHealth MarkersHCF provides user with extended email reports which contain 25 “OK/Failed” Health Markers in order toindicate important metrics changes in your QRadar deployment. In case of marker fire you’ll receive awarning with description and some basic recommendations.Health Markers fire on the following metrics: Console Disk Usage: if used disk space on the Console/AiO appliance exceeds the giventhreshold (95% by default). Deleted Log Sources: if at least one Log Source was deleted during the last days (3 days bydefault). Modified Log Sources: if at least one Log Source was modified during the last days (3 days bydefault). Autoupdate Errors: if at least one Autoupdate failed during the last days (3 days by default). Asset Risk Level: if at least one Asset reached Risk level, which exceeds the top-10 averagelevel on more than given threshold (70% by default). Offense Types: if at least one Offence type occurs more often (80% by default) than the top-10average periodicity. Nightly Backups: if at least this many (0 by default) failures occurred among last 5 backups. System Notifications: if at least one error/warning was detected in System Notificationsjournal during the last days (3 days by default). Inactive Log Sources: if at least one Log Source became inactive during the last days (3 days bydefault). Disabled Log Sources: if at least one Log Source was disabled during the last days (3 days bydefault). 2017 ScienceSoft Page 16 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin Guide Protocol Errors: if at least one Log Source has protocol configuration errors. Modified Searches: if at least one Search was modified during the last days (3 days by default). Data Integrity: if at least one event/flow data file corrupted or integrity check failed. Rules Execution Time: if at least one correlation rule executes longer than the top-10 averagerules execution time on more than given threshold (70% by default). Rules Response Time: if at least one correlation rule responses longer than the top-10 averagerules response time on more than given threshold (70% by default). Reports Execution Time: if at least one report executes longer than the average executiontime among top-10 heaviest reports on more than given threshold (70% by default). Distributed EPS: if at least one managed host reached EPS utilization more than the giventhreshold (95% by default). Distributed FPI: if at least one managed host reached FPI utilization more than the giventhreshold (95% by default). Managed hosts RAM: if at least one managed host runs below the given amount (10% bydefault) of free RAM. Managed hosts CPU: if at least one managed host has CPU load over the given threshold (95%by default) in the last 15 minutes. Managed hosts /store partition: if at least one managed host has used /store partition spaceover the given threshold (90% by default). Managed Hosts Status: if at least one managed host is in state different than Active orStandby (normal operation of HA appliances). Generic DSM: if at least one SIM Generic DSM Log Source generates more than given numberof events (50 by default). Unknown Events: if at least one Log Source generates more than given threshold of unknownevents (90% by default).Default thresholds can be modified either by editing the markers.json file stored in /opt/scnsoft/hcf/folder or through HCF Manager by defining required values per marker:NOTE: markers.json file will be overwritten during update via command line. If you have made anychanges in markers configuration, please backup this file before updating. When updating via HCFManager, the config will be backed up and restored automatically. 2017 ScienceSoft Page 17 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideCustom LogoFor branding purposes, HCF allows to use a custom logo picture in report headers.The following requirements must be met: File name: logo.png (not necessary when uploading via HCF Manager)Image format: PNGColor depth: 24 bitImage size: 296x59 or lessImage resolution: 72 dpiReport header background color is RGB 22, 54, 92 (HEX #16365C). For better logo readability usetransparent image background or contrast colors.Adding custom logo with HCF Manager Prepare your logo file according to the requirements aboveLogin to QRadar UINavigate to HCF tabOpen Reports sectionClick Select logo file button and locate your logo fileClick Upload button. A warning message will be shown if some requirements are not met.Click Delete custom logo button to remove your custom logo when necessary.NOTE: only one logo file can be stored at one time. Any existing logo file will be overwritten after pressingUpload button.Adding custom logo without HCF ManagerPrepare a Custom logo file according to the requirements above and upload it via SCP to/opt/scnsoft/hcf/store folder on your HCF Server. 2017 ScienceSoft Page 18 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideDisabling HCF ListenerDuring HCF installation the following cron task will be added:*/5 * * * * /opt/scnsoft/hcf/hcflistenerIf you don’t use HCF Manager application, change the schedule and stop already running instance asfollows: Login as root to HCF Server via SSHExecute crontab -e commandNavigate to the line stated above with cursor keysComment out o Press i key to enter Edit Modeo Insert # in the beginning of the lineo Press Esc key to exit Edit ModeOr delete o press d key twice to delete entire linePress Shift z keys twice to save changes and exit from cron editorExecute pkill hcflistener command. 2017 ScienceSoft Page 19 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideTroubleshootingIf you have problems with HCF execution or reports generation, run it with debug mode enabled:/opt/scnsoft/hcf/hcfmain –d [other parameters]Or execute via HCF Manager with Enable debug checkbox selected.HCF-YYYY-mm-DD-HH-MM-DebugInfo.zip file will be generated, stored at /opt/scnsoft/hcf/reportsfolder and attached to the report email. Forward this file and your Excel report to the following addressfor investigation: hcf.support@scnsoft.com 2017 ScienceSoft Page 20 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideAppendix A: Monitoring metricsThe following metrics are monitored with HCF:7.2.87.3.0QRadar software version Console IP address Console UUID Version history Users IP Address HA IP Address Hostname HA host role Console Host status Uptime, days Average CPU load, % Total RAM, MBs Free RAM, % Total /store space, GBs Free /store space, % Disk usage alert Appliance Type Disk usage details per Managed host Recent backups Integrity of Events for recent 24h Integrity if Flows for recent 24h Last warnings and errors from System Notification Last autoupdate errors Number of active Log Sources Number of active Log Source Groups Last inactive Log Sources (list) Last Disabled Log Sources (list) Protocol Configuration Errors (list) Last added Log Sources (list) Last modified Log Sources (list) Last deleted Log Sources (list) Last 10 modified Log Sources (list) GenericDeployment: QRadar hostsDeployment: System healthDeployment: Log SourcesAll Log Sources list:Log Source name 2017 ScienceSoft Page 21 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideLog Source Identifier Activity (Active/Inactive/Warning/Error) Last seen (date) Average EPS Peak EPS Peak EPS date Protocol Source type Extension (LSX/LSE) Added (date) Addition type (Manual/Auto-discovered) Bulk added (Yes/No) Status (Enabled/Disabled) Log Source Groups Description Modified (date) Collector Event (flow) processor IP address Average EPS Average FPM EPS license limit FPM license limit EPS utilization, % FPM utilization, % Average EPS from qradar.log Average FPM from qradar.log EPS per Log Source type Average Peak Average License limit Category coverage, % Category name Average event severity Number of seen event types Number of supported event types Total seen event count in category QRadar database IDEnvironment: EPS/FPM statisticsEPS/FPM statistics per host:Environment: Raw inbound EPS per managed hostLicense limitEnvironment: Raw inbound FPM per managed hostEnvironment: Data Quality by device type 2017 ScienceSoft Page 22 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideEvent coverage within category, % Device type Log Source name Average seen event severity Number of seen event types Number of supported event types Total seen event count Event coverage, % Best coverage chart Worst coverage chart Source IP Log Source name Device type Total seen event count Unknown events, % IP address Total number of seen events Event average payload size, bytes Event average rate Event average record size, bytes Event records dropped (queue full) Event dropped records count Flow average payload size, bytes Flow average rate Flow average record size, bytes Flow records dropped (queue full) Flow dropped records count QRadar DSM information Number of assets Top most risky assets Top unique offenses Offense closing reasons Rule name Rule description Rule tests Offense ID Environment: Data Quality by Log SourceEnvironment: Unknown events and sourcesUnknown events:Unknown sources:Environment: Runtime statisticsEnvironment: AssetsCorrelation: OffensesOffense analysis: 2017 ScienceSoft Page 23 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideOffense source (Source IP/username/Event name/etc.) Count (events/flows) Number of enabled rules Number of disabled rules Number of Building Blocks Number of custom rules Number of modified rules Top fired rules count Top average execution time Top average action time Top average response time Top heavy reports Last modified searches Correlation: RulesCorrelation: Reports 2017 ScienceSoft Page 24 from 31

Health Check Framework for IBMSecurity QRadar SIEM: Admin GuideAppendix B: Release notesHCF 3.0.0 (5 Aug 2017): New: rearranged and polished XLS reportNew: platform change (CentOS 6 to CentOS 7)New: multiple CLI parameters for metrics customizationNew: ability to run on a secondary HA hostHCF 2.6.1 (7 Jul 2017): Fixed: handling of empty/failed UUID in ListenerFixed: proper handling POST requests in ListenerImproved: SSH key creation in ListenerHCF 2.6.0 (10 May 2017): Improved: user input validation for execution parametersFixed: handling of canceled Ariel searchesHCF 2.5.9 (6 Apr 2017): Fixed: handling empty values in Disk Usage metricsImproved: disk usage alert highlighting in Managed HostsNew: CLI parameter to change FROM field in report emailsNew: CLI parameter to change URL under custom logoHCF 2.5.8 (31 Mar 2017): New: QRadar 7.3.0 supportImproved: SIM audit user information for log sources metricsFixed: handling missing rule data in Offense analysisHCF 2.5.7 (3 Mar 2017): New: upda

Health Check Framework (HCF) for IBM Security QRadar SIEM is a tool that allows QRadar users, administrators and security officers to perform periodical and on-demand monitoring of a range of statistical, performance and behavioral parameters of QRadar deployment including All-in-One and distributed environments. .