Transcription
Facing the challenge(s) ofWindows logs collectionto leverage valuableIOCsMichel de CrevoisierSecurity Analyst, Radar Cyber Security15.10.2019, Berne RadarServices // Classification: Public.
The five challenges RadarServices // Classification: Public
#1 High diversity of log sourcesBuilt-inServerrolesMicrosoftsoftware3rd partysoftwareApplicationADFSAdvanced ThreatAnalytics (ATA)Ivanti softwareCertification authorityExchangeDHCP serverSkypeDNS serverSQL ServerIIS web serverSYSMONNPS RadiusDefenderPowerShellKasperskySecuritySystem[ ] RadarServices // Classification: PublicVeeam Backup[ ]3
#2 Different log extensionsEVTXETL(standard Windows logsin XML format)(analytical logs, like DNSServer or PowerShell) RadarServices // Classification: PublicTXT(IIS, NPS, DHCP,PowerShell Transcript,former DNS logs)4
#3 Multiple architectural approachesAccess method / Protocol (MS-EVEN6, RPC, WMI, )Push vs PullAgent vs AgentlessIntermediate collector VS Direct sending to receiverCentral file store vs Shared folderManaged agent VS Unmanaged agent RadarServices // Classification: Public5
#4 Disabled and restrictive event logsValuable eventlogs disabled Protected users (if configured, on DCs only) LSA (Local Security Authority) IIS web server DNS clientEvent logs withrestrictiveaccess SMB server SMB client IIS web server RadarServices // Classification: Public6
#5 Operational constraintsSecurityData exchangePerformance Avoid usage ofhigh privileges Isolationbetweencustomer andsecurityprovider Dataencryption Securedauthenticationmethod Highavailability Compression RadarServices // Classification: PublicConfiguration Easydeployment Minimizeconfigurationchanges Low impact onoperatingsystemEnvironment Cloud Domain VSWorkgroup OT(OperationalTechnology)7
Collecting standardWindows logs RadarServices // Classification: Public
WEF/WEC introductionUnified & built-in solution to collect standard Windows logsWEF (Windows Event Forwarding)WEC (Windows Event Collector)Authentication and encryptionthrough Kerberos in a domain orTLS certificates in a WorkgroupCollects and stores all requestedevents from WEF clients accordingXML subscriptionsData exchange over WinRM (pushor pull)High availability capacities whereclients send events to each WECcollectorXML-based language to controlevent IDs to collect or to suppressnoisy eventsSettings control over GPOEPS control rateCertain 3rd party software can also:Emulate a WEC server by spoofing aWinRM listener(e.g.: SYSLOG-NG Premium, NXLog Enterprise,AlienVault USM actually uses NXLog)Manage multiple WEC servers with acentral management console(e.g.: SuperCharger from Logbinder) RadarServices // Classification: Public9
Who is publishing about WEF/WEC?HP/ArcSight, Australian Cyber Security, 2017201520132017 & 2019 RadarServices // Classification: Public10
WEF/WEC performanceScaling outTechnical characteristicsLimitationsUp to 4.000 source clients percollector (source: Microsoft)All collected events are saved inForwarded Events log fileAverage logging is 5.000 EPS, cango up to 10.000 EPS (source: Microsoft)All events are mixed without anytagging possibilitiesMaximum recommended size perevent log file: 4GBOnly standard event logs (EVTX)can be forwardedMaximum recommended size for allWindows logs files: 16GBCompression possible with event logsize reduction RadarServices // Classification: Public11
WEF/WEC advanced approachThe Palantir approach to the rescueMultiple event channels Different size and rotation strategy Channel can be tagged for SIEM ingestion Channel can be placed on differentstorage for better performancePreconfigured subscriptions XML query to specify the events to collect Specify the event channel destination RadarServices // Classification: Public12
WEF/WEC advanced approachA look in production on a WEC serverDeployment isnot automatizedRequires severalmanual actionsPotential sourceof incorrectconfigurationEvent channels RadarServices // Classification: PublicSubscriptions13
WEF/WEC deployment enhancementPowerShell at the rescueAutomatedWEC serverrole setupAutomatedPalantir toolsetdeploymentCovers eventchannel andsubscriptionsAdjusts log filesize andlocationFixes SDDLpermissions onWinRM serviceAvailable llector auto-deploy RadarServices // Classification: Public14
WEF/WECInjecting data with agent from the WEC server to your SIEMArcSight agentNXLog agentCommunityRSYSLOG agentJSONSnare agentSource clientsWEC collectorSIEMCEFSplunk UF agentWinCollect agentChosen agent software solution RadarServices // Classification: PublicWinlogbeat agentOther/ Externaltarget / providerOther targetExternalprovider/ Archivingsolution15
WEF/WECInjecting data without agent from the WEC server to your SIEMNXLog agentEnterpriseSYSLOG-NGPremiumSource clientsCertificates pushed on hostsChosen software for WinRM serverlistener emulation RadarServices // Classification: PublicSIEMCertificatesare requiredon eachsource client !16
Collecting Windows DNStransaction logs RadarServices // Classification: Public
Collecting DNS transaction logsTechnical possibilities overviewDNStransactionslogsLinux/UnixOSWindows OSDNS serverlogs1DNSdebuggingDNS clientlogs2ETWFirewall or 3rdparty solutionNIDS solutionMirrored traffic3ETLServer 2012 R2 RadarServices // Classification: PublicBind,Unbound,Dnsmasq, Passive DNSDNS EventlogSYSMON(ID 22)Disabled18
1Collecting DNS transaction logsOld school approach with Debugging DNS logsVery simpleaccessHigh impact onperformanceOnly fordebuggingpurposeNot supportedby MS forproductionDoes notinclude DNSanswerTimestampstructure maychangeDelay beforedata is written( 1min)No event ID RadarServices // Classification: Public19
2About ETWEvent Tracing for WindowsEfficient kernel-level tracing facility that allows to save kernel or application-defined eventsAllows to dynamically enable or disable logging in real time without any restart of the systemGreat open source projects available:KrabsETW (Microsoft) Performant C library to interact with bsEtwTA-DNSETWPowerShell module built around the KrabsETW unk plugin to collect DNS events from ETW using ETW)SilkETW (FireEye)Flexible C# ETW wrapper running as a service - Blackhat 19(https://github.com/fireeye/SilkETW)NXLog CommunityWindows agent provided with a native ETW module (im etw). Logs canbe saved in a file and/or sent to a remote target RadarServices // Classification: Public20
2Collecting DNS transaction logsAdvanced approach with native ETWSolutions for productionSystem tools:Low impact onperformanceEvent IDprovidedDNS answeris provided(but encoded)Notcompatiblewith WECRequiresagent or scriptinstallation RadarServices // Classification: Public Built-in: Logman, Perfmon, Netsh Installable: Xperf, Tracelog, NetMon,Microsoft MMA, TraceloggingSplunk App “TA-DNSETW”: read ETW usingthe KrabsETW library from MicrosoftNXLog CommunityNo cache file Built-in module to read and forwardETW logs21
3About ETLEvent Tracing LogsETW trace session are saved into ETL log filesETL files can be placed on a shared folder on each DNS server to be read remotelyGreat open source tools available:ETL-to-EVTXPowerShell script that reads ETL logs and writes them into Windows EventViewer (https://github.com/acalarch/ETL-to-EVTX)ETLParser (GCPartners) Executable which can decodes several types of ETL licePython script that parses DNS ETL files(https://github.com/nerdiosity/DNSplice)DNS Analytical App(Splunk)PowerShell script for Splunk UF that reads ETL logs(https://splunkbase.splunk.com/app/2937)NXLog CommunityWindows agent provided with a native ETL module. Logs can be saved in afile and/or sent to a remote targetETW2JSON (Microsoft) Read ETL file and convert it to JSON(https://github.com/microsoft/ETW2JSON) RadarServices // Classification: Public22
3Collecting DNS transaction logsAdvanced approach with ETLSolutions for productionLow impact onperformanceEvent IDprovidedETL file can beplaced in ashared folderDNS answer isprovided (butencoded)Not compatiblewith WEC perdefault (*)*ETL-to-EVTX script can convert ETL logs to EVTX log file RadarServices // Classification: PublicSystem tools: Built-in: Tracerpt Installable: Microsoft MessageAnalyzer (MMA)Splunk App “DNS analytical”: PowerShellscript that extracts ETL logs and sendit to a remote listenerNXLog Community Built-in module to read and forwardETL logs (**)**Currently in preview. Will be fully released in NXLog agent v5according NXLog support23
Steps and solutions overview RadarServices // Classification: Public
Overview of collecting methods1: requires PowerShell script that extracts ETL content into EVTX log files2: requires agent or plugin with ETL or ETW capacities3: data in event log has no structure RadarServices // Classification: Public4: not recommended, requires to query SCCM SQL Server database5: requires SQL Server advanced configuration6: pulling requires dealing with firewall, credentials and double NAT issues7: only a limited set of logs are available. Per default, format and mapping are notmaintained. SCOM is not a SIEM.25
Steps for a proper log collectionDownload Palantir toolset ngDownload and run the Radar deployment script https://github.com/rs-dev/windows-event-collector auto-deployConfigureadvanced auditpoliciesEnablePowerShellauditingConfigure clients to target your WEC server(s)Install and configure your agent solution on yourWEC server(s) to forward logs to your SIEMEnable auditingfor permissionchanges (SACL)Start gathering data in your SIEM RadarServices // Classification: Public26
.Thank You RadarServices // Classification: Public
ArcSight agent NXLog agent Community RSYSLOG agent Snare agent Splunk UF agent WinCollect agent Winlogbeat agent Injecting data with agent from the WEC server to your SIEM WEF/WEC 15 Chosen agent software solution Source clients WEC collector SIEM Other target / External provider JSON
