Azure Sentinel Deployment Best Practices - IT Best Of Breed

Transcription

Microsoft SecurityAzure SentinelDeploymentBest PracticesAuthors:Adrian Grigorof, CISSP, CRISC, CCSKMarius Mocanu, CISSP, SABSA, CISM, CEHJordan Shaw-Young, CISM

Table of ContentsIntroduction.4Azure Sentinel cloud-native SIEM.5Azure Sentinel unified integration.5Cloud SIEM architecture.6Core Azure Sentinel solution components.6Azure Log Analytics workspace.6Azure Sentinel.8Azure Logic Apps.8Data sources.10Implementing a new Azure Sentinel solution.20Project resourcing.20Project planning.20Project manager.21Security architect.21Cloud engineer.21Engineering–systems owner.21Engineering–SIEM.21Network engineer.22Business analyst.22Security operations.23Developer.23Compliance manager.23

Benchmark project effort and duration.23Design planning.26Architecture planning and considerations. 26Deploy.34Azure resources. 34Log source onboarding. 36Automation playbooks. 43Deploying workbooks. 47Deploying user and entity behavior analytics. 48Deploying notebooks. 50Deploying cyber threat intelligence functionality. 50Deploying alert rules. 57Migration from existing SIEM solutions.60Scenario 1. 60Scenario 2. 62Scenario 3. 63Azure Sentinel—business considerations.65Cost management.65Evaluating your data ingestion against use cases. 65Log ingestion strategies. 65Budgeting for Azure Sentinel costs. 69Ongoing Cost monitoring and evaluation. 70Conclusion and resources.71Additional resources.71

IntroductionThe purpose of this whitepaper is to providesecurity organizations with a practical field guideto assist in developing a deployment strategyfor Microsoft Azure Sentinel that will employbest practices to support a stable, cost-effective,and operationally effective implementation ofMicrosoft’s cloud-native security informationand event management (SIEM) platform. Thisdocument is written from a security practitionerperspective, based on experience deploying andmanaging Azure Sentinel in a wide rangeof organizations.We intend for this guide to serve as a referenceand planning document primarily for chiefinformation security officers, security architects,and enterprise architecture and projectmanagement leaders in defining adoption andmigration strategies and budgets and in planningproject and resourcing requirements for asuccessful implementation of Azure Sentinel. Itcan be read as a companion document to otherAzure Sentinel technical whitepapers such as theAzure Sentinel Technical Playbook for MSSPs.1Azure Sentinelcloud-native SIEMAzure Sentinel is Microsoft’s cloud-native SIEMsolution and the first cloud-native SIEM from amajor public cloud provider. Azure Sentinel isdeployed in an organization’s Azure tenant andaccessed via the Microsoft Azure portal,ensuring alignment with preexistingorganizational policies.Azure Sentinel Deployment Best PracticesLeveraging native integrations with MicrosoftDefender tools and Azure services such asLog Analytics and Logic Apps for analysis andautomation capabilities, Azure Sentinel allowsorganizations to ingest, correlate, and analyzesecurity signals from across the enterprise.The ability to leverage elastic compute andstorage capabilities inherent in Azure for dataintensive applications such as SIEM is a significantadvantage over premise-based log analysissolutions. Additionally, Azure Sentinel can makeuse of infrastructure as a service (IaaS) andplatform as a service (PaaS) available in Azureto deliver capabilities like workflow automationand long-term log retention that are typicallyprovided as add-on services from otherSIEM providers.Azure Sentinel unifiedintegrationAzure Sentinel integrates with Microsoft 365Defender and Azure Defender to provide aunified way to manage risk in your digitallandscape under a single umbrella. Incidents,schema, and alerts can be shared between AzureSentinel and Microsoft 365 Defender, providing aholistic view with seamless drill down for context.24

SEIM Azure SentinelIdentitiesEndpointsAppsSQLEmailDocsCloud AppsNetworkTrafficMicrosoft 365 DefenderServerVMsIndustrialIoTContainersAzure AppServicesAzure DefenderXDR Microsoft DefenderSIEM Azure SentinelVisibility across your entire organization3PreventProtectMicrosoft365 DefenderAzure DefenderSecure your infrastructureSecure your infrastructureXDRAzure Sentinel Deployment Best Practices5

Cloud SIEM ArchitectureWe take a two-sided view to the Azure Sentinelarchitecture. The first is the SIEM solution wheresecurity information and events are processedand analyzed. The second includes the multitudeof data sources themselves. In our experience,addressing both the setup and operation of theSIEM solution, as well as a thoughtful approachto the data sources themselves, is critical to thesuccess of any SIEM project.Here we will look at both key aspects of SIEMarchitecture and at the considerations thatorganizations can take when approachinga project.Core Azure Sentinel solutioncomponentsIn this section, we provide guidance ondeployment of the core Azure Sentinelsolution components to be deployed in yourAzure subscription.Azure Log Analytics workspaceThe first deployment prerequisite of AzureSentinel is a Log Analytics workspace where allingested data will be stored. A Log Analyticsworkspace is created within a specific Azureregion and has a configurable retention period,defining how long data will be stored within theLog Analytics workspace (database). The defaultis 30 days, but this can be configured to as longas 730 days (2 years).Azure Sentinel Deployment Best PracticesVarious forms of data may be ingested into theLog Analytics database. Data sources include awide variety of structured data such as systeminformation from Azure Monitor Agents (AMAs)or Microsoft Monitoring Agents (MMAs) installedon Windows or Linux network endpoints,4application programming interface (API)integrations, and Azure PaaS services.Log Analytics is a component of overall AzureSentinel cost and is calculated based on thevolume of ingested data and the data retentionperiod. Special consideration should be paid tothe extended retention period, as certain eventtables might only contain system performancemetrics or verbose logging of services, which maynot be ideally suited for analysis within an SIEMsolution. Data unrelated to security monitoringmay not be worth storing over a long periodof time when balanced against ingestion costs.Conducting a thorough analysis of the incomingdata and aligning to organizational compliancepolicies will determine if raw data must be keptonline in Log Analytics or if alternative storageoptions are possible. Alternate solutions existwithin the Azure ecosystem to store raw data incheaper storage options, where required.6

There are a few initial best practices to followwhen configuring Azure Log Analytics for usewith Azure Sentinel:In multi-region architectures, deploy your LogAnalytics workspace in an Azure region thatwill minimize the egress cost of data transferbetween regions. In complex architectureswith multiple Azure Sentinel instances, initialconsideration should be paid to the regionwhere most data are produced and consumedto avoid data export charges when providingAzure Sentinel with data from disparate Azureregions. In most cases, data export chargesbetween Azure regions are usually lower thanthe price difference for Log Analyticsbetween regions. Export charges betweenregions for Azure resources are only applicableto IaaS services (virtual machines [VMs]) andnot to Azure PaaS services.Limit the number of Log Analytics workspaces,where possible. Understanding the relationshipbetween security and operational data early inthe project, and how each will be ingested, cansave data ingestion charges at later dates.Implement a comprehensive role-based accesscontrol (RBAC) strategy for Log Analytics accessearly in the project.storage capabilities of Azure Cloud. As such, itcan dynamically scale on demand to meet eventhe most demanding data ingest requirements.For larger enterprises—organizations thatsee more than 1 TB/day—Microsoft offers anoptional dedicated cluster for Azure Sentinelwithin Azure’s infrastructure. This can improvesearch performance and, depending on yourconfiguration of Azure Sentinel workspaces, canprovide cost savings and efficiencies.5For organizations that need to keep dataavailable for longer than 90 days in a costeffective storage repository while still beingable to perform real-time queries or KustoQuery Language (KQL), there is the option touse ADX, which is a big data analytics platformthat is highly optimized for all types of logs andtelemetry data analytics. It provides low latency,high throughput ingestions with fast queriesover extremely large volumes of data. It is featurerich in time series analytics, log analytics, fulltext search, advanced analytics visualization,scheduling, orchestration, automation, and manymore native capabilities.Learn more about Microsoft ADX here: er/Configure Azure Sentinel analytic rules formonitoring various parameters related todata ingestion and costs. Often analytic rulerequirements are built based purely on securityoperations needs; however, analytic rules arepowerful and can be configured to performmonitoring on operational aspects of AzureSentinel itself.Data requiring longer retention periods can bestored in alternative solutions, such as AzureData Explorer (ADX) or Azure Blob Storage.Azure Sentinel benefits from the inherent elasticAzure Sentinel Deployment Best Practices7

Case study–Global retailerACME Corporation, a large retailer, is currently using Azure Sentinel SIEM as its core cybersecurityanalytics monitoring tool. There are several log sources running in Azure Cloud, including Azure PaaSand IaaS resources, on-premises infrastructure, and numerous SaaS applications used by the financedepartment. The current volume of ingested log is 125 GB/day and ACME Corporation is using capacityreservation for 100 GB/day for both Log Analytics and Azure Sentinel.ACME Corporation is subject to Payment Card Industry (PCI) data security standard (DSS) regulatorycompliance and, therefore, has a log retention requirement of 90 days online and 1 year offline.While investigating other options for extended retention, the company decided to extend the onlineretention period in Log Analytics to 365 days, paying an additional 3,500/month. Based on East U.S.Azure Region, the company currently pays a combined fee of 15,515/month.Availability of less costly storage options such as Azure Blob Storage or Cosmos DB are good ones toconsider to meet compliance requirements. Our experience from performing cost analysis exercisesshows that most organizations below 100 GB/day of data ingestion often choose to retain data inLog Analytics, primarily to maintain advanced security capabilities present in Log Analytics and AzureSentinel. Capacity reservations are of great benefit once your organization is beyond 100 GB/day, butfor larger ingestion and retention requirements, alternative storage options should be considered.Azure SentinelWith Log Analytics deployed, the Azure Sentinelresource is available for configuration to performSIEM functions. We will cover Azure Sentinel itselfin greater depth further in this whitepaper.Azure Logic AppsAzure Logic Apps provides security orchestrationand automated response (SOAR) capabilitiesin Azure Sentinel. Azure Logic Apps power“playbooks” and are, effectively, a sequenceof procedures that can be run in response to asecurity alert. Playbooks can help automate andorchestrate response actions that would typicallybe undertaken by security analysts. These can betriggered manually or set to run automaticallywhen specific alerts are triggered.Azure Sentinel Deployment Best PracticesAzure Logic Apps is a great beneficiary ofthe capabilities of elastic compute and usesthe power of the Azure Cloud platform toautomatically scale and meet demand—youdo not have to worry about the complexity ofinfrastructure capacity, hosting, maintenance,or availability for your workflows. It is highlylikely that if an organization has workloads inAzure Cloud, Logic Apps are already used inautomations for other services.Azure Logic Apps comes with many different outof-the-box connectors that enable organizationsto easily create Azure Sentinel playbooks forautomated workflows.6 Azure Logic Apps pricingstructure is based on the number of transactions(or executions) and the type of connector.8

As a Microsoft Azure Cloud service, Azure LogicApps runs under a consumption-based pricingand metering model. This means that the fees arerelated only to how many workflow actions AzureLogic Apps execute.The monthly price for deploying and using LogicApps to orchestrate security event response isgenerally not an significant factor in the total costof running Azure Sentinel. The versatility providesorganizations with a wide range of options forreporting, alerting, and orchestration involvingAzure Sentinel alerts. Other enterprise-gradeconnectors for non-security applications cancome with higher cost price tags, so evaluationon the volume of playbook runs should beundertaken before deployment.Case study–Community collegeEDU Research, a community college with 9,000 employees, is currently using Azure Sentinel SIEM forsecurity monitoring and automated security incident response. The following Azure Sentinel playbookswere configured using Azure Logic Apps, as part of EDU Research Sentinel alert rules response:Azure Usage runs a daily query in the Log Analytics usage table and sends an emailnotification to the EDU SOC team with aggregate daily Azure ingest costs per log source.Incident Notification runs when a Sentinel alert triggers and automatically opens aServiceNow ticket with incident details.Health Monitoring runs when an Azure Sentinel playbook fails and sends an emailnotification to EDU SOC team.Running in the East U.S. Azure Region, EDU Research pays a total monthly aggregated fee for AzureLogic Apps of 95.00, which includes standard connector actions, built-in actions, and data retentionfees. Relative to the value produced by automating response actions, the cost for building securityworkflow automations in Logic Apps is often negligible and a powerful way to drive business value froman Azure Sentinel deployment.Azure Sentinel Deployment Best Practices9

Data sourcesWe regularly encounter a commonmisconception among security executives andpractitioners that Azure Sentinel can only be usedfor Azure Cloud resources. In fact, Azure Sentinelis successfully used to ingest and correlate datafrom a wide range of log sources located ina variety of cloud platforms (Azure, AmazonWeb Service [AWS] and Google Cloud), onpremises network and compute infrastructure,3rd party security tools, or software as a service(SaaS) applications. In addition to the growingset of out-of-the-box data connectors, theAzure Sentinel public community is regularlydemonstrating new use cases and dataconnectors that expand the capabilitiesof the solution.Case study–Transportation companyWheels Transportation is a transportation and logistics company that has been in business for over100 years. At the time of initial engagement, Wheels did not have any workloads in Azure, includingcommon productivity tools such as Microsoft Office 365. Infrastructure was primarily on premise,including servers and network infrastructure.The security operations team at Wheels had been using an on-premises SIEM solution that requiredregular capital expenditure on storage and server hardware, which had to be aligned to predictedevents per second (EPS) ingestion requirements several years into the future. This EPS forecast was oftenmissed, resulting in unbudgeted purchases of additional hardware and log ingestion licenses.Wheels chose to deploy Azure Sentinel as its first Azure resource and primary SIEM solution because ofthe ability to operationalize log ingestion costs using public cloud. Although Wheels did not have anyexisting Azure Cloud infrastructure, the flexibility of the Azure Sentinel SIEM solution was the right fit.Azure Sentinel Deployment Best Practices10

The wide variety of potential data types as log sources means that the consideration paid to eachdifferent data type is important at the outset of an Azure Sentinel project. Azure Sentinel includesmore than 100 connectors, out of the box, with the ability to create custom sources to meet individualrequirements. We have collected a summary table of some of the more common data source types, withexperiential commentary relevant for deployment teams configuring new data ingest sources.To learn about the more than 100 connectors included with Azure Sentinel, go el/connect-data-sources.Log ingestion methodTypical log sourcesMicrosoft native dataconnectorsBuilt-in Azure Sentinel dataconnectors for Microsoft nativelog sources, such as AzureServices (e.g., Active Directory,distributed denial of service[DDoS] protection, KubernetesService, Web ApplicationFirewall [WAF]), Dynamics 365,Microsoft Office 365, MicrosoftDefender security services.Several of these data sourcesare free, such as alerts fromMicrosoft 365 Defender orAzure Defender, while additionaladvanced hunting logs are paid.Windows and Linux machinesdeployed on-premises or in anyother cloud environments.Windows events are typicallyvery noisy, and it will requirean advanced MMA/AMA agentconfiguration. Also filteringspecific Windows event IDslevel (done via Group Policyconfiguration) may be requiredon the source servers or via theAMA agent configuration.Log Analytics agentMicrosoft Internet InformationServer (IIS) Web Servers logs canbe collected via this agent.Any logs from other applicationsrunning on the same machinewhere MMA agent is running.This is collected via MMACustom Log settings.Azure Sentinel Deployment Best PracticesExperienceAzure Sentinel can nativelyingest security alerts fromDefender for Identity, Defenderfor Office 365, Defender forEndpoints, and Microsoft cloudapp security.Windows PerformanceCounters can be collected viathis agent, too. Depending onthe sample rate interval, thiscan increase the volume ofcollected events, increasing theoverall Azure consumption11

Syslog log forwarderFirewalls, intrusion preventionsystems (IPSs), L2/L3 networkdevices, and others.Syslog data is collected in Syslogformat and will require creationof log parsers in Azure Sentinel.All Syslog messages are storedin a single Log Analytics table(Syslog table).This method does not allow anycontrol on the volume or type oflog ingested.Common event format(CEF) log forwarderLogic App playbooksSome type firewalls, SoftwareDefined Wide Area Network(SDWAN), and Secure AccessService Edge (SASE) platformsCEF has a standard schemaused by many security vendors,allowing interoperabilityamong different platforms.CEF is an industry standard format.Microsoft provides the installationscripts and documentation for aLinux agent that can be deployedon the same machine where aSyslog agent runs.It does not require additionallog parsingPaaS and SaaS applications.Using remote application RESTAPI calls, the data connectorscan be set up to extract specificevents only, which is an effectivetool for log optimization.An Azure Logic Apps playbookusing a REST API call can be usedto pull events from an applicationor tool.Azure Sentinel Deployment Best PracticesMany platforms, like firewalls,allow customization of CEFtemplates, which is a greattool to optimize the volume ofingested logs at thesource point.12

This method is typically used forSaaS applications. Data is ingestedin Azure Sentinel Log Analyticsworkspace and is placed in acustom table.Log ingestion is based on“pull,” within a predefinedtime interval (no real-timecapabilities).This relies on availability ofAzure Sentinel playbooks;therefore, additionalmonitoring is required forplaybook health and status.The customer has controlover Log Analytics tableschema definition.REST API(Azure function)SaaS applications.This method requires customdevelopment using remoteapplication REST APIs and Azurefunctions.Customer does not need to run aseparate machine/VM.Logs are ingested in a LogAnalytics custom table.Log optimization is dependent onremote application REST API.This is the Microsoftrecommended method for logcollection from customlog sources.Logstash collectorFirewalls, IPS, network devices,and others.A Logstash collector needs tobe deployed on-premises or in acloud environment on a VM.Data enrichment (such asgeo-location) can be done oncollection point.This allows log optimization, bycollecting only requiredlog fields.Once ingested in Azure SentinelLog Analytics workspace, datawill be stored in a custom table.Azure Sentinel Deployment Best Practices13

Many parsers and dataenrichment tools areavailable in open-sourceElasticsearch-LogstashKibana (ELK) community.Microsoft recentlyannounced that the Logstashcollector method will besupported as well.Azure diagnosticsAzure PaaS resources.Not always considered a separatelog ingestion method, collectingevents via Azure Diagnosticsis applicable to Azure PaaSresources only.Turning Azure Diagnostics forAzure PaaS resources, the auditlogs and metrics events can becollected and stored in an AzureDiagnostics Log Analytics table.The customer must createlog parsers for eachresource type.Data ingested via AzureDiagnostics is very noisyand will increase the overallAzure consumption.No data optimization canbe done via this method.Microsoft provides in AzureSentinel a set of standarddata connectors (e.g.,Azure Key Vault, AzureKubernetes) for a few PaaSresources.If the customer does nothave any compliancerequirements for logretention, using AzureDefender for monitoringand protecting Azure PaaSresources, in general, is amore cost-effective solutionfor this situation.Azure Sentinel Deployment Best Practices14

File-based ingestionAmazon Web ServicesUsed for ingestion of datafrom files located on the samemachines where Log AnalyticsAgent is running. This option usesthe Custom Logs configuration ofthe Log Analytics Agent.Logs are collected in a LogAnalytics custom table ( CL).Microsoft provides an AWSCloudTrail Azure Sentinel dataconnector out of the box.Limited to AWS CloudTrailonly (default).Collecting events from other AWSresources, a REST API function canbe developed.Threat intelligenceplatforms: StructuredThreat InformationExpression (STIX) andTrusted AutomatedeXchange of IndicatorInformation (TAXII)Available from external ThreatIntelligence Platforms and fromSTIX/TAXII feeds, both open sourceand premium, these can enrichAzure Sentinel with additionalindicators of compromise (IoC)such as known malicious internetprotocols (IPs) or domain nameservice (DNS) names.Using the Log AnalyticsAgent configuration tool, aspecific record delimiter anda path to the file can be used.Data is placed in AWSCloudTrail table, and it isalready parsed.Employ either the Azure SentinelThreat Intelligence Platforms dataconnector or Azure Sentinel TAXIIdata connector.This requires one or more externalThreat Intelligence Platforms and/or STIX/TAXII feeds.STIX/TAXII is an ingest protocolunique to importing securityintelligence data.Microsoft provides a set of guidelines for vendors and Azure Sentinel community for development ofnew data connectors.Azure Sentinel uses Azure Log Analytics as the backend for the log storage and querying capabilitiesthrough KQL. A wealth of information is available from various log sources stored in Log Analyticsas “tables.” There are a variety of default tables, though not always populated with data while otherscreated as specific Azure Sentinel data connectors are enabled and configured. Another range of tables,not covered in the list following, are represented by custom logs that can be used to ingest logs fromcustom applications that fall outside the scope of standard SIEM log sources.Azure Sentinel Deployment Best Practices15

Azure SentineltableDescriptionLog sourcesRelevant dataAuditLogsAzure Active Directoryactivities audit such ascreation/modification ofusers, groups, applicationsAzure dTrailAWS CloudTrail log ureActivityAzure activity such as creation/modification/deletion of Azureresources, policy age of diagnostic logs forAzure resources (resourcesmust be configured to sendthe diagnostics logs tothe specific Log aYesAzureMetricsProvides storage of metricsrecorded by var

Case study-Global retailer ACME Corporation, a large retailer, is currently using Azure Sentinel SIEM as its core cybersecurity analytics monitoring tool. There are several log sources running in Azure Cloud, including Azure PaaS and IaaS resources, on-premises infrastructure, and numerous SaaS applications used by the finance department.