WIXLIB

Monitoring Cisco ACI FabricsWith APCON Network Visibility SolutionsApplication Centric Infrastructure (ACI)Cisco’s Application Centric Infrastructure (ACI) provides network anddata center architects/operators a new level of automation and scale.With ACI, there are new approaches to network-wide managementand policy along with underlying protocols that affect monitoringoptions. This overview provides a framework of monitoring optionsalong with insights to leveraging APCON’s network visibility solutionsto maximize ROI investment of existing network/security tools.Concepts covered in this document include:APCON has developed solutions for a comprehensive approach togain insight into physical, virtual and cloud networks. The scalablefamily of IntellaFlex XR systems can accommodate evolving networkfabrics, such as the Cisco ACI fabric and the need for higher portcount 40G and 100G environments all while providing packetprocessing functions to allow organizations to maintain ROI on theirexisting monitoring and security tool investments.Cisco ACI abstracts underlying component configuration viaApplication Network Profiles. Policies define interaction betweenApplication Profiles and End Point Groups. Implemented in a leafspine fabric, routing is enabled between any two endpoints. Inaddition, overlay protocols, such as virtual extensible local areanetwork (VXLAN) allow workloads to exist anywhere in the network.For management, the Application Policy Infrastructure Controller(APIC) manages and configures policy switches in the ACI fabric. TheAPIC is central control point for all policies and can rapidly provisionor reconfigure hardware as needed.This Technical Brief will highlight options for holistic monitoring ofCisco ACI environments using APCON network visibility solutions.Leveraging new concepts within Cisco ACI fabrics and combinedwith packet processing, capture, VM analysis and tool optimizationfeatures on the APCON platform allows best of breed capabilities forholistic network monitoring. Cisco ACI concepts/componentsMonitoring options using APCON network visibility solutionsAPCON Tool Optimization featuresIntegrated APCON capture/VM analysis optionsCisco ACI concepts and componentsSOLUTION BRIEF

The APIC provides conceptual representation of the entire fabric as a single entity to user space endpoints. Pictured below is a topologyrepresentation of a Cisco ACI fabric including Spine/Leaf nodes and APIC controller.Nexus 9000Spine/Leaf Nodes40G/100GFabric InterconnectsInfrastructure SpaceUser SpaceClustered Application Policy Infrastructure Controller (APIC) appliancesCisco ACI Traffic MonitoringFor networking hardware, the following componentsare needed to implement a Cisco ACI fabric:Cisco ACI presents network architects and operators with new levelsof scale, automation and ease of deployment. Understanding themonitoring options is important to complement these deploymentinnovations to attain comprehensive monitoring. Below is a highlevel overview of how an APCON visibility solution can be used tocomplement a Cisco ACI deployment with conceptual options forgaining access to monitor feeds and data. Subsequent sections willcover these in more detail. Nexus 9000 series switches running in ACI mode Application Policy Infrastructure Controller (APIC)deployed in clustered configurationTopology features following physical and virtual constructs: The physical Spine and Leaf fabric architecture The ACI VXLAN overlay, which enables the decouplingfrom the physical network and the creation of virtualized L2segments regardless of the endpoint locationTopics will include: Cisco ACI SPAN typesAPCON use cases for capturing Cisco ACI SPAN trafficTAP optionsCisco ACI Copy ServicesNetFlow GenerationERSPAN Type I / ERSPAN Type IITunnel EndpointTAPACI ERSPANAVSI NTELLA F LEX Hyper EnginePacket ProcessorACI–3072–XR240 Gbps Ethernet110.1.102.72 / 255.255.0.026.7ºcJJ3072-XRS/N: 72020004Ver: 157911DOWN ENTERINTELLAFLEX BladeACI-3030-E32-724681012ACI Local Access SPAN40G BiDi TAP131517192123141618202224252729312628303240 Gbps Ethernet310 Gbps EthernetUP110 Gbps EthernetHit [Enter] for configurationCANCELStatusBPowerINTELLAFLEX BladeACI-3033-E02-1Packet Aggregator10 Gbps / 40 GbpsStatusPowerA

Within a Cisco ACI fabric, TAP points between the spine and the leafswitches is an option to consider with additional setup covered laterin this paper. It is important to note that TAP points will likely involveuse of 40G BiDi optics which are supported by APCON TAPs and 40GQSFP ports. In addition, encapsulation and virtualization conceptsare important to consider. A Cisco ACI Nexus fabric will normalizeall traffic between leaf and spine by encapsulated Cisco VirtualExtensible LAN (VXLAN) protocol. APCON network visibility solutionshave been developed to optimize tool performance by factoringVXLAN in deduplication hashing scheme and decapsulating VXLANfeeds. Previous monitoring concepts of TAP or SPAN rule of thumbstill stand. It is important to consider some of the new options forCisco ACI SPANs.SPAN concepts in Cisco ACICisco ACI introduces a new logical networking concept EndpointGroup (EPG) for mapping applications to the network. EPGs act as acollection of applications and components used for forwarding andpolicy definition. This is a key concept in enabling dynamic networkprovisioning as EPGs consume hardware resources only whenmember endpoints (tenants) are present. EPGs will expand or contractin real time as tenants and workloads move around a datacenter.There are monitoring concepts that still apply such as SPAN/ERSPAN.SPAN Types in Cisco ACIFrom an APCON monitoring setup perspective, it is important tounderstand ERSPAN type based on SPAN type selected. Below is abrief description of SPAN types along with type methodology. Thethree SPAN options are as follows in Cisco ACI environments.Access SPAN – Mirrors all traffic to and from leaf host portslocally with source and destination on the same leaf switch oracross multiple leaf switches with a remote destinationTenant SPAN – Mirrors all traffic to and from EPGs associated toa common tenant to a remote destinationFabric SPAN – Mirrors all traffic to and from a spine switch to aremote destinationAccess, Tenant and Fabric SPANs use the encapsulated remoteextension of SPAN (ERSPAN) Type I, while Fabric SPAN uses ERSPANType II. APCON supports all Cisco ACI SPAN types. Configuration ofthese SPAN and ERSPAN instructions can be found in the ConfiguringSPAN chapter of the APIC NXOS CLI User Guide.The Cisco ACI SPAN options offer different levels of visibility. Thesefactors are summarized below.APCON platforms enable compatibility with all available SPAN andoverlay options in a Cisco ACI environment.Tenant SPANFabric SPANAccess SPAN Aggregates SPAN sessions acrossmultiple switches Source must be fabric port Source must be host port Mirrors traffic to/from Spine switches Mirrors traffic to/from Endpoints(Leaf switch host ports) Mirrors traffic to/from specifiedEndpoint Group (EPG) ERSPAN only ERSPAN only ERSPAN Type II encapsulation ERSPAN Type I encapsulation Supports aggregation of multiple switches No filtering possible Filterable by private networkor bridge domain Local SPAN or ERSPAN ERSPAN Type I encapsulation Supports aggregation of multipleswitches Filterable by tenant, application profile,or EPG

For Cisco ACI environment, the following shows conceptual setup in configuring SPAN source from within Cisco ACI environment via APIC andconfiguring APCON installation to receive and decapsulate this feed.Configuring SPAN sessions from Cisco ACI deploymentsCisco ACI SPAN sessions utilize ERSPAN Type I & II for exportand can be terminated on HyperEngine or IntellaStore.I NTELLA F LEX 240 Gbps Ethernet1JJ911DOWN ENTERINTELLAFLEX 8202224252729312628303240 Gbps Ethernet710 Gbps EthernetUP510 Gbps EthernetHit [Enter] for configurationCANCEL3Access SPANAccess portTenant SPANEndpoint groupVirtual SPANVirtual machine –interfaceDestinationRemote (ERSPAN Type II)Remote (ERSPAN Type I)Remote (ERSPAN Type I)Remote (ERSPAN Type I)BPowerINTELLAFLEX BladeACI-3033-E02-11Filter Bridge domain Private network Tenant Application profile Endpoint group–Hyper EnginePacket ProcessorACI–3072–XR10.1.102.72 / 255.255.0.03072-XRS/N: 72020004Ver: 1SourceFabric portACI SPAN sessionscentrally configuredon APIC ControllerACI-3033-E02-1 HyperEngine26.7ºcSPAN TypeFabric SPANPacket Aggregator10 Gbps / 40 GbpsStatusAPowerTerminate up to 16 sessions on HyperEngine,up to 200Gb/s throughputImplementing APCON Network VisibilityCisco SPAN Guidelines and RestrictionsReceiving SPAN Types on APCON XR PlatformThere are important configurations when setting up SPAN monitorfeeds on Cisco ACI environments.Tenant, Fabric, or Access SPANs are centrally configured on CiscoAPIC. If configuring ERSPAN Type I or II, this will require a destinationIP address set on an APCON port. APCON supports ERSPANdecapsulation options on the HyperEngine or IntellaStore II blades.The HyperEngine blade terminates tunneled traffic as required byCisco ACI and virtual network environments. This includes support fordecapsulation of GRE, NVGRE, VXLAN, GENEVE and ERSPAN Types I,II and III feeds for up to 200Gbps of tunneled traffic per blade. SPAN traffic competes with user traffic for switch resources. Tominimize the load, configure SPAN to copy only the specific trafficthat you want to analyze. A SPAN source will take entire port for monitoring traffic fromexternal sources. Tenant and Access SPANs use the encapsulated remoteextension of SPAN (ERSPAN) Type I, while Fabric SPAN usesERSPAN Type II. ERSPAN destination IPs must be learned in the fabric as anendpoint. SPAN supports IPv6 traffic but the destination IP for the ERSPANcannot be an IPv6 address.Refer to Cisco APIC Troubleshooting Guide for more information.

ERSPAN Type I / ERSPAN Type IITunnel EndpointACI ERSPANAVSHyper EnginePacket Processor240 Gbps Ethernet110.1.102.72 / 255.255.0.026.7ºcJJBladeACI-3033-E02-1UPDOWN 33352123121416182022111315171921GPSANTDMulti Function1/10 GbpsStatusCPower1/10 Gbps Ethernet1/10 Gbps EthernetGPSANTMulti Function1/10 303234361/10 Gbps Ethernet411/10 Gbps Ethernet2PPS/IRIGINOUTINTELLAFLEX BladeACI-3032-E36-13119I NTELLA F LEX ACI–3144–XR2917PPS/IRIGINOUTINTELLAFLEX BladeACI-3032-E36-12715PPS/IRIGINOUTINTELLAFLEX BladeACI-3032-E36-125131/10 Gbps EthernetHit [Enter] for configuration11/10 Gbps Ethernet3144-XRS/N: 72020004Ver: 1CANCELStatusPowerINTELLAFLEX GPSANTMulti Function1/10 GbpsStatusAPowerDeployment Options: ACI ERSPAN DeploymentFor ERSPAN deployments, one or more IP addressable ports will be exposed to Cisco ACI fabric and connected to APCON installation with theHyperEngine or IntellaStore II blade. The blade will provide function to set IP destination address and decapsulate appropriate ERSPAN Typefeed. SPAN feed will be configured from Cisco ACI environment. Once set, defined traffic from anywhere in the fabric will be sent to set APCONdestination port.From within APCON’s WebXR GUI for the HyperEngine, service point would be set to “Tunnel Termination” option with appropriate IP addressand Type I or Type II De-Encapsulate option.APCON Network Visibility SolutionACI-3033-E02-1: HyperEngine Blade- ERSPAN Type I & Type II Tunnel Termination- Deduplication / VXLAN Protocol Stripping- NetFlow Generation- Deep Packet Inspection/Masking- 40G Ingress with BiDi SupportACI-3032-E36-1: Multi Function Blade- Aggregation and filtering functions- Tool Optimization features including packetslicing, deduplication, protocol strippingand time stampingACI-3030-E20-1: High Density 40G Blade- 20 40G ports with BiDi Support