WIXLIB

Monitoring Cisco ACI FabricsWith APCON Network Visibility SolutionsApplication Centric InfrastructureCisco’s Application Centric Infrastructure (ACI) provides network anddata center architects/operators with a new level of automationand scale. While ACI provides a new approach to network-widemanagement and policy, it also introduces underlying protocolsthat may affect monitoring capabilities. This overview provides aframework of monitoring options, along with insights to leveragingAPCON’s network visibility solutions, to maximize ROI investment ofexisting network/security tools.APCON has developed comprehensive solutions for insight intophysical, virtual and cloud networks. IntellaFlex XR is a scalablesolution that can accommodate evolving network fabrics andhigher port count environments while simultaneously providingpacket processing functions to allow organizations to maintainROI on their existing monitoring and security tool investments.This Technical Brief will highlight options for a holistic monitoring ofCisco ACI environments using APCON network visibility solutions.Combining the new concepts within Cisco ACI fabrics with APCON’stool optimization features creates a comprehensive networkmonitoring solution.Concepts covered in this document include: Cisco ACI concepts/componentsMonitoring options using APCON network visibility solutionsAPCON Tool Optimization featuresIntegrated APCON capture/VM analysis optionsCisco ACI concepts and componentsCisco ACI abstracts underlying component configuration viaApplication Network Profiles. Policies define interaction betweenApplication Profiles and End Point Groups. In a leaf-spine fabric,routing is enabled between any two endpoints. In addition,overlay protocols, such as virtual extensible local area network(VXLAN), allow workloads to exist anywhere in the network. Formanagement, the Application Policy Infrastructure Controller(APIC) manages and configures policy switches in the ACI fabric.The APIC is a central control point for all policies and can rapidlyprovision or reconfigure hardware as needed.SOLUTION BRIEF

The APIC provides a conceptual representation of the entire fabric as a single entity to user space endpoints. Pictured below is a topologyrepresentation of a Cisco ACI fabric including Spine/Leaf nodes and APIC controller.Nexus 9000Spine/Leaf Nodes40G/100GFabric InterconnectsInfrastructure SpaceUser SpaceClustered Application Policy Infrastructure Controller (APIC) appliancesCisco ACI Traffic MonitoringFor networking hardware, the following componentsare needed to implement a Cisco ACI fabric: Nexus 9000 series switches running in ACI modeApplication Policy Infrastructure Controller (APIC)deployed in clustered configurationTopology features following physical and virtual constructs: The physical Spine and Leaf fabric architectureThe ACI VXLAN overlay, which enables the decouplingfrom the physical network and the creation of virtualized L2segments regardless of the endpoint locationCisco ACI presents network architects and operators with new levelsof scale, automation and ease of deployment. Understanding themonitoring options is important to complement these deploymentinnovations to attain comprehensive monitoring. Below is a highlevel overview of how an APCON visibility solution can be used tocomplement a Cisco ACI deployment with conceptual options forgaining access to monitor feeds and data. Subsequent sections willcover these in more detail.Topics will include: Cisco ACI SPAN typesAPCON use cases for capturing Cisco ACI SPAN trafficTAP optionsCisco ACI Copy ServicesNetFlow GenerationERSPAN Type I / ERSPAN Type IITunnel EndpointTAPACI ERSPANAVSI NTELLA F LEX Hyper EnginePacket ProcessorACI–3072–XR240 Gbps Ethernet110.1.102.72 / 255.255.0.026.7ºcJJ3072-XRS/N: 72020004Ver: 157911DOWN ENTERINTELLAFLEX BladeACI-3030-E32-724681012ACI Local Access SPAN40G BiDi TAP131517192123141618202224252729312628303240 Gbps Ethernet310 Gbps EthernetUP110 Gbps EthernetHit [Enter] for configurationCANCELStatusBPowerINTELLAFLEX BladeACI-3033-E02-1Packet Aggregator10 Gbps / 40 GbpsStatusPowerA

Within a Cisco ACI fabric, TAP points between the spine and the leafswitches is an option to consider with additional setup covered laterin this paper. It is important to note that TAP points will likely involve40G BiDi optics, which are supported by APCON TAPs and 40GQSFP ports. Additionally, encapsulation and virtualization conceptsare important to consider. A Cisco ACI Nexus fabric will normalizeall traffic between leaf and spine through the encapsulated Cisco(VXLAN) protocol. APCON network visibility solutions have beendeveloped to optimize tool performance by factoring VXLAN indeduplication hashing scheme and decapsulating VXLAN feeds.While previous monitoring concepts of TAP or SPAN rule of thumbstill stand, it is important to consider some of the new concepts forCisco ACI SPANs.SPAN Types in Cisco ACIFrom an APCON monitoring setup perspective, it is important tounderstand encapsulated remote extension of SPAN (ERSPAN)type based on the SPAN type selected. The three SPAN options inCisco ACI environments are as follows:Access SPAN – Mirrors all traffic to and from leaf host portslocally with source and destination on the same leaf switch oracross multiple leaf switches with a remote destinationTenant SPAN – Mirrors all traffic to and from EPGsassociated to a common tenant to a remote destinationFabric SPAN – Mirrors all traffic to and from a spine switch toa remote destinationSPAN concepts in Cisco ACICisco ACI has introduced a new logical networking concept:Endpoint Group (EPG) for mapping applications to the network.EPGs act as a collection of applications and components forforwarding and policy definition. This is a key concept in enablingdynamic network provisioning as EPGs consume hardwareresources only when member endpoints (tenants) are present.EPGs will expand or contract in real-time as tenants andworkloads move around a datacenter.Access, Tenant and Fabric SPANs use the encapsulated remoteextension of SPAN (ERSPAN) Type I, while Fabric SPAN uses ERSPANType II. APCON supports all Cisco ACI SPAN types. Configurationof these SPAN and ERSPAN instructions can be found in theConfiguring SPAN chapter of the APIC NXOS CLI User Guide.APCON platforms enable compatibility with all available SPANand overlay options in a Cisco ACI environment. The Cisco ACISPAN options offer different levels of visibility. These factors aresummarized below.Tenant SPANFabric SPANAccess SPAN Aggregates SPAN sessions acrossmultiple switches Source must be fabric port Source must be host port Mirrors traffic to/from Spine switches Mirrors traffic to/from Endpoints(Leaf switch host ports) Mirrors traffic to/from specifiedEndpoint Group (EPG) ERSPAN only ERSPAN only ERSPAN Type II encapsulation ERSPAN Type I encapsulation Supports aggregation of multipleswitches No filtering possible Filterable by private networkor bridge domain Local SPAN or ERSPAN ERSPAN Type I encapsulation Supports aggregation of multipleswitches Filterable by tenant, applicationprofile,or EPG

For Cisco ACI environment, the following shows conceptual setup in configuring SPAN source from within a Cisco ACI environment viaAPIC and configuring APCON installation to receive and decapsulate this feed.Configuring SPAN sessions from Cisco ACI deploymentsCisco ACI SPAN sessions utilize ERSPAN Type I & II for exportand can be terminated on HyperEngine or IntellaStore.I NTELLA F LEX 240 Gbps Ethernet1JJ911DOWN ENTERINTELLAFLEX 8202224252729312628303240 Gbps Ethernet710 Gbps EthernetUP510 Gbps EthernetHit [Enter] for configurationCANCEL3Access SPANAccess portTenant SPANEndpoint groupVirtual SPANVirtual machine –interfaceDestinationRemote (ERSPAN Type II)Remote (ERSPAN Type I)Remote (ERSPAN Type I)Remote (ERSPAN Type I)BPowerINTELLAFLEX BladeACI-3033-E02-11Filter Bridge domain Private network Tenant Application profile Endpoint group–Hyper EnginePacket ProcessorACI–3072–XR10.1.102.72 / 255.255.0.03072-XRS/N: 72020004Ver: 1SourceFabric portACI SPAN sessionscentrally configuredon APIC ControllerACI-3033-E02-1 HyperEngine26.7ºcSPAN TypeFabric SPANPacket Aggregator10 Gbps / 40 GbpsStatusAPowerTerminate up to 16 sessions on HyperEngine,up to 200Gb/s throughputImplementing APCON Network VisibilityCisco SPAN Guidelines and RestrictionsReceiving SPAN Types on APCON XR PlatformThere are important configurations when setting up SPANmonitor feeds on Cisco ACI environments.Tenant, Fabric, or Access SPANs are centrally configured onCisco APIC. If configuring ERSPAN Type I or II, this will requirea destination IP address set on an APCON port. APCONsupports ERSPAN decapsulation options on the HyperEngineor IntellaStore II blades. The HyperEngine blade terminatestunneled traffic as required by Cisco ACI and virtual networkenvironments. This includes support for decapsulation of GRE,NVGRE, VXLAN, GENEVE and ERSPAN Types I, II and III feeds forup to 200Gbps of tunneled traffic per blade. SPAN traffic competes with user traffic for switch resources.To minimize the load, configure SPAN to copy only thespecific traffic that you want to analyze. A SPAN source will take entire port for monitoring trafficfrom external sources. Tenant and Access SPANs use the encapsulated remoteextension of SPAN (ERSPAN) Type I, while Fabric SPAN usesERSPAN Type II. ERSPAN destination IPs must be learned in the fabric as anendpoint. SPAN supports IPv6 traffic but the destination IP for theERSPAN cannot be an IPv6 address.Refer to Cisco APIC Troubleshooting Guide for more information.

ERSPAN Type I / ERSPAN Type IITunnel EndpointACI ERSPANAVSHyper EnginePacket Processor240 Gbps Ethernet110.1.102.72 / 255.255.0.026.7ºcJJBladeACI-3033-E02-1UPDOWN 33352123121416182022111315171921GPSANTDMulti Function1/10 GbpsStatusCPower1/10 Gbps Ethernet1/10 Gbps EthernetGPSANTMulti Function1/10 303234361/10 Gbps Ethernet411/10 Gbps Ethernet2PPS/IRIGINOUTINTELLAFLEX BladeACI-3032-E36-13119I NTELLA F LEX ACI–3144–XR2917PPS/IRIGINOUTINTELLAFLEX BladeACI-3032-E36-12715PPS/IRIGINOUTINTELLAFLEX BladeACI-3032-E36-125131/10 Gbps EthernetHit [Enter] for configuration11/10 Gbps Ethernet3144-XRS/N: 72020004Ver: 1CANCELStatusPowerINTELLAFLEX GPSANTMulti Function1/10 GbpsStatusAPowerDeployment Options: ACI ERSPAN DeploymentFor ERSPAN deployments, one or more IP addressable ports will be exposed to Cisco ACI fabric and connected to APCON installationwith the HyperEngine or IntellaStore II blade. The blade will provide function to set the IP destination address and decapsulateappropriate ERSPAN Type feed. SPAN feed will be configured from Cisco ACI environment. Once set, defined traffic from anywhere inthe fabric will be sent to set APCON destination port.From within APCON’s WebXR GUI for the HyperEngine, service point would be set to “Tunnel Termination” option with appropriate IPaddress and Type I or Type II De-Encapsulate option.APCON Network Visibility SolutionACI-3033-E02-1: HyperEngine Blade- ERSPAN Type I & Type II TunnelTermination- Deduplication / VXLAN Protocol Stripping- NetFlow Generation- Deep Packet Inspection/Masking- 40G Ingress with BiDi SupportACI-3032-E36-1: Multi Function Blade- Aggregation and filtering functions- Tool Optimization features includingpacket slicing, deduplication, protocolstripping and time stampingACI-3030-E20-1: High Density 40G Blade- 20 40G ports with BiDi Support