WIXLIB

WHITE PAPERMellanox Seamlessly Integrates With OpenStack,Increasing Efficiency and Reducing Operational CostsHigh-speed Ethernet with hardware offloads deliver total infrastructureefficiency for NFV and Cloud Data CentersExecutive SummaryOpenStack, commonly referred to as "the Linux of the Cloud", allowscompanies to utilize open-source initiatives and a transparent andcollaborative approach to implement public and private cloud solutions toachieve business agility, infrastructure elasticity and operational simplicity.While OpenStack provides a flexible framework, it is particularly importantthat the cloud infrastructure, composed of compute, network and storageresources, runs at maximum performance and efficiency to guaranteeoverall application performance. This paper discusses the requirements forcloud network infrastructure to properly support web-scale IT with Red HatOpenStack as the cloud management platform, and tightly integrated withMellanox end to end networking solutions.Challenges to Efficient Cloud DeploymentIn order to achieve multi-tenancy and automation goals, cloud deploymentsoften leverage disaggregation and virtualization technologies. However, thiscomes at a cost of significant performance penalties which manifestthemselves as low data communication and storage access performance,and heightened CPU utilization. As a result, organizations compensate byover-provisioning CPU cores to improve the performance resulting in a largerhardware footprint and capital expenditures, thus reducing totalinfrastructure efficiency.Understanding Compute Virtualization PenaltiesIn virtualized environments, multiple virtual machine (VM) instances runsimultaneously over physical server hardware. This has necessitated virtualswitch software that often reside alongside the hypervisor in the OS kernelto handle network I/O traffic to and from VMs. While virtualization doesKey Benefits Achieve 10X Better Performance thanvanilla OVS with ASAP2 OVS Offloads Free up 100% CPU Cores with ASAP2 The Best Industry Throughput and 50%CapEx Savings for DPDK Improve Latency by 20X with SR-IOV Improve CPU Utilization by 80% withOverlay Networks Offloads Deliver 6X Throughput with RDMA Improve TCO by Marrying Bare-Metaland Virtualized OpenStack Clouds withMellanox Spectrum Switches andConnectX Intelligent Network Adapters

bring flexibility, it also results in significantly degraded I/Operformance degradation due to increased layers ofprocessing in software and the CPU is burdened with amajority of the virtual network I/O processing.(OVS), OVS over DPDK, Network Overlay Virtualization, SRIOV, and RDMA.Avoid Network Virtualization PenaltyOpen vSwitch (OVS), vRouter, VPP and Linux Bridge aresome of the most popular virtual switch platforms used inOpenStack cloud deployments today. OVS hardwareoffloads accelerate the traditional slow virtual switch packetperformance by an order of magnitude. Essentially, offeringthe best of both worlds: Hardware acceleration of the datapath (fast-path) for high-throughput flows along withunmodified standard OVS control path for flexibility andprogramming of match-action rules.Overlay tunneling protocols such as VXLAN, NVGRE orGENEV are not all recognized by every server NIC andprocessing these new packet formats without NIC hardwareoffload needs to be done by the OS kernel in CPU, resultingin lower and nondeterministic I/O performance andincreased CPU load.Solving Storage Virtualization PenaltiesThe TCP/IP protocol stack is not the most efficient to powerstorage networks. The protocols were designed toincorporate a lot of handshakes between endpoints, and it isalmost impossible to offload all protocol handling operationsinto the NIC hardware. As a result, complex protocolsoftware needs to run in the CPU. These can result in lowstorage access bandwidth, low IOPS, and high CPUoverhead.To overcome these penalties and achieve ultimateinfrastructure efficiency and application performance, cloudoperators are looking to implement efficient virtual networksolutions that provide excellent virtualization and bypass theTCP/IP stack in processing storage IO to achieve accelerationand efficiency in cloud networks.Mellanox OpenStack Cloud NetworkSolutionThrough an end-to-end suite of interconnect products ofadapters, switches, cable/optics, and associated networkdriver and management software, Mellanox enables clouddata centers to achieve the highest efficiency through ahigh-performance, low-latency cloud network with richnetwork offload with acceleration and automation features.Mellanox can mitigate the above-mentioned penalties,delivering cloud networks that can handle line-rateprocessing at 10, 25, 40, 50, and 100Gb/s speeds,supporting high-throughput, high-IOPS storage operations,with minimal CPU overhead so that infrastructure resourcescan be dedicated to actual application workload.Increase OVS Efficiency with ASAP2Mellanox offers an open source and high performance OVSoffload solution called Accelerated Switching and PacketProcessing (ASAP2). ASAP2 fully and transparently offloadsnetworking functions such as overlays, routing, security andload balancing to the adapter’s embedded switch (e-switch).ASAP2 provides throughput of 66Mpps for small packets and line rate performance for large packets while completelyfreeing up the CPU cores.ASAP2 is a significant improvement over traditionalapproaches because it ensures that SDN and networkprogrammability capabilities are maintained, and at thesame time, network I/O achieves highest performance oncompute nodes. With virtual switch/router being utilized inalmost all production OpenStack deployments, it makessense to use ASAP2 with any virtual switch/routerimplementation to give a tremendous performance boost interms of higher packet throughput and lower latencies, andimprove cloud efficiency. Mellanox ASAP2 is fully integratedwith RHEL 7.5 and Red Hat OSP 13.The Best Performance for DPDKOvercoming PenaltiesData Plane Development Kit (DPDK) reduces overheadcaused by interrupts that are sent each time a new packetarrives for processing. DPDK implements a polling processfor new packets to achieve the key benefits of significantlyimproving processing performance while eliminating PCIoverhead and maintaining hardware independence.Although DPDK technology consumes CPU cycles, MellanoxConnectX-5 adapters offer the industry’s highest bare-metalpacket rate of 139 million packet per second for runningOVS or VNF cloud applications over DPDK. Mellanox DPDK isfully Red Hat supported for RHEL 7.5 and OSP 13.Mellanox provides the foundation for efficient cloudinfrastructure through disaggregation and virtualizationsolutions that mitigate performance penalties associatedwith compute, network, virtualization and storage. Thisenables cloud applications to run at the highest performanceand efficiency. Mellanox achieves higher cloud efficiencythrough the following solutions; Open vSwitch OffloadsMellanox started offloading VXLAN protocol processing tothe NIC since the ConnectX-3 generation of NICs. The VXLANOffload feature enables the NIC to handle statelessprocessing of VXLAN packets such as checksum calculation,Network Virtualization with VXLANOffload, and VTEP Gateway

(100GbE/IB)Figure 1. Comprehensive OS Integration for Mellanox Switch and AdapterReceive Side Scaling (RSS), Large Segmentation Offload(LSO), etc, significantly improving throughput and latencyperformance, and reducing CPU overhead associated withoverlay packet processing. In addition to VXLAN, MellanoxNICs also support offload of other overlay encapsulationprotocols such as NVGRE and GENEVE.Oftentimes VXLAN networks needs to communicate withother networks such as VLAN networks that supports baremetal servers, or wide area networks for data centerinterconnect, and North-South user traffic. This necessitateVXLAN Tunnel Endpoint (VTEP) gateway which allow toconnect VXLAN networks to other type (i.e., VLAN) ofnetworks. Mellanox Spectrum switches support VTEPgateway functionality in hardware, ensuring highestperformance when heterogeneous networks in the cloudcommunicate with each other.Overcome Compute VirtualizationPenalty with SR-IOVSingle Root I/O Virtualization (SR-IOV) allows a device, suchas a network adapter, to separate access to its resourcesamong various PCIe hardware functions. This allows trafficstreams to be delivered directly between the virtualmachines and their associated PCIe partitions, givingapplications direct access to the I/O hardware. As a result,the I/O overhead in the software emulation layer iseliminated. SR-IOV enables VMs to achieve networkperformance that is nearly the same as in non-virtualizedenvironments.Mellanox NICs support basic SR-IOV as well as advancedfeatures such as SR-IOV High Availability (HA) and Quality ofService (QoS). SR-IOV HA provides a redundancy mechanismfor VFs by using Link Aggregation Group (LAG) to bind twoVFs from two different port on the same NIC together, andexposes the bundle as one VF to the VM. When one VF inthe bundle fails, the other VF continues forwarding trafficwithout affecting VM I/O operations.Overcome Storage VirtualizationPenalty with RDMA/RoCEThe large overhead associated with stateful protocols suchas TCP dictates that it is not an ideal transport protocol forsoftware defined scale-out storage applications, especiallywhen storage media gets faster as it transitions from harddisks to solid-state drives (SSD) to Non-Volatile Memory(NVM). Remote Direct Memory Access (RDMA), on the otherhand, is a protocol designed for high-speed links within datacenter environment that can overcome the inefficiencies ofTCP. RDMA can run over InfiniBand (IB) or over ConvergedEthernet (RoCE). RDMA kernel bypass, read and writenetwork semantics and full transaction offload to RDMAcapable NIC devices guarantee the highest possiblethroughput, lowest latency, and minimal CPU overhead,making it ideal for storage access. Typically RDMA overConverged Ethernet (RoCE) requires the network to beconfigured for lossless operation, however, Mellanox hasrecently enhanced RoCE with built-in error recoverymechanisms. While a lossless network has never been astrict requirement, customers typically configure