Openstack Networking Design - NANOG

Transcription

Openstack Networking DesignPete Lumbis – CCIE #28677, CCDE 2012::3Cumulus Networks Technical Marketing Engineer1

Openstack OverviewTakes a pool of serversDeploys VMs (OS, disk, memory, CPU cores, etc)Attaches VM to networksResource management“Microservice” style. Each thing is a stand alone component§§§§Openstack “Projects” (i.e., Neutron, Nova, Cinder, etc)Why Openstack is seen as complexNetworking vs Compute vs Storage vs GUIWork via APIs, not tightly coupled2

Openstack Nova – Compute ServicesDeploys VMsManages CPU, memory, disk size“Nova Nodes” are servers that can run VMs3

Openstack Neutron – Networking ServicesManages list of tenant networks§ Tenant VLAN/VxLANAssigns tenant network to VM§ Programs network stack on “Nova Nodes” based on user config(Optional) ML2 Driver: Neutron server to switch API§ Switch runs the driver to translate Openstack API to local config/stateNeutron focus is layer 2. L3 generally done on server software§ L3 gateway, NAT, ACLs done on a centralized Neutron x86 Node§ DVR (Distributed vRouter) allows for L3 on Local Compute Node (Nova)4

Relevant ComponentsController Nodes(Global Openstack Manager)Neutron Nodes(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Nova Nodes(Where VMs Run)5

Relevant ComponentsController Nodes(Global Openstack Manager)11User requests a new VM, supplyingparameters like amount of RAM and thenetwork (vlan/l2 segment) they want tobe onNeutron Nodes(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Nova Nodes(Where VMs Run)6

Relevant ComponentsController Nodes(Global Openstack Manager)12User requests a new VM, supplyingparameters like amount of RAM and thenetwork (vlan/l2 segment) they want tobe onOpenstack Controller magically selectsa Nova Node to deploy the VM on.2Neutron Nodes(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Nova Nodes(Where VMs Run)7

Relevant ComponentsController Nodes(Global Openstack Manager)12222User requests a new VM, supplyingparameters like amount of RAM and thenetwork (vlan/l2 segment) they want tobe onOpenstack Controller magically selectsa Nova Node to deploy the VM on.Simultaneously Openstack Controllertells Neutron Node about a new VM,the Nova node and desired VLANNeutron Nodes(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Nova Nodes(Where VMs Run)8

Relevant ComponentsController Nodes(Global Openstack Manager)12Neutron Nodes(L3 Gateway, NAT,?Programmer of VLAN/3VxLANs)User requests a new VM, supplyingparameters like amount of RAM and thenetwork (vlan/l2 segment) they want tobe onOpenstack Controller magically selectsa Nova Node to deploy the VM on.2Simultaneously Openstack Controllertells Neutron Node about a new VM,the Nova node and desired VLAN3Neutron decides if the network alreadyexists. If no, a new L3 gateway iscreated on the Neutron Node.Nova Nodes(Where VMs Run)9

Relevant ComponentsController Nodes(Global Openstack Manager)124Neutron Nodes(L3 Gateway, NAT,Programmer of VLAN/VxLANs)User requests a new VM, supplyingparameters like amount of RAM and thenetwork (vlan/l2 segment) they want tobe onOpenstack Controller magically selectsa Nova Node to deploy the VM on.2Simultaneously Openstack Controllertells Neutron Node about a new VM,the Nova node and desired VLAN3Neutron decides if the network alreadyexists. If no, a new L3 gateway iscreated on the Neutron Node.4Neutron builds L2 config on the NovadeviceNova Nodes(Where VMs Run)10

Relevant ComponentsController Nodes(Global Openstack Manager)12ML25Neutron Nodes(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Openstack Controller magically selectsa Nova Node to deploy the VM on.2Simultaneously Openstack Controllertells Neutron Node about a new VM,the Nova node and desired VLAN3Neutron decides if the network alreadyexists. If no, a new L3 gateway iscreated on the Neutron Node.4Nova Nodes(Where VMs Run)User requests a new VM, supplyingparameters like amount of RAM and thenetwork (vlan/l2 segment) they want tobe on5Neutron builds L2 config on the NovadeviceIf deployed to do so, Neutron willspeak, via the ML2 Driver, to ahardware switch to provision a VLAN orVxLAN (along with or instead of step 4)11

Relevant ComponentsController Nodes(Global Openstack Manager)612ML2Neutron Nodes(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Openstack Controller magically selectsa Nova Node to deploy the VM on.2Simultaneously Openstack Controllertells Neutron Node about a new VM,the Nova node and desired VLAN3Neutron decides if the network alreadyexists. If no, a new L3 gateway iscreated on the Neutron Node.4Nova Nodes(Where VMs Run)User requests a new VM, supplyingparameters like amount of RAM and thenetwork (vlan/l2 segment) they want tobe onNeutron builds L2 config on the Novadevice5If deployed to do so, Neutron willspeak, via the ML2 Driver, to ahardware switch to provision a VLAN orVxLAN (along with or instead of step 4)6Local ML2 plugin translates OpenStackconfig into device specific config12

Openstack Network Design OptionsVLANs§ Most common§ Most fragileEVPN-VxLAN§ Network centric§ Scalable, resilientVxLAN on Servers§ Most scalable§ Simplest network§ More complex servers13

Openstack Networking: Preprovisioned VLANsNetwork trunks all VLANsNeutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Nova Nodes(Where VMs Run)14

Openstack Networking: Preprovisioned VLANsNetwork trunks all VLANsServers may trunk all VLANsNeutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Nova Nodes(Where VMs Run)15

Openstack Networking: Preprovisioned VLANsNeutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Network trunks all VLANsServers may trunk all VLANsNew server creation only linksphysical trunk to VMVMVMNova Nodes(Where VMs Run)16

Openstack Networking: Preprovisioned VLANs17

Openstack Networking: Preprovisioned VLANsNeutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Pros:§ Easiest deployment§ Physical network is static§ No ML2Cons:§ Limited scale§ Very large blast radiusVMVMNova Nodes(Where VMs Run)18

Variation on a Theme: ML2 Provisioned VLANsNothing pre-configuredEmpty trunks on network andcomputeML2ML2ML2 Agent running on all switches§ Including Core/SpinesNova Nodes(Where VMs Run)19

Variation on a Theme: ML2 Provisioned VLANsNeutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Neutron provisions VLANs onswitches via ML2 agentML2ML2VMVMNova Nodes(Where VMs Run)20

Variation on a Theme: ML2 Provisioned VLANsNeutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Neutron provisions VLANs onswitches via ML2 agentML2 Agent provisions switch VLANsML2ML2VMVMNova Nodes(Where VMs Run)21

Variation on a Theme: ML2 Provisioned VLANsNeutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Pros:§ Simple server networking§ Slightly more scalable*ML2ML2Cons:§ Requires ML2 on switches§ Still limited by L2 scale§ Likely to still have large blastradius as environment growsVMVMNova Nodes(Where VMs Run)22

Sidebar: Openstack Agents and ML2 on SwitchesOpenstack is designed for “clouds”Everything is ephemeral§ It can die at any time and no one should care§ This includes networking§ No one actually accepts this factThere may be no “config” for ML2 stateA reloaded switch may lose all provisioned state§ Depends on vendor implementationLost state requires all rack VMs to be destroyed and recreated23

Sidebar: Openstack Agents and ML2 on Switches24

VxLAN-EVPN for Better L2VxLAN provides L2 over L3EVPN provides VxLAN controlplane (Where MACs live)EVPNEVPN configured from TOR to TORPre-Provision VxLAN Tunnels§ Scale improvement since MACsare not pushed local a host existsOpenstack doesn’t care aboutEVPNNova Nodes(Where VMs Run)25

VxLAN-EVPN for Better L2Neutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Neutron provisions VLAN on hostVMVMNova Nodes(Where VMs Run)26

VxLAN-EVPN for Better L2Neutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Neutron provisions VLAN on hostNeuron also provisions VLAN onnetwork via ML2ML2ML2Switch pre-provisioning mapsVLAN to VxLANVMVMNova Nodes(Where VMs Run)27

VxLAN-EVPN for Better L2Neutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Alternative deployment:§ Pre provision switch VLANs§ Neutron only deploys serverVLANsVMVMNova Nodes(Where VMs Run)28

Server Networking: Pre-provisioned EVPNNeutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Pros:§ Allows for L3 underlay§ Easily scales 100-1000 tenants§ Preprovisioning is easyML2ML2Cons:§ May require ML2 on switches§ Network still involvedVMVMNova Nodes(Where VMs Run)29

Server Networking: Pre-provisioned EVPN30

Scalable Openstack: Server based VxLANHost connects to TOR on routedportHost runs Free Range Routing(FOSS routing suite)eBGP (L3 Fabric)Host and TOR run eBGPunnumbered§ Dual attach does not requiremLAGServer advertises /32 loopbackinto the networkNo relationship betweenOpenstack and BGPFRRFRRNova Nodes(Where VMs Run)31

Scalable Openstack: Server based VxLANNeutron programs VxLAN tunnelfrom host to hostvxLANServer loopback interface is theTunnel Endpoint (VTEP)Neutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)Host only sends encapsulatedVxLAN traffic into the networkSwitches only do basic L3 routingOpenstack links VxLAN to VM§ VM still only has normal ethernet,no VM based VxLAN orVLAN tagVMVMNova Nodes(Where VMs Run)32

Scalable Openstack: Server based VxLANPros:§ Operationally easy, no mLAGvxLANPlug and play servers, no IPAMNeutron Node(L3 Gateway, NAT,Programmer of VLAN/VxLANs)0 packet loss network changes§ Extremely scalableCons:§ CPU performance hit if NICsdon’t support VxLAN§ Requires FRR on servers§ Ironic still requires ML2VMVMNova Nodes(Where VMs Run)33

What Should I Do?34

What Should I Do?How many tenants?§ 1-50: L2 everywhere, pre-provisioned will be easiest§ 50-1000: consider pre-provisioning VxLAN-EVPN§ 1000: dynamic (ML2, server VxLAN) options are required to scaleDo you want the network to be programmed by Openstack?§ Yes: ML2 is acceptable§ No: Pre-provision or use server-server VxLANVLANs or VxLANs?§ Always prefer VxLANs§ Network hardware needs VxLAN support§ Server NICs need VxLAN offload35

A Final Note about L3All this is about L2 connectivityL3 is a different plugin (Layer 3 Plugin)§ Less network vendor support for L3 vs L2 pluginL3 usually requires NAT§ Most Network Hardware L3 plugins don’t support NAT functionalityOther services often required (FWaaS, LBaaS)§ Easy to scale out with FRR on hosts36

Thank you!Visit us at cumulusnetworks.com or follow us @cumulusnetworks 2018 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of CumulusNetworks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademarkLinux is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.37

01.10.2018 · Openstack Networking Design Cumulus Networks Technical Marketing Engineer . 2 Openstack Overview Takes a pool of servers Deploys VMs (OS, disk, memory, CPU cores, etc) Attaches VM to networks Resource management “Microservice” style. Each thing is a stand alone component § Openstack “Projects” (i.e., Neutron, Nova, Cinder, etc) § Why Openstack is seen as complex §