WIXLIB

CarrierEthernet.Raj JainWashington University in Saint LouisSaint Louis, MO 63130Jain@cse.wustl.eduThese slides and audio/video recordings of this class lecture are at:http://www.cse.wustl.edu/ jain/cse570-13/Washington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-1 2013 Raj Jain

OverviewTechnologies for Data Center Interconnection:PDH, SDH, MPLS, PWE32. Metro Ethernet Services: E-Line, E-LAN, E-Tree, E-Access3. Administration and Management Ethernet Services4. Extensions to Ethernet for carriers:PB (Q-in-Q), PBB (MAC-in-MAC), PBB-TENote: Although these technologies were originally developed forcarriers, they are now used inside multi-tenant data centers(clouds)1.Washington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-2 2013 Raj Jain

Options to Connect Two Data Centers?Danforth Campus1.2.3.4.5.6.7.8.9.Medical CampusDedicated Optical fiber (leased from the phone company)Ethernet over Optical Transport Network (all-Optical Switches)Ethernet over Wavelength Division Multiplexing (DWDM)Ethernet over Synchronous Digital Hierarchy (SDH)Ethernet over Plesiochronous Hierarchy (PDH)Ethernet over Pseudo-wire over MPLSEthernet over Micro-waveSingle Pair High-Speed Digital Subscriber Line (SHDSL)Ethernet with enhancementsWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-3 2013 Raj Jain

Plesiochronous Digital Hierarchy (PDH) Plesios Synchronous Near synchronousPhone Line 64 kbps 1 User channelNorth America T1 1.544 Mbps 24 User channels T2 6.312 Mbps 96 Channels T3 44.736 Mbps 480 ChannelsEurope: E1 2.048 Mbps 32 Channels E2 8.448 Mbps 128 Channels E3 139.264 Mbps 2048 ChannelsWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-4T1 2013 Raj Jain

SONET/SDHEEthernet ESSSS SDHSONET Synchronous optical networkStandard for digital optical transmissionStandardized by ANSI and then by ITU Synchronous Digital Hierarchy (SDH)Protection: Allows redundant Line or pathsFast Restoration: 50ms using ringsSophisticated managementIdeal for Voice: No queues. Guaranteed delayFixed Payload Rates: OC1 51.84 Mbps, OC3 155M,OC12 622M, OC48 2.4G, OC192 9.5GRates do not match data rates of 10M, 100M, 1G, 10GStatic rates not suitable for bursty trafficOne Payload per Stream http://www.cse.wustl.edu/ jain/cse570-13/ High CostWashington University in St. Louis7-5 2013 Raj Jain

Multiprotocol Label Switching (MPLS)PBX1 PBX3523Allows virtual circuits in IP Networks (May 1996)Each packet has a virtual circuit number called ‘label’Label determines the packet’s queuing and forwardingCircuits are called Label Switched Paths (LSPs)LSP’s have to be set up before useAllows traffic engineeringWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-6 2013 Raj Jain

Label Switching ExampleEthernet Header IP Header PayloadEthernet Header64IP Header Payload3 64 ALabelR15 3 5 R3B 5 5Washington University in St. LouisR2 2 3 C23http://www.cse.wustl.edu/ jain/cse570-13/7-7 2013 Raj Jain

IP over MPLS over EthernetCEPECarrier NetworkPECEPE Provider EdgeCE Customer EdgeDest. Adr Src. Type MPLSCustomerCustomerL3 CRCNext Hop Adr /Len Tag Dest. MAC Adr Src. MAC Adr PDU48b48b 16b32b48b48b32bLabel ExperimentalStack Position TimeCoS/Drop-Preced. 1 Bottom to Live20b3b1b8b Allows 220 Label switched paths (LSP)Each path can have reserved capacity MPLS became a very popularWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-8 2013 Raj Jain

Martini Draft 1995-1999: IP over ATM, IP over EthernetL3L2 L2L3 EthernetIPATMPPP2000 : ATM over IP, Ethernet over IPSONET over IPEthernetATMIPPPPIdea proposed by Luca Martini ofLevel 3 Communications and thenCiscoWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-9 2013 Raj Jain

Pseudo Wire: L2 Circuits over IPCE1ATMNetworkPE1PE2Pseudo WireEmulated ServiceAFrame RelayNetworkEthernetCE2ATMNetworkBIPNetworkFrame RelayNetworkEthernetTunnel Hdr Demux Field [Control] ATM ATM ATM ATMPayload Type How to de-assemble payloadMPLS/GRE/L2TP- How to get to egresshttp://www.cse.wustl.edu/ jain/cse570-13/Washington University in St. Louis 2013 Raj Jain7-10

Ethernet over PWE3 over MPLSMPLS LabelPW Label[Control]Ethernet Frame w/o FCSPID Flags FRG Length Sequence #4b 4b2b6b16b PW1MPLSLSPPW2Pseudo-Wire Emulation Edge-to-Edge (PWE3)Multiple pseudo-wires per MPLS LSPPW3Payload ID (PID): 5 Untagged Ethernet, 4 VLAN tagged, 4 VLAN tag put by carrier and customers may or may not berelevent for forwarding. Determined administratively by PE.Flags: Payload specific. FRG: Used for fragmentationPause frames are obeyed locally. Not transported.May put 802.1p priority in exp field of MPLS labelWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-11 2013 Raj Jain

Virtual Private LAN Service (VPLS) Allows multi-point Ethernet servicesover MPLS networks using pseudo-wiresNon-Hierarchical VPLS: Edge routers do complete routingtable, label distribution, multicast/broadcast replication Suitable for small deploymentsHierarchical VPLS (H-VPLS): Edge routers communicate withonly other edge devices and do not have large routing tables Suitable for large APECEBCE: Customer EdgePE: Provider EdgeRef: H. Saboowala, M. Abid, S. Modali, "Designing Networks and Services for the Cloud: Delivering business-grade cloudapplications and services," Cisco Press 2013, ISBN:1587142945http://www.cse.wustl.edu/ jain/cse570-13/Washington University in St. Louis 2013 Raj Jain7-12

Differentiated Services A way for IP routers to provide QoSExpedited Forwarding (EF): Also known as Premium Service Virtual leased line Guaranteed minimum service rate Policed: Arrival rate Minimum Service Rate Not affected by other forwarding classesAssured Forwarding (AF): Four Classes: No particular ordering Three drop preference per class:Low, Medium, HighBest Effort ServiceDifferentiated Service Code Point (6 bits) encode the service,E.g., 101110 EFWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-13 2013 Raj Jain

Carriers vs. EnterpriseWe need to exchangetopology foroptimal routing.Washington University in St. LouisSorry, We can’t tell youanything about ourinternal network.http://www.cse.wustl.edu/ jain/cse570-13/7-14 2013 Raj Jain

Issue: UNI vs Peer-to-Peer Signaling Two Business Models: Carrier: Overlay or cloud Network is a black-box User-to-network interface (UNI)to create/destroy light paths (in OIF) Enterprise: Peer-to-Peer Complete exchange of informationUNIWashington University in St. LouisUNIhttp://www.cse.wustl.edu/ jain/cse570-13/7-15 2013 Raj Jain

UNI vs. ENNI User to Network Interface (UNI): Separates responsibilities between the user and the provider.(Troubleshooting, failures etc). Like the wired phone box outside your home. Only one customer’s traffic.External Network to Network Interface (ENNI): Separates responsibilities between two providers. Many customer’s traffic passes through an ENNI Tier 2 operators sell services to Tier 3 service providers.CustomerUNIProvider 1Tier 3ENNI Provider 2Tier 2Ref: Fujitsu, “Carrier Ethernet Essentials,” tl.edu/ jain/cse570-13/Washington University in St. Louis 2013 Raj Jain7-16

Operator Virtual Connection (OVC) Between UNI and ENNI or between two ENNIs.For wholesale service providersTwo types: Point-to-Point and Multipoint-to-MultipointUntagged or single tagged frames at NNI. Q-in-Q at ENNIUNIs may be 10 to 100 Mbps. ENNIs at 1 to 10 Gbps.UNIUNIENNIUNIENNIWashington University in St. LouisENNIENNIUNIhttp://www.cse.wustl.edu/ jain/cse570-13/7-17 2013 Raj Jain

Metro Access Ethernet Private Line Access Ethernet Private Line (Access-EPL): Port-based service for Internet accessLike the service at your home. Ends at your access provider, where many other AccessEPLs may end Access provider has only one interfaceShared by many Access-EPLs Different from p2p EPL.Customer 1ENNICustomer nE-AccessWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-18 2013 Raj Jain

End-to-End Metro Ethernet Connection An EC may go through multiple service providers Multiple OVCs can be concatenated to create an ECENNIWholesaleAccess ProviderOVC1UNISubscriberWashington University in St. LouisENNIWholesaleTransport ProviderOVC2RetailService ProviderOVC3EChttp://www.cse.wustl.edu/ jain/cse570-13/7-19UNISubscriber 2013 Raj Jain

Ethernet Virtual Connections (EVCs) Port-based ECs: Forwarding not based on VLANs.Frames delivered to remote UNI/ENNI for P2P orBased on destination address for P2MPVLAN-based ECs: Forwarding based on VLAN tag. Multiple Virtual UNIs Ethernet Virtual Connection (EVC)More cost-effective for Enterprise customersEVCTypes of EVCs:UNI EVC1. Ethernet Virtual Private Line (EVPL)EVC2. Ethernet Virtual Private Tree (EVP-Tree)3. Ethernet Virtual Private LAN (EVPLAN)4. Access Ethernet Virtual Private Line (Access EVPL)Note: Service providers always share an ENNI for multipleconnections OVCs are always virtual No OCsWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-20 2013 Raj Jain

Metro Ethernet Service Attributes Bandwidth Profiles: Limits on data dates Ingress Profile: Incoming data rate Egress Profile: Outgoing data ratePer UNI, Per EVC or OVC, orDataPer EVC/OVC per Class of Service (CoS)CoS is indicated by the 3-bits in the priority field orCBS4-bit Differentiated Services Code Point (DSCP)Rate specified by 5 parameters1. Committed Information Rate (CIR)CIREBS2. Committed Burst Size (CBS)Dropped3. Excess Information rate (EIR)Mark EIR4. Excess Burst Size (EBS)DEI5. Color Mode (CM): Customer does/does notForwardedmark drop eligibility indicator (DEI)Washington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-21 2013 Raj Jain

Metro Ethernet derMEGUNI-CSubscriberWholesaleServiceProviderUNI-N ENNI ENNI UNI-N Operation, Administration and Maintenance (OAM)Defined in IEEE 802.1ag, IEEE 802.3ah, and ITU Y.1731Maintenance end points (MEPs)Maintenance Intermediate Points (MIPs)Maintenance Entity Group (MEG): Level of AdministrationUNI-C SubscriberMEPMIPService Provider MEGSubscriber MEGWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-22 2013 Raj Jain

Metro Ethernet OAM eProviderENNISubscriberUNI-N Performance Monitoring: Measure throughput and latencyConnectivity Fault Management: Monitor downtime Service Fault Management Link Fault ManagementUNI-C SubscriberIEEE 802.3ah Link OAMLinkOAMITU-T Y.1731 End-to-End Performance MonitoringIEEE 802.3ag End-to-End Connectivity Fault ManagementWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-23ServiceOAM 2013 Raj Jain

Metro Ethernet OAM iceProviderENNISubscriberUNI-N Continuity Check Message (CCM) in both directions(Similar to IP Ping)Link Trace Message (LTM): Locates fault. Link TraceResponse (LTR) is returned by each end point and intermediatepoint (similar to IP trace route)UNI-C SubscriberService Provider MEGLTMLTRsWashington University in St. Louishttp://www.cse.wustl.edu/ jain/cse570-13/7-24 2013 Raj Jain

Metro Ethernet Use Cases2.InternetSubscriber 1Building BUNIP2P EVCRetailServiceProviderUNIUNISubscriber 1Building AAccess EVCENNIHead office to Satellite offices and/or InternetUNI1.Customers to Cloud Service ProviderRetailServiceProviderWashington University in St. erhttp://www.cse.wustl.edu/ criber 2UNIOVCAENNIUNISubscriber 1UNIEVC1 2013 Raj Jain