Transcription
Time-Sensitive Networking: A Technical IntroductionWhite PaperCisco PublicTime-Sensitive Networking:A Technical Introduction 2017 Cisco and/or its affiliates. All rights reserved.
Time-Sensitive Networking: A Technical IntroductionWhite PaperCisco PublicWhat is time-sensitive networking (TSN)? In its simplest form, TSN is the IEEE802.1Q defined standard technology to provide deterministic messaging on standardEthernet. TSN technology is centrally managed and delivers guarantees of deliveryand minimized jitter using time scheduling for those real-time applications that requiredeterminism.TSN is a Layer 2 technology. The IEEE 802.1Q standards work at OSI Layer 2. TSN isan Ethernet standard, not an Internet Protocol standard. The forwarding decisions madeby the TSN bridges use the Ethernet header contents, not the IP address. The payloadsof the Ethernet frames can be anything and are not limited to Internet Protocol. Thismeans that TSN can be used in any environment and can carry the payload of anyindustrial application.TSN was developed to enable deterministic communication on standard Ethernet.The market for deterministic communication is using nonstandard technologies ornonstandard Ethernet. Prior to the IEEE 802.1 TSN standards, standard Ethernet didn’thave pure Layer 2 deterministic capability.Deterministic communication is important to multiple industries (for example,aerospace, automotive, manufacturing, transportation, and utilities). Providing ameans for determinism over standard Ethernet enables new levels of connectivity andoptimization, leading to cost savings for many industries.This short document is intended to give the interested reader more technical detailsabout TSN and an introduction to the Cisco implementation.As the name suggests, “time” is the primary aspect of TSN. TSN is a technologyfocused on time. TSN was developed to provide a way to make sure informationcan travel from point A to point B in a fixed and predictable amount of time. Beingpredictable enables increased efficiency.There’s an implied requirement for those networking devices implementing TSN (enddevices and bridges) to share a common sense of time. Precision Time Protocol (PTP)is used to maintain a common sense of time. The PTP profiles chosen to work withTSN are IEEE 802.1AS and IEEE 802.1ASRev. 2017 Cisco and/or its affiliates. All rights reserved.2
Time-Sensitive Networking: A Technical IntroductionWhite PaperCisco PublicTSN Solution ComponentsThere are five main components in the TSN solution: TSN flow: Term used to describe the time-criticalcommunication between end devices. Each flowhas strict time requirements that the networkingdevices honor. Each TSN flow is uniquely identifiedby the network devices. End devices: These are the source anddestinations of the TSN flows. The end devices arerunning an application that requires deterministiccommunication. These are also referred to astalkers and listeners. Bridges: Also referred as Ethernet switches.For TSN, these are special bridges capable oftransmitting the Ethernet frames of a TSN flow ona schedule and receiving Ethernet frames of a TSNflow according to a schedule. Central network controller (CNC): For TSN,the CNC acts as a proxy for the Network (theTSN Bridges and their interconnections) and thecontrol applications that require deterministiccommunication. The CNC defines the schedule onwhich all TSN frames are transmitted. The CNCapplication is provided by the vendor of the TSNbridges. Cisco has developed a CNC applicationfor controlling its TSN bridges for TSN. Centralized user configuration (CUC): Anapplication that communicates with the CNCand the end devices. The CUC represents thecontrol applications and the end devices. TheCUC makes requests to the CNC for deterministiccommunication (TSN flows) with specificrequirements for those flows. The CUC is anapplication that is vendor specific. Typically thevendor of the TSN end devices will supply a CUCfor those end devices.Table 1 describes TSN standards.Table 1.TSN StandardsStandardArea of DefinitionTitle of StandardIEEE 802.1ASrev, IEEE 1588Timing and synchronizationEnhancements and performanceimprovementsIEEE 802.1Qbu and IEEE 802.3brForwarding and queuingFrame preemptionIEEE 802.1QbvForwarding and queuingEnhancements for scheduledtrafficIEEE 802.1QcaPath control and reservationPath control and reservationIEEE 802.1QccCentral configuration methodEnhancements and performanceimprovementsIEEE 802.1QciTime-based ingress policingPer-stream filtering and policingIEEE 802.1CBSeamless redundancyFrame replication and eliminationfor reliabilityNot all standards shown in Table 1 are required to support TSN. Cisco’s initial implementation uses802.1Qbv, 802.1AS (predecessor to 802.1ASRev), and 802.1Qcc. 2017 Cisco and/or its affiliates. All rights reserved.3
Time-Sensitive Networking: A Technical IntroductionWhite PaperCisco PublicA Shared Concept of TimeOverview of Cisco TSN SolutionThe key to providing determinism is a sharedconcept of time. The implementation of 802.1AS(and in the future 802.1ASRev) by all networkelements (end devices and bridges) is requiredfor TSN. The 802.1AS PTP profile allows all TSNnetwork elements to share the same concept oftime.Cisco is supporting TSN on the IE-4000 productfamily. All models of IE-4000 support TSN. Startingwith Cisco IOS Software release 15.2(5)E2, theTSN functionality is available. The IE-4000s have anFPGA in the data path that enable them to supportTSN. The IE-4000 operates as a TSN bridge. TheIE-4000 implements IEEE 802.1Qbv and IEEE802.1AS. The IE-4000 supports hundreds of TSNflows across its Ethernet interfaces.The key to providing on-time delivery of TSN framesis 802.1Qbv. Qbv defines a means to transmitcertain TSN Ethernet frames on a schedule, whileallowing non-TSN Ethernet frames to be transmittedon a best effort basis around the TSN frames.Because all network elements share the same time,end devices and bridges implementing Qbv candeliver critical communication very quickly and withno discernible jitter in delivery.A good analogy for IEEE 802.1Qbv is a train system.Think of the Ethernet bridges as cities and the linksbetween Brides as train tracks between the cities.There is one track between each city. Trains go fromone city to another based on a schedule. The trainfrom city A to city X is scheduled well in advance.Although there are many cities and destinations,there are is only one track between two adjacentcities (just like the cat5 cabling between twoEthernet bridges). The track is the scarce resourcethat has to be time shared. Two trains cannot beon the track at the same time. The train systemschedules the departure time for each train froma station, making sure that the track is well usedand there are no conflicts. With TSN, the Ethernetbridges allow for use of the links to unscheduledtraffic when there is no scheduled traffic. For thetrain system, this could be the regional and localtrains using the tracks when the intracity trains arenot using them.IEEE 802.1Qcc is focused on definition ofmanagement interfaces and protocols to enableTSN network administration. 802.1Qcc is a largespecification with many aspects. Cisco is usingthe centralized approach to network managementdefined in 802.1Qcc. 2017 Cisco and/or its affiliates. All rights reserved.As the name suggests, the TSN central networkcontroller (CNC) controls the TSN bridges in thenetwork. The CNC is a software application runningon customer premises (as opposed to cloud).The hardware housing the CNC application is notrelevant and can be anything. The CNC has twoprimary responsibilities. First, it is responsiblefor determining routes and scheduling the TSNflows through the bridged network. Second, it isresponsible for configuring the TSN bridges forTSN operation.The CNC communicates with the CUC to receivethe communications requirements that the networkmust provide. The CNC aggregates all the requests,computes the route for each communicationrequest, schedules the end-to-end transmission foreach TSN flow, and finally transfers the computedschedule to each TSN bridge. As part of theschedule computation, the CNC provides a uniqueidentifier for each TSN flow. This unique identifier isused by the TSN bridges to differentiate one TSNflow from another. The unique identifier includes thedestination MAC address, VLAN ID, and CoS value.With these three items, the TSN bridges can identifythe TSN flow and transmit the flow based on thecorrect schedule.In Figure 1, you can see all the components andhow they relate. The CUC communicates with theCNC using REST APIs. The CNC communicateswith both CUC and TSN bridges. The CUCcommunicates with the talker/listener end devices.The TSN bridges switch the TSN communicationEthernet packets between the talkers and listeners.4
Time-Sensitive Networking: A Technical IntroductionWhite PaperCisco PublicAfter completion, the CUC issues a request ofthe CNC to return the discovered topology. Theengineer at this point could verify that the CNCdiscovered the topology correctly if they choose.Figure 1. Sample Topology for TSN NetworkingComponentsCUCStep 2: CUC Requests Network ResourcesSchedulerREST APIsCNCTL/1TL/2TSN BridgeTSN BridgeTSN flow TL/1 and TL/2The APIs between the CUC and CNC aredefined and available as part of the Cisco TSNdocumentation. The APIs are there for everythingneeded by the CUC to completely program the TSNflows in the network. The APIs are so complete,the CUC can hide the CNC and the network. It’spossible for the control engineer to interface withthe CUC only, never actively using the CNC orconfiguring a TSN bridge.TSN Sample WorkflowFigure 1 shows lines of control plane communicationof steps that occur between the CUC and CNCbefore the talkers and listeners can start exchangingTSN communications (called TSN flows). Thefollowing steps are typical for a workflow and aredescribed only for the purpose of showing howand why the entities communicate with each other.Disclaimer: This is not the only order in which thesesteps can be executed.Step 1: CUC Initiates Physical TopologyDiscoveryBefore the schedule can be successfully computed,the CNC must learn the physical topology. The CUCwill initiate a request to the CNC to discover thephysical topology. Using LLDP and a seed device,the CNC walks the physical topology, discoveringeach device and how they are connected. Thisincludes the end devices that support LLDP. 2017 Cisco and/or its affiliates. All rights reserved.Performed by the engineer responsible for definingthe end to end communication. The engineer worksout which end device (talker) has to communicatewith other end devices (listener). The engineer isresponsible for identifying all listeners, becausethere can be more than one for each TSNflow. The engineer can also define the latencyrequirements for the communication (for example,the listeners must receive within 500µs from startof transmission), the maximum size of the Ethernetpacket that will be sent, and other dependencies(for example, whether there is a sequence order tothe TSN flows). The CUC will gather this for all TSNflow requests and submit to the CNC using an APIfor accepting requests.Step 3: Compute ScheduleSatisfied that the topology has been discovered andthe CNC has received all TSN flow requests, theCUC will initiate a request that the CNC computethe schedule. The CNC will return success orfailure for the request. The end schedule cannotbe computed unless the CNC knows the physicaltopology. Step 3 depends on steps 1 and 2.Step 4: View Computation ResultsTypically the network engineer would want to viewthe schedule and verify before making it go live.The CUC requests the CNC return the details of thecomputed schedule. This includes the details foreach device involved in the TSN flows. The detailsinclude everything the end devices and bridgesneed to know for configuring TSN: Unique identifiers for each TSN flow (destinationMAC address, VLAN, CoS) Start and end of transmit window at each hop(talkers and bridges) Start and end of receive window at each hop(listeners and bridges) End-to-end latency as computed5
Time-Sensitive Networking: A Technical IntroductionWhite PaperCisco PublicStep 5: Distribute ScheduleSatisfied that the schedule will work, the CUCissues a request to the CNC to distribute thecomputed schedule to the TSN bridges. The CUCwill also program the talkers and listeners for theTSN flows. The talkers are expected to transmitevery TSN flow according to a schedule.Figure 2. Physical Topology for Example WorkflowTSN Bridge1Gi1/5Gi1/4Step 6: Verify Schedule Distribution on TSNBridgesThis step is optional. In troubleshooting scenarios,users can log in to the TSN bridges to verify theschedule for the TSN flows.Actual Example Using Two Flows with TwoEndpointsHere’s an example of how an engineer might createtwo TSN communication flows between two enddevices.In this scenario, TL1 and TL2 are TSN end devicesthat operate in sync with each other. They operatein a very fast control loop that iterates 1000 timesa second. Every millisecond (1/1000 of a second),TL1 sends a time-sensitive message to TL2. TL2receives the message, does computation, andsends a time-sensitive message back to TL1. TL2needs to receive the time-sensitive message fromTL1 before 500µs into the 1ms. TL2 has 250µs to350µs to do the computation based on the datereceived from TL1, then transmit a response backto TL1, which must arrive before the 1ms time is up.Each time a message is sent between TL1 and TL2,the network has 100µs to deliver that message.It’s worth noting that the control engineer in thisexample never has to configure the TSN bridges. TheCNC takes care of all TSN bridge communication andconfiguration. The control engineer only interfaceswith the CUC. This makes it easy for the controlengineer to implement TSN in the network.Figure 2 shows an example physical networktopology. The network devices and the connectionswere set up in advance and are static. The TSNflows are between the two end devices (identifiedas TL1 and TL2). 2017 Cisco and/or its affiliates. All rights reserved.tsn-TL1Flow1Flow2Gi1/4Gi1/6tsn-TL2TSN Bridge2Step 1. Using the CUC, the control engineerasks the CNC to discover the physical network inpreparation for schedule computation.Step 2. Using the CUC, the control engineer makestwo requests to create TSN flows of the network.The first request is to create a TSN flow from TL1to TL2. TL1 will start transmitting at 350µs after thestart of the millisecond. The network has 100µs ofmaximum latency to deliver the message to TL2.The message has maximum size of 64 bytes. Thesecond request is the response message. Thecontrol engineer requests the network to create aTSN flow from TL2 to TL1. TL2 will start transmittingat 850µs after the start of the millisecond. Thenetwork has a 100µs maximum latency to deliverthe message to TL1. The response message is 64bytes maximum size.Step 3. Using the CUC, the control engineer asksthe CNC to compute a schedule for the requestedTSN flows. The CNC returns a response to thecompute request based on success or failure of thescheduling logic.Step 4. The control engineer views thecomputation results. In tabular format, the results ofthe computation look like Table 2.6
Time-Sensitive Networking: A Technical IntroductionWhite PaperCisco PublicTable 2.Sample Computation oSVLANTalkerTransmitWindowReceiver ReceiverNameWindowTransmissionDurationFlow -TL2463–476µs100µsFlow -TL1963–976µs100µsWhen transmitting a TSN flow, the talker is given a window to transmit. The window is approximately 13µswide in terms of time (for links operating at Gigabit speeds) The 13µs is required to account for the chancethat a large 1518-byte Ethernet frame will be transmitted just before the TSN flow Ethernet frame and willdelay the TSN flow by 12µs. A 64-byte frame takes 0.7µs to transmit.Step 5. Satisfied the schedule computation ismeeting the requirements, the control engineerinstructs the CUC to distribute the schedule to thenetworking elements.Step 6. This step is optional. This how theschedule will appear on a Cisco TSN Bridge. This isTSN-Bridge 1.The transmit of flow 1 from tsn-TL1 is the receivefor TSN-Bridge1. and the transmit of flow 2 fromTSN-bridge1 is the receive for tsn-TL1. There issome delay on the TSN bridge that is added by thescheduler. In this case, the delay is at 27µs. TSNbridges have store and forward architectures, andthis adds to the end-to-end latency of theEthernet frame.TSN Bridge1 show tsn flow detailOrganizations Driving TSNFlow 1001Stream ID : flow1Stream Address : 0300.5EA0.03E9Frame Size : 64BIngress Interface : Rx ScheduleGi1/5: 350-363 (us)Egress Interface : TX ScheduleGi1/4: 390-403 (us)Period cycle time : 1000 (us)Flow 1002Stream ID : flow2Stream Address : 0300.5EA0.03EAFrame Size : 64BIngress Interface : Rx ScheduleGi1/4: 923-936 (us)Egress Interface : TX ScheduleGi1/5: 963-976 (us)Just implementing to the IEEE standards is notenough to make sure of vendor interoperability.For instance, different vendors could implementthe standard differently. Each vendor could claimsupport for the standard. It would be up to the endcustomer deploying products that claim to supportthe standard to solve any incompatibilities betweenthe vendors. Neither the customer nor the vendorswant such a situation.To help with interoperability, organizations areformed to focus on solutions that are basedon new standards. For TSN, two of the mostactive organizations are AVnu Alliance and theIndustrial Internet Consortium. Both organizationsare composed of vendors that are building TSNproducts.AVnuAVnu Alliance is a community creating aninteroperable ecosystem servicing the precisetiming and low-latency requirements of diverseapplications using open standards throughcertification.Period cycle time : 1000 (us) 2017 Cisco and/or its affiliates. All rights reserved.7
Time-Sensitive Networking: A Technical IntroductionWhite PaperCisco PublicAVnu writes standards, which are profiles of IEEEstandards that help implementers in selectedvertical industries make sense of large, generalpurpose standards documents.AVnu tests and certifies devices for interoperability,providing a simple and reliable networking solutionfor AV network implementation based on the IEEE802.1 Audio Video Bridging (AVB) and TimeSensitive Networking (TSN) standards.AVnu provides marketing support for the concepts,standards, and products supporting AVB and TSNstandards.Industrial Internet Consortium (IIC)The IIC brings together organizations andtechnologies necessary to accelerate the growthof industrial Internet by identifying, assembling, andpromoting best practices. The goals of the IIC areto drive innovation, define reference architectures,facilitate open forums and test beds, and influencethe global development standards process for theInternet and industrial systems.Q&AQWhat value does TSN bring tocustomers?ACustomers employing TSN-baseddeterministic solutions will be able toconsolidate multiple services over a single,standard physical network.QDoes the TSN technology fromCisco require a license?AYes. To enable the TSN technology on CiscoIE switches, a license must be purchased.QIs TSN a new type of quality ofservice?ANo. Quality of service is used by networkingdevices (bridges and routers) to prioritizetraffic. In congestion conditions, QoSenables bridges and routers to prioritize thedifferent network traffic. TSN flows are highpriority, but do not compete with non-TSNflows regardless of the priority of the nonTSN flows.QCan TSN flows be configuredon the Cisco IOS Softwarecommand line?ANo. TSN flows can only be configured fromthe CNC. From the Cisco IOS Software CLI,only TSN troubleshooting is permitted. 2017 Cisco and/or its affiliates. All rights reserved. Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S.and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property oftheir respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)C11-738950-00 05/17
The market for deterministic communication is using nonstandard technologies or nonstandard Ethernet. Prior to the IEEE 802.1 TSN standards, standard Ethernet didn’t have pure Layer 2 deterministic c
