WIXLIB

Flow table capacity can be tested by first deleting all entries in a flow table and then insertingentries in that table until error messages indicate that the table is full. As a part of the test itshould be verified with an Ethernet tester that at least some of the entries have actually beeninstalled: If the entry specifies that a flow of data packet should be forwarded from port A to portB, an Ethernet tester can be connected to the two ports, set up to generate data packetsmatching the flow and verify that the packets actually go from port A to port B. Numerousstreams of data packets should be generated – emulating different flows – to see that all flowsare handled in accordance with the contents of the flow tables.Packet-in/packet-out capacity: Packet-in is a facility in the OpenFlow protocol that transfers thecontrol of a data packet to the OpenFlow controller. The data packet is embedded in anOpenFlow packet-in message, which also contains the reason for sending the packet to thecontroller: No matching flow (“table-miss”) An action explicitly defines that the packet must be sent to controller Packet has an invalid TTLThe controller can send the packets received in the packet-in message back to the data plane ina packet-out message.The packet-in/packet-out facility can be used for applications like MAC learning and topologydiscovery. The capacity for sending packet-in/packet-out is typically limited and should bechecked. Packet-in capacity can be checked by making an entry in the flow table defining thatpackets must be output to the controller. Then the Ethernet tester in figure 6 is used to sendthat type of packets starting with very few per second and increasing number of packets until thesystem is saturated. Packet-out capacity is checked in a similar way, with the controller sendingpacket-out messages. The communication between the switch under test and the controllershould be captured and presented in Wireshark for detailed analysis.Timeout verification: Newer versions of the OpenFlow specification include a “Timeout” sectionin the table flow entries. Two types of timeouts are supported: Hard timeouts – the maximum amount of time an entry is active in the flow table regardlessof the activity in the flow.WHITE PAPERFlow-Mod response time is the time it takes from a flow table entry is sent from a controller untilthe actions/instructions in the entry is implemented in the OpenFlow switch. This can be testedby adding (or changing) a flow table entry specifying that a flow of data packet should beforwarded from port A to port B. If you capture the OpenFlow messages to the switch under teston the open southbound API and data packets from an Ethernet tester generating traffic fromport A to B you can use Wireshark to see the time stamp of the Flow-Mod message setting up thetable entry and the time stamp of the first data packet that is sent on port B. With the two timestamps you can calculate the time it took to implement the flow table entry. A similar test can beconducted for flow table entries that are deleted.Xena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – www.xenanetworks.com8

Idle timeouts – the time without activity in the flow before the related table entry expiresTo verify hard timeouts you can capture the FlowMod message creating a flow table entrydirecting a flow of data packets from port A to port B. If you send packets from port A to port Band also capture these packets, you can use Wireshark to compare the timestamp of theFlowMod message and the last received data package on port B to verify the hard timeout.Idle timeouts for a flow table entry that defines data packets to be sent from port A to port B canbe verified by first sending traffic from port A to B, then stopping and then send one data packetinto port A after the idle timeout has expired. No data packets should appear on port B. Insteada packet-in message may appear sent from the switch under test to the controller to informabout the packet that don’t fit with the flow table entries (table-miss). This can be seen bycapturing the OpenFlow messages sent between the switch and the controller.OpenFlow pipeline latency measurements will show the implication of having multiple tables in aswitch with many matches and action set updates compared with simple tables with only one ortwo matches. This is measured by creating flow table entries that emulates the two situationsand measure the latency through the switch in the setup shown in figure 6.OpenFlow Data Packet Forwarding PerformanceFigure 7: Test setup for OpenFlow performance testing – Data packet forwardingWhen an SDN is configured the Data packet forwarding performance of the data plane can betested. This includes verification of: Throughput – maximum data rate that can be transferred through the device/path Latency – time it takes to transfer data through the device/path Frame loss – data frames lost during the transfer of data through the device/pathWHITE PAPER Xena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – www.xenanetworks.com9

Back-to-back frames – identifies the longest burst of frames with minimum inter-frame gapsthat can be sent through the device/path without frame loss Jitter – variation in delay of transferred data packets (also known as Frame Delay Variation)Testing is in typically done in accordance with the RFC 2544 Benchmarking Methodology forNetwork Interconnect Devices tests. RFC 2544 specifies throughput, latency, frame loss and backto-back testing – most Ethernet testers also include jitter testing in their RFC 2544 test suite. Thetest setup in figure 7 should be used for end-to-end RFC 2544 testing of a path through a numberof OpenFlow switches. If a single OpenFlow switch is tested in accordance with RFC 2544 the testsetup in figure 6 will be relevant.Enterprise customers typically sign Service Level Agreements (SLAs) with a network operator fortheir communication path through a network. This will also apply for paths through a SDN withOpenFlow switches in the data plane. Therefore it will be relevant to verify that the requirementsin the SLA are fulfilled when flow tables are set up to support the customers’ requirements. ITU-Thas defined the Y.1564 standard for turning up, installing and trouble-shooting Ethernet-basedservices. Y.1564 allows a complete validation of Ethernet SLAs in a single test.Y.1564 focuses on the following service level parameters: Information rate (IR) – or Bandwidth Frame transfer delay (FTD) – or latency Frame delay variation (FDV) – or packet jitter Frame loss ratio (FLR) Availability (AVAIL)The test setup in figure 7 is relevant for the Y.1564 test. It will measure the parameters for thetwo directions of a connection separately with two testers – one at each end of the connection.To identify FTD for each direction it is however necessary the synchronize time stamps in the twotesters, typically using Precision Time Protocol (PTP), Network Time Protocol (NTP) or GlobalPositioning System (GPS).A SLA will guarantee Service Acceptance Criteria (SAC), which are worst case values for FTD, FDV,FLR and AVAIL at a Committed Information Rate (CIR) – the maximum bandwidth guaranteed bythe network operator to the customer. The SLA can also include a commitment on how largebursts the network will accept - the Committed Burst Size (CBS). The network operator may allowthat both CIR and CBS are exceeded to some extend: Excess Information Rate (EIR) and ExcessBurst Size (EBS). However when CIR or CBS are exceeded it is not expected that the SACs are met.WHITE PAPER Xena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – www.xenanetworks.com10

A number of vendors are now offering OpenFlow solutions, including: Cisco Hewlett Packard Enterprise Ericsson Huawei Technologies FiberHome Telecommunication Technologies NEC Corporation Hangzhou DPtech Technologies ZTE CorporationONF maintains a list of ONF OpenFlow Conformant/Certified Products.Xena Networks OpenFlow Performance Test SolutionsFigure 8: The versatile and powerful Xena Networks Layer 2-3 testers ValkyrieBay andValkyrieCompactThe versatile and powerful Xena Networks Valkyrie Layer 2-3 testers ValkyrieBay andValkyrieCompact are the obvious choice for OpenFlow performance testing. When the Xenatesters are equipped with the ValkyrieTimeSynch their clocks can be synchronized.Testing up to Layer 3Based on Xena’s advanced architecture, ValkyrieBay and ValkyrieCompact equipped withrelevant test modules are proven solutions for Ethernet testing at layers 2 and 3. Advanced testscenarios can be performed with ValkyrieBay and ValkyrieCompact equipped with relevant testmodules using the free test applications for the modules:ValkyrieManager test software is used to configure and generate streams of Ethernet trafficbetween Xena test equipment and Devices Under Test (DUTs)/Systems Under Test (SUTs) andanalyze the results. ValkyrieManager features include: Multistream traffic generation at line rate Generation of traffic streams with different rates and packet sizes Generation of IPv4 and IPv6 traffic Generation of traffic streams with UDP and TCP headers Generation of frames with VLAN tags, MPLS labels and PBB tagsWHITE PAPEROpenFlow VendorsXena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – www.xenanetworks.com11

A “Port-to-Port” mode where frames received on one port are sent out on another port. Thiscan be used for capturing OpenFlow messages between the controller and the switch undertest in figure 6. The captured messages can be analyzed in Wireshark A Stream Scheduler, which can be used to build a sequence of actions like sending traffic,stopping after some time and resuming sending traffic (which could be just a single datapacket) after another period of timeOne ValkyrieManager can control multiple ValkyrieCompact and ValkyrieBay test chassis, whichcan be located far away from each other, e.g. at the ends of connections-to-be-tested supportingone-way measurements.Valkyrie2544 offers full support for the 4 test types specified in RFC 2544: Throughput Latency Frame loss Back-to-back frames Jitter (Frame Delay Variation) is also supportedValkyrie2544 supports flow-based learning, which will emit a brief traffic preamble beforestarting the actual test. This can be used to ensure that the flow-based switch has learned allnecessary addresses, preventing a latency spike.Valkyrie1564 provides full support for both the configuration and performance test typesdescribed in Y.1564 for complete validation of Ethernet Service Level Agreements (SLAs) in asingle test.ValkyrieTimeSynch enables multiple ValkyrieCompact or ValkyrieBay test chassis to synchronizetheir local time to each other. This can be used for One-Way Latency (OWL) measurementsbetween two test chassis, synchronized traffic start between multiple chassis and accuratetimestamping of captured packets in exported PCAP files. ValkyrieTimeSynch is compatible withValkyrie1564 test methodology and can be used for Y.1564 OWL measurements.WHITE PAPER Xena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – www.xenanetworks.com12

Valkyrie2889 is an application for benchmarking the performance of Layer 2 LAN switches. Thefollowing RFC 2889 test types are supported: All Throughput and Forwarding rate tests (both Fully and Partially meshed) Congestion Control Address Caching Capacity Address Learning Rate Broadcast Frame Forwarding and Latency Forward Pressure and Maximum Forwarding RateValkyrie3918 makes it easy to create, edit and execute all test types specified in RFC 3918. RFC3918 describes tests for measuring and reporting the throughput, forwarding, latency andInternet Group Management Protocol (IGMP) group membership characteristics of devices thatsupport IP multicast protocols.ValkyrieCLI is another free application for ValkyrieBay and ValkyrieCompact. It is a powerful andeasy-to-use command-line-interface (CLI) scripting API that makes test automation easier for testengineers: Ideal for test automation of e.g. production environments Controls ValkyrieBay and ValkyrieCompact chassis with installed test modules Powerful CLI approach from any TCP/IP capable tool environment Unified syntax for CLI- and GUI-generated test port configurations makes it easy to learn Script examples of Tcl, Perl, Java, Ruby, BASH and Python available Intelligent console tool bundled free with ValkyrieManagerCONCLUSIONTraditional communication networks have been built with dedicated, autonomous devices likerouters, switches, and firewalls, each being designed for a specific task. The result is totallydistributed network control, where decisions on data forwarding are done in the network devicesbased on various algorithms and protocols. Once set up and configured the network would workfine, but changing the behavior of the network can be costly, cumbersome and time consuming –if possible at all.Software-Defined Networking (SDN) is generating huge interest because it promises to addhigher flexibility and faster configuration of networks.WHITE PAPERFigure 9: ValkyrieTimeSynch synchronizes reference time for the Xena Valkyrie Layer 2-3testers.Xena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – www.xenanetworks.com13

WHITE PAPERIt is important to know the performance of SDN devices likeOpenFlow switches. This can provide information on how wellthe switch performs OpenFlow message processing and datapacket forwarding. The Valkyrie L2-3 test platformsValkyrieBay and ValkyrieCompact equipped with appropriatetest modules provide the features needed to quickly and costeffectively test and verify the OpenFlow performance ofswitches in the SDN.Xena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – www.xenanetworks.com14

OpenFlow Specification Version Released 1.0.0 December 2009 1.1.0 February 2011 1.2 December 2011 1.3.0 June 2012 1.3.1 September 2012 1.4.0 October 2013 1.5.0 December 2014 1.5.1 March 2015 Table 1: A selection of the OpenFlow switch specifications published by ONF OpenFlow products will in most cases comply with specification version 1.0.0 .