WIXLIB

NETWORKSthey took their ideas to the IETF (Internet Engineering Task Force) to standardize these protocols, butthis was a slow process that ultimately frustrated many researchers.In response, some networking researchers pursued an alternative approach of opening upnetwork control, roughly based on the analogy of reprogramming a stand-alone PC with relativeease. Conventional networks are not “programmable” in any meaningful sense of the word. Activenetworking represented a radical approach to network control by envisioning a programminginterface (or network API) that exposed resources (e.g., processing, storage, and packet queues) onindividual network nodes and supported the construction of custom functionality to apply to asubset of packets passing through the node.This approach was anathema to many in the Internet community who advocated that simplicityin the network core was critical to Internet success. The active networks research program exploredradical alternatives to the services provided by the traditional Internet stack via IP or ATM(Asynchronous Transfer Mode), the other dominant networking approach of the early 1990s. In thissense, active networking was the first in a series of clean-slate approaches to network architecture14subsequently pursued in programs such as GENI (Global Environment for Network Innovations)33and NSF FIND (Future Internet Design)31 in the United States, and EU FIRE (Future Internet Researchand Experimentation Initiative)32 in the European Union.The active networking community pursued two programming models: The capsule model, where the code to execute at the nodes was carried in-band in data packets.82 The programmable router/switch model, where the code to execute at the nodes was established byout-of-band mechanisms.8,68The capsule model came to be most closely associated with active networking. In intellectualconnection to subsequent efforts, though, both models have a lasting legacy. Capsules envisionedinstallation of new data-plane functionality across a network, carrying code in data packets (as inearlier work on packet radio88) and using caching to improve the efficiency of code distribution.Programmable routers placed decisions about extensibility directly in the hands of the networkoperator.Technology push and use pull. The “technology pushes” that encouraged active networkingincluded a reduction in the cost of computing, allowing more processing in the network; advancesin programming languages such as Java that offered platform portability and some code executionsafety; and virtual machine technology that protected the host machine (in this case the activenode) and other processes from misbehaving programs.70 Some active-networking research projectsalso capitalized on advances in rapid code compilation and formal methods.An important catalyst in the active-networking ecosystem was funding agency interest, inparticular the Active Networks program created and supported by DARPA (U.S. Defense AdvancedResearch Projects Agency) from the mid-1990s into the early 2000s. Although not all research intoactive networks was funded by DARPA, the funding program supported a collection of projectsand, perhaps more important, encouraged convergence on a terminology and set of active networkcomponents so that projects could contribute to a whole meant to be greater than the sum of theparts.14 The Active Networks program emphasized demonstrations and project interoperability,with a concomitant level of development effort. The bold and concerted push from a fundingagency in the absence of near-term use cases may have also contributed to a degree of communityskepticism about active networking that was often healthy but could border on hostility, and it may4

NETWORKShave obscured some of the intellectual connections between that work and later efforts to providenetwork programmability.The “use pulls” for active networking described in the literature of the time15,74 are remarkablysimilar to the examples used to motivate SDN today. The issues of the day included network serviceprovider frustration with the time needed to develop and deploy new network services (so-callednetwork ossification); third-party interest in value-added, fine-grained control to dynamically meetthe needs of particular applications or network conditions; and researcher desire for a platform thatwould support experimentation at scale. Additionally, many early papers on active networking citedthe proliferation of middleboxes, including firewalls, proxies, and transcoders, each of which had tobe deployed separately and entailed a distinct (often vendor-specific) programming model. Activenetworking offered a vision of unified control over these middleboxes that could ultimately replacethe ad hoc, one-off approaches to managing and controlling these boxes.74 Interestingly, the earlyliterature foreshadows the current trends in NFV (network functions virtualization),19 which alsoaims to provide a unifying control framework for networks that have complex middlebox functionsdeployed throughout.Intellectual contributions. Active networks offered intellectual contributions that relate to SDN.Here are three of particular note: Programmable functions in the network that lower the barrier to innovation. Research inactive networks pioneered the notion of programmable networks as a way of lowering the barrierto network innovation. The notion that it is difficult to innovate in a production network and pleasfor increased programmability were commonly cited in the initial motivation for SDN. Much of theearly vision for SDN focused on control-plane programmability, whereas active networks focusedmore on data-plane programmability. That said, data-plane programmability has continued todevelop in parallel with control-plane efforts,5, 21 and data-plane programmability is again coming tothe forefront in the emerging NFV initiative. Recent work on SDN is exploring the evolution of SDNprotocols such as OpenFlow to support a wider range of data-plane functions.11 Also, the concepts ofisolation of experimental traffic from normal traffic—which have their roots in active networking—also appear front and center in design documents for OpenFlow51 and other SDN technologies (e.g.,FlowVisor29). Network virtualization and the ability to demultiplex to software programs based on packetheaders. The need to support experimentation with multiple programming models led to workon network virtualization. Active networking produced an architectural framework that describesthe components of such a platform.13 The key components of this platform are a shared NodeOS(node operating system) that manages shared resources; a set of EEs (execution environments), eachof which defines a virtual machine for packet operations; and a set of AAs (active applications)that work within a given EE to provide an end-to-end service. Directing packets to a particularEE depends on fast pattern matching on header fields and demultiplexing to the appropriate EE.Interestingly, this model was carried forward in the PlanetLab60 architecture, whereby differentexperiments run in virtual execution environments and packets are demultiplexed into theappropriate execution environment on their packet headers. Demultiplexing packets into differentvirtual execution environments has also been applied to the design of virtualized programmablehardware data planes.5 The vision of a unified architecture for middlebox orchestration. Although the vision was never5

NETWORKSfully realized in the active-networking research program, early design documents cited the need forunifying the wide range of middlebox functions with a common, safe programming framework.Although this vision may not have directly influenced the more recent work on NFV, various lessonsfrom active-networking research may prove useful as the application of SDN-based control andorchestration of middleboxes moves forward.Myths and misconceptions. Active networking included the notion that a network API would beavailable to end users who originate and receive packets, though most in the research communityfully recognized that end-user network programmers would be rare.15 The misconception thatpackets would necessarily carry Java code written by end users made it possible to dismiss activenetwork research as too far removed from real networks and inherently unsafe. Active networkingwas also criticized at the time for its inability to offer practical performance and security. Whileperformance was not a first-order consideration of the active-networking research community(which focused on architecture, programming models, and platforms), some efforts aimed to buildhigh-performance active routers.84 Similarly, while security was under-addressed in many of theearly projects, the secure active network environment (SANE) architecture project2 was a notableexception.In search of pragmatism. Although active networks articulated a vision of programmablenetworks, the technologies did not see widespread deployment. Perhaps one of the biggest stumblingblocks was the lack of an immediately compelling problem or a clear path to deployment. Asignificant lesson from the active-network research effort was that killer applications for the dataplane are hard to conceive. The community proffered various applications that could benefit fromin-network processing, including information fusion, caching and content distribution, networkmanagement, and application-specific quality of service.15,74 Unfortunately, although performancebenefits could be quantified in the lab, none of these applications demonstrated a sufficientlycompelling solution to a pressing need.Subsequent efforts, described in the next subsection, focused more narrowly on routing andconfiguration management. In addition to a narrower scope, the next phase of research developedtechnologies that drew a clear distinction and separation between the functions of the control anddata planes. This separation ultimately made it possible to focus on innovations in the control plane,which not only needed a significant overhaul but, because it is commonly implemented in software,presented a lower barrier to innovation than the data plane.SEPARATING CONTROL AND DATA PLANESIn the early 2000s, increasing traffic volumes and a greater emphasis on network reliability,predictability, and performance led network operators to seek better approaches to certain networkmanagement functions, such as control of the paths used to deliver traffic (commonly known astraffic engineering). The means for traffic engineering using conventional routing protocols wereprimitive at best. Operators’ frustration with these approaches was recognized by a small, wellsituated community of researchers who either worked for or regularly interacted with backbonenetwork operators. These researchers explored pragmatic, near-term approaches that were eitherstandards-driven or imminently deployable using existing protocols.Specifically, conventional routers and switches embody a tight integration between the controland data planes. This coupling made various network management tasks, such as debugging6

NETWORKSconfiguration problems and predicting or controlling routing behavior, exceedingly challenging. Toaddress these challenges, various efforts to separate the data and control planes began to emerge.Technology push and use pull. As the Internet flourished in the 1990s, the link speeds inbackbone networks grew rapidly, leading equipment vendors to implement packet-forwarding logicdirectly in hardware, separate from the control-plane software. In addition, ISPs (Internet serviceproviders) were struggling to manage the increasing size and scope of their networks, as well as thedemands for greater reliability and new services (such as virtual private networks). In parallel withthese trends, the rapid advances in commodity computing platforms meant that servers often hadsubstantially more memory and processing resources than the control-plane processor of a routerdeployed just one or two years earlier. These trends catalyzed two innovations: An open interface between the control and data planes, such as the ForCES (Forwarding andControl Element Separation)86 interface standardized by the IETF and the Netlink interface to thekernel-level packet-forwarding functionality in Linux.65 Logically centralized control of the network, as seen in the RCP (Routing Control Platform)12,26and SoftRouter47 architectures, as well as the PCE (Path Computation Element)25 protocol at the IETF.These innovations were driven by industry’s demands for technologies to manage routing withinan ISP network. Some early proposals for separating the data and control planes also came fromacademic circles, in both ATM10,30,78 and active networks.69Compared with earlier research on active networking, these projects focused on pressing problemsin network management, with an emphasis on innovation by and for network administrators (ratherthan end users and researchers); programmability in the control plane (rather than the data plane);and network-wide visibility and control (rather than device-level configuration).Network management applications included selecting better network paths based on the currenttraffic load, minimizing transient disruptions during planned routing changes, giving customernetworks more control over the flow of traffic, and redirecting or dropping suspected attack traffic.Several control applications ran in operational ISP networks using legacy routers, including theIRSCP (Intelligent Route Service Control Point) deployed to offer value-added services for virtualprivate network customers in AT&T’s tier-1 backbone network.77 Although much of the work duringthis time focused on managing routing within a single ISP, some work25,26 also proposed ways toenable flexible route control across multiple administrative domains.Moving control functionality off of network equipment and into separate servers made sensebecause network management is, by definition, a network-wide activity. Logically centralized routingcontrollers12,47,77 were made possible by the emergence of open-source routing software9,40,64 thatlowered the barrier to creating prototype implementations. The advances in server technology meantthat a single commodity server could store all of the routing state and compute all of the routingdecisions for a large ISP network.12,79 This, in turn, enabled simple primary-backup replicationstrategies, where backup servers store the same state and perform the same computation as theprimary server, to ensure controller reliability.Intellectual contributions. The initial attempts to separate the control and data planes wererelatively pragmatic, but they represented a significant conceptual departure from the Internet’sconventional tight coupling of path computation and packet forwarding. The efforts to separate thenetwork’s control and data planes resulted in several concepts that have been carried forward insubsequent SDN designs:7

NETWORKS Logically centralized control using an open interface to the data plane. The ForCES workinggroup at the IETF proposed a standard, open interface to the data plane to enable innovation incontrol-plane software. The SoftRouter47 used the ForCES API to allow a separate controller to installforwarding table entries in the data plane, allowing the complete removal of control functionalityfrom the routers. Unfortunately, ForCES was not adopted by the major router vendors, whichhampered incremental deployment. Rather than waiting for new, open APIs to emerge, the RCP12,26used an existing standard control-plane protocol (the Border Gateway Protocol) to install forwardingtable entries in legacy routers, allowing immediate deployment. OpenFlow also faced similarbackward compatibility challenges and constraints: in particular, the initial OpenFlow specificationrelied on backward compatibility with hardware capabilities of commodity switches. Distributed state management. Logically centralized route controllers faced challengesinvolving distributed state management. A logically centralized controller must be replicated tocope with controller failure, but replication introduces the potential for inconsistent state acrossreplicas. Researchers explored the likely failure scenarios and consistency requirements. At least inthe case of routing control, the controller replicas did not need a general state management protocol,since each replica would eventually compute the same routes (after learning the same topology androuting information), and transient disruptions during routing protocol convergence were acceptableeven with legacy protocols.12 For better scalability, each controller instance could be responsiblefor a separate portion of the topology. These controller instances could then exchange routinginformation with each other to ensure consistent decisions.79 The challenges of building distributedcontrollers would arise again several years later in the context of distributed SDN controllers.46,55These controllers face the far more general problem of supporting arbitrary controller applications,requiring more sophisticated solutions for distributed state management.Myths and misconceptions. When these new architectures were proposed, critics viewed themwith healthy skepticism, often vehemently arguing that logically centralized route control wouldviolate fate sharing, since the controller could fail independently from the devices responsible forforwarding traffic. Many network operators and researchers viewed separating the control and dataplanes as an inherently bad idea, as initially there was no clear articulation of how these networkswould continue to operate correctly if a controller failed. Skeptics also worried that logicallycentralized control moved away from the conceptually simple model of the routers achievingdistributed consensus, where they all (eventually) have a common view of network state (e.g.,through flooding). In logically centralized control, each router has only a purely local view of theoutcome of the route selection process.In fact, by the time these projects took root, even the traditional distributed routing solutionsalready violated these principles. Moving packet-forwarding logic into hardware meant that a router’scontrol-plane software could fail independently from the data plane. Similarly, distributed routingprotocols adopted scaling techniques, such as OSPF (Open Shortest Path First) areas and BGP (BorderGateway Protocol) route reflectors, where routers in one region of a network had limited visibilityinto the routing information in other regions. As discussed in the next section, the separation ofthe control and data planes somewhat paradoxically enabled researchers to think more clearlyabout distributed state management: the decoupling of the control and data planes catalyzed theemergence of a state-management layer that maintains a consistent view of network state.In search of generality. Dominant equipment vendors had little incentive to adopt standard data-8

NETWORKSplane APIs such as ForCES, since open APIs could attract new entrants into the marketplace. Theresulting need to rely on existing routing protocols to control the data plane imposed significantlimitations on the range of applications that programmable controllers could support. ConventionalIP routing protocols compute routes for destination IP address blocks, rather than providing a widerrange of functionality (e.g., dropping, flooding, or modifying packets) based on a wider range ofheader fields (e.g., MAC and IP addresses, TCP and UDP port numbers), as OpenFlow does. In theend, although the industry prototypes and standardization efforts made some progress, widespreadadoption remained elusive.To broaden the vision of control- and data-plane separation, researchers started exploring cleanslate architectures for logically centralized control. The 4D project35 advocated four main layers:the data plane (for processing packets based on configurable rules); the discovery plane (for collectingtopology and traffic measurements); the dissemination plane (for installing packet-processing rules);and a decision plane (consisting of logically centralized controllers that convert network-levelobjectives into packet-handling state). Several groups proceeded to design and build systems thatapplied this high-level approach to new application areas,16,85 beyond route control. In particular,the Ethane project16 (and its direct predecessor, SANE17) created a logically centralized, flow-levelsolution for access control in enterprise networks. Ethane reduces the switches to flow tables thatare populated by the controller based on high-level security policies. The Ethane project, and itsoperational deployment in the Stanford computer science department, set the stage for the creationof OpenFlow. In particular, the simple switch design in Ethane became the basis of the originalOpenFlow API.OPENFLOW AND NETWORK OPERATING SYSTEMIn the mid-2000s, researchers and funding agencies gained interest in the idea of networkexperim

role throughout the historical evolution of SDN, substantially predating SDN yet taking root as one of the first significant use cases for SDN. ACTIVE NETWORKING The early to mid-1990s saw the Internet take off, with applications and appeal that far outpaced the early applications of file transfer and e-mail for scientists.