Diversifying the Internet

In a relatively short period of time, the Internet has become a critical infrastructure component of our global information-based society. However, as it has grown, it has developed in ways never anticipated by its original designers [CL03], leading to a wide range of ad hoc extensions designed to satisfy new requirements, but never integrated into the Internet architecture in a systematic way. Over time, it has become increasingly difficult to deploy significant improvements to the core protocols on which the Internet is based. This difficulty stems primarily from two factors: the significant capital investment required to effect major change and the need for universal agreement before fundamental improvements can be usefully deployed. The latter factor is the more difficult to overcome, since the competing interests of Internet stakeholders make universal agreement very hard to come by.

There is a growing recognition that virtualization offers a means to break this impasse [AN05]. A virtualized testbed [GE06] has been proposed to enable experimentation with new network architectures and real-world deployments of sufficient scale to enable realistic evaluation and to stimulate adoption by the commercial sector. Virtualization is already a common feature of the Internet. We have virtual LANs, that allow organizations to manage traffic flows along organizational boundaries [SM98], virtual private networks that enable large enterprises to provide more secure communication among remote sites [KO98], a wide range of link virtualization technologies (frame relay, IP tunnels, ATM, MPLS), and a growing array of host virtualization mechanisms that range from user-level virtualization techniques to hardware mechanisms that virtualize processor, memory, disk and network interfaces [IN06]. We have also seen a rapid increase in the use of virtual networks that are implemented as overlays on top of the Internet, either to enhance the performance of conventional applications or to enable new kinds of services [AN03, SA98, SB04]. This notion of network virtualization has been taken still further in the PlanetLab testbed [PE02, CH03], which virtualizes an overlay network infrastructure, to allow new overlay networks to be rapidly deployed and evaluated, without the need to install any new physical equipment. A variety of other projects have advanced the use of virtualized networks as well [CA99, HA98, TO01, WA02].

While virtualization has become a common fact of life throughout the Internet, it has never been integrated into the Internet architecture in any systematic way. Virtualization is used by network administrators to organize traffic flows within their own domains, but is applied in a largely ad hoc and mostly manual fashion. The growing popularity of overlay and peer-to-peer networks shows that higher level applications of virtualization, enabled by automated mechanisms, can be extremely powerful and broadly useful. By integrating these ideas into the architecture of the Internet, they can be applied in a more systematic way and can stimulate the development of more flexible network platforms, capable of delivering sophisticated services to a mass market.

In this project, we explore the implications of making virtualization a central architectural component of a new Internet architecture, with the explicit objective of encouraging a diversity of end-to-end networks. The creation of such a diversified Internet has the potential to break down the barriers to entry that currently inhibit innovation in networking and enable competitive forces to operate more freely, stimulating the development and deployment of new network technologies and advanced applications. We are exploring how virtualization can be integrated into the Internet architecture and how it can be delivered, by multiple cooperating organizations, on a large scale and with sufficiently high performance to make it economically compelling. We are developing a complete architecture for a diversified Internet and plan to demonstrate major components of that architecture experimentally. The project will include the development of the underlying protocols and mechanisms for enabling multiple end-to-end networks to co-exist within a shared infrastructure that is owned and operated by multiple organizations. It will also explore how these mechanisms can be effectively used, by developing and demonstrating two complete end-to-end networks that operate within the shared infrastructure.

Design Principles for a Diversified Internet

The role of the substrate is to provide resources and support services to metanetworks. Metanetworks are implemented by metarouters, which are hosted by substrate routers, and are joined to each other by metalinks. Substrate routers provide a collection of generic processing resources that can be programmed to provide the packet processing required by different metanetworks. These resources can be divided among metarouters in a variety of ways, depending on the nature of the particular processing resource. The substrate can be viewed as a new protocol layer that lies between layers 2 and 3 of the OSI model. In a diversified Internet, the substrate layer forms the narrow waist of the Internet hourglass, allowing variation in both the end-to-end networks that lie above it and the link layer subnets that lie below it. These concepts are illustrated at right.

The design of our diversified network model is driven by a few key architectural principles. These are guiding the design choices being made as specific protocols and mechanisms are developed.

• Support varied roles of participants. We recognize three major classes of participants in diversified networks: end users, substrate network providers and metanetwork providers. Each has an important role to play. To ensure that the overall system operates effectively, its important to provide mechanisms that enable each class of participants to contribute to the overall operation of the net, while pursuing their own individual objectives.
• Interoperability among substrate domains. If diversified networking is to become the basis for a new Internet, it must be possible for multiple organizations to provide the substrate functionality. At the same time, its important that the system operate as a cohesive whole, in spite of its distributed management, so that metanet providers can deploy services globally and serve end users anywhere in the world.
• Architectural neutrality. The substrate should be architecturally neutral, with respect to the metanetworks. This means that it should place as few limits as possible on metanetworks and should not favor any specific subclass of metanets by providing capabilities that can be provided by metanets themselves. While this may imply some duplication of effort by different metanets, our view is that this is preferable to embedding capabilities in the substrate that different metanets may prefer to provide in different ways.
• Design for security. In keeping with the architectural neutrality principle, we allow metanetworks to provide varying levels of security. In particular, it must be possible to support metanetworks that provide robust, secure communication. At the same time, the substrate should not dictate how metanets provide security and should avoid placing excessive burdens on users and metanets for which security is not a central concern. Moreover, this must be done in a context where not all substrate domains can be trusted.
• Design for mobility. Todays Internet was designed to enable communication among computers at static locations. Today, statically connected devices are the exception, rather than the rule, and the Internet has accommodated mobile devices using a variety of ad hoc extensions. Any new Internet architecture should be designed with mobility in mind. In a diversified Internet, we must find ways to support mobility while maintaining architectural neutrality.
• Adaptability to lower level network technologies. The substrate should be able to exploit the full range of existing link technologies (including optical, wireless and virtual) as well as future link technologies. In addition, substrate routers should be amenable to a wide range of implementations, and should be capable of incorporating new technology components as they become available.
• Expose lower level technology characteristics to metanets. To enable metanets to take full advantage of the underlying links, their properties (e.g. multi-access, wireless, bit error rate) should be visible to metanets. Similarly, the characteristics of the metarouter implementation technologies should be visible to metanets, so that they can take full advantage of the underlying technology capabilities.

• Metalink abstraction. Metalinks are an abstraction of an underlying physical link. Point-to-point metalinks have two end points and may have a provisioned bandwidth (although they need not). Metanetworks can specify required characteristics for a metalink during metanet configuration, although substrates may not always be able to comply with these specifications. Multipoint metalinks can have an arbitrary number of endpoints and support contention-based multiple access. The specific form this takes depends on the underlying physical link technology. The abstraction supports only best-effort traffic over multipoint metalinks, but does support a bandwidth-provisioned interface to a multipoint metalink.
• Metarouter abstraction. To enable a wide range of metarouter implementations, the model assumes no particular implementation technology. The substrate provides a meta-interface abstraction to metarouters, where a meta-interface is the logical endpoint for a metalink (either point-to-point or multipoint). Metarouters send and receive packets across meta-interfaces and it is up to the substrate to handle the mapping between a meta-interface and the associated metalink. The substrate provides packet processing resources to metarouters in the form of Processing Engines (PE), which can take various forms, including virtual machines running on a conventional computer, network processor subsystems, or FPGAs. Metarouters that use multiple PEs may interconnect them through a metaswitch that provides an abstraction of a nonblocking switch (see Figure 1).
• Metaterminal abstraction. The physical devices that communicate over a diversified network infrastructure may span a wide range, from supercomputers, to PDAs. Some devices may be designed to operate in only a single metanetwork, while others will be able to use multiple metanetworks. Each device will implement one or more metaterminals, and in the case of multiple metaterminals, the substrate functions needed to allow metaterminals to co-exist. The metaterminal is an abstraction for the endpoint protocol stack of a communicating device. It sends and receives packets using one or more meta-interfaces. It also provides a high-level interface that can be used by applications. The nature of these interfaces is determined by the individual metanetworks and the characteristics of the hosting device (e.g., its operating system).

• Peering metalink. While most metalinks will connect components within a given metanetwork, we envision many situations where interconnection of metanets may be desirable. Such interconnection is provided by peering metalinks. Its up to the metanets terminating a peering metalink to address whatever interoperability issues may be present. From the substrates perspective, the primary difference between a peering metalink and an ordinary metalink is that its creation requires coordination with two different metanetworks.
• Gateway. While this proposal focuses primarily on diversified networking as a replacement for the current Internet, it makes sense for a diversified Internet to enable metanets to connect to other networks, including the current Internet, as well. A gateway is essentially a peering metalink through which a metanetwork can exchange packets with another network.
• Meta-cross-connects. Conventional networks use transport level cross-connects to enable rearrangement of transmission resources in response to changing traffic conditions or network failures. Such capabilities can also be useful to metanetworks. For this reason, our model includes a meta-cross-connect that a metanet can use to rearrange its transport level resources. A meta-cross-connect has a specified number of metalink interfaces that can be mapped to one another in a one-to-one fashion.