As businesses look to cut computing costs, improve service levels and adopt new delivery mechanisms such as cloud computing, it is causing them to rethink their network architecture.
As businesses look to cut computing costs, improve service levels and adopt new delivery mechanisms such as cloud computing, it is causing them to rethink their network architecture. In doing so, they find using a hierarchical Ethernet structure and multiple protocols no longer meet their needs. To provide greater flexibility and performance, they are turning to Ethernet fabric switches.
“Traditionally, data centers deployed a number of dedicated bandwidth technologies – such as T1. DS3 circuits, MPLS, SONET and Fibre Channel — according to data/throughput requirements,” says Charles King, principal analyst, Pund-IT, Inc. in Hayward, Calif. “Deploying Ethernet fabric is less complex and costly, yet it still supports levels of service adequate for many if not most organizations.”
What is an Ethernet Fabric?
In its September 2010 white paper Clearing the Confusion About Fabric-Based Infrastructure: A Taxonomy, Gartner defines a fabric as “a high-speed switched interconnect that links data center physical resources such as blade and rack servers with storage. In the LAN market, it has been called a network while the SAN market adopted the term ‘fabric.’ As vendors sought to unify data center interconnects around a common Ethernet-based technology, the term ‘fabric’ has been adopted. … Fabrics are used to aggregate components into a system.”
Ethernet was developed in the early 1970s, with standards adoption and commercialization occurring in the 1980s when it started displacing the Token Ring and Token Bus technologies. Ethernet was well designed for the computing architectures of its time, when servers and applications stayed in one spot for years. It doesn’t meet the dynamic needs of today’s data centers where virtualization and cloud computing mean rapidly shifting of resources or where iSCSI and FCoE use Ethernet to move massive amounts of storage traffic.
“In general, the migration to cloud computing will increase the amount of interprocess communication (the amount of cross-talk between application components), which happens horizontally across the data center network. If the cloud is big and complex enough, this traffic forces users to build switch hierarchies that exhibit increasingly large performance variations depending on which two systems are communicating, and that presents a large failure risk as a collection,” says Tom Nolle, president of CIMI Corp. in Voorhees, N.J. “That’s what moves you out of Ethernet and into fabrics.”
Traditional Ethernet employs a hierarchical structure with a single connection (inter-switch link or ISL) between any two switches. Not only does this mean added latency as the traffic has to move through additional stops along the way, but bottlenecks at that single point of connection. An Ethernet fabric, like a storage fabric, replaces the hierarchical structure with flat, point-to-point connections, and offers multiple paths so the data can always take the shortest path which eliminates bottlenecks and boosts performance.
“A fabric is deterministic in that its performance is the same between any pair of ports, which isn’t true with a hierarchical LAN switching array,” says Nolle. “That means that you don’t see variations in performance depending on where components of software might run, and that means you can allocate server resources to software more effectively.”
Ethernet fabrics support tree, ring, mesh and core/edge network topologies, giving administrators greater flexibility in designing services and network architectures that meet their business needs.
An Ethernet fabric allows data centers to adopt new technologies without abandoning familiar Ethernet. In addition to providing faster connections through a flatter architecture, it also offers greater scalability, efficiency and ease of administration.
Traditional Ethernet uses Spanning Tree Protocol (STP), which allows just a single path between switches. Although the later development of link aggregation groups permits multiple physical links to share a link, this still must be manually configured on every port, which gets increasingly complex as the network scales and new switches are added. With Ethernet fabric, no configuration is required. When a new switch is added to the fabric, it discovers the other switches and devices forming the fabric and the rules and protocols in force. Logical trunks automatically form whenever more than one link is needed between two switches.
Ethernet fabric is also more efficient since it uses link state routing with equal-cost multipath routes rather than STP. This allows all traffic to take the shortest path available at that moment in time, rather than having to route through a preconfigured path through an aggregation group and ISL. Whenever the network topology changes due to adding or removing links, it does not disrupt other network traffic.
Finally, an Ethernet fabric is simpler to administer than traditional Ethernet networks. Since the fabrics have distributed intelligence, a common configuration can be entered and shared by all switch ports in the fabric. When a virtual machine migrates from one port to another on the network, the QoS, security and other policies migrate along with the VM without the administrator having to intervene. When a switch is added to the fabric, it gathers the configuration information from other switches and also notifies them of its existence. Similarly, when a new server, disk array or other device connects to the fabric, all the switches are aware of its presence.
All this means that that the administrators can do their job once — setting the security policies, QoS and protocols for the entire fabric, instead of having to redo it every time a new device is added or removed, a link goes down, a VM moves to a new host or a new application or services is brought on line. No one enjoys doing repetitive grunt work. Ethernet fabric allows the network administrators to take on a more strategic role within their organization.
“Many, if not most, I/O and networking roads lead to Ethernet,” says Greg Schulz, founder of the Server and StorageIO Group in Stillwater, Minn., and author of the book Cloud and Virtual Data Storage Networking. “Granted, it may not be the Ethernet you have been using, however with FCoE, NFS, CIFS, object access, converged voice and video, the importance of Ethernet in data centers becomes more important.”