Designing and Building Data Center Fabrics
Home » Courses » Data Center Infrastructure » Designing and Building Data Center Fabrics
Designing and Building Data Center Fabrics
Introduction to Data Center Fabrics |
||
54:02 What is a Data Center Fabric |
||
Before starting the Designing and Building Data Center Fabrics online course, let’s try to answer a seemingly simple question “what is a data center fabric?” |
||
What Is a Data Center Fabric | 16:33 | 2016-05-19 |
Market Overview | 18:16 | 2019-01-03 |
Choosing a Solution | 12:39 | 2019-01-03 |
Keep It Simple | 6:34 | 2018-11-22 |
Test your knowledge |
||
Review Questions | 152K | 2017-10-09 |
1:22:50 Data Center Fabric Requirements |
||
What services should your data center fabric provide? What requirements should you consider when designing it? This section lists a number of potential requirements and explains why they might be relevant to your design. |
||
Optimal Bandwidth Utilization | 15:59 | 2016-05-19 |
Redundant Edge Connectivity | 4:14 | 2016-05-19 |
Layer-2 Transport and VM Mobility | 9:03 | 2016-05-19 |
Enhanced Layer-2 Transport | 11:22 | 2016-05-19 |
Optimal Layer-3 Forwarding | 10:50 | 2016-05-19 |
Multiple Routing Domains | 5:35 | 2016-05-19 |
Storage Integration | 7:20 | 2016-05-19 |
Lossless Transport and Data Center Bridging | 4:46 | 2016-05-19 |
Table Sizes and Packet Buffers | 13:41 | 2016-05-19 |
Slide Deck |
||
Data Center Fabrics - Requirements | 9.6M | 2016-05-22 |
Test your knowledge |
||
Review Questions | 186K | 2017-10-09 |
3:54:41 Free items Typical Fabric Technologies Overview |
||
Data center fabrics often rely on technologies you might not find anywhere else in your network. This section describes those technologies and their impact on fabric design. |
||
3:03:22 Networking technologies |
||
Some of the technologies used in data center fabrics are standardized. Others are not, but they’re still used by most vendors in one form or another. This section describes some technologies you’ll find in most data center fabric solutions, including Multi-Chassis Link Aggregation (MLAG), Unified Forwarding Tables (UFT), chassis architectures, ASIC buffering, and 400 Gigabit Ethernet. |
||
MLAG Technology Overview | 17:28 | 2016-06-22 |
UFT Technology Overview | 13:00 | 2017-08-02 |
Speeds and Breakout Options | 6:42 | 2019-01-03 |
In March 2020, Mark Nowell did a deep dive into 400 GbE technology, optics, and form factors |
||
In April 2020, Pete Lumbis described ASIC architectures and behavior from software engineer's perspective. |
||
51:19 Storage technologies and lossless transport |
||
Read this first | 963 | 2022-12-12 |
Storage Protocols 101 | 11:12 | 2011-04-06 |
Fibre Channel over Ethernet | 14:10 | 2011-04-06 |
iSCSI | 6:26 | 2011-04-06 |
Long-Distance Storage | 3:06 | 2011-04-06 |
Storage Myths | 4:27 | 2011-04-06 |
Storage Q&A | 11:58 | 2011-04-06 |
Test your knowledge |
||
Review Questions | 177K | 2017-11-06 |
Leaf-and-Spine Fabrics |
||
1:24:47 Introduction to leaf-and-spine fabrics |
||
Traditional data center networks used a 3-tier design that was mostly mandated by hardware limitations, resulting in unequal bandwidth between endpoints based on their locations. In the last few years the networking industry rediscovered the work of Charles Clos (from 1953) and everyone started promoting leaf-and-spine fabrics. |
||
Challenges of Traditional Data Center Networks | 16:04 | 2017-11-05 |
Clos Networks and Leaf-and-Spine Fabrics | 16:39 | 2017-11-05 |
Additional Resources |
||
Slide deck | 1.3M | 2017-11-04 |
52:04 From the ipSpace.net Design Clinic |
||
Leaf-and-Spine Fabrics Outside of Data Centers | 22:59 | 2021-12-27 |
Integrating Storage in Leaf-and-Spine Fabrics | 22:32 | 2022-04-02 |
Migrating to a Leaf-and-Spine Fabric | 6:33 | 2022-04-02 |
Further Reading |
||
Demystifying DCN Topologies: Clos Networks | ||
Demystifying DCN Topologies: Fat trees and leaf-and-spine fabrics | ||
Test your knowledge |
||
Review Questions | 166K | 2017-11-04 |
45:54 Physical Leaf-and-Spine Fabric Design |
||
After mastering the basic principles of leaf-and-spine fabrics described in the Introduction section we're moving on to the physical design: how do you build a leaf-and-spine fabric given number of edge ports and oversubscription ratio? What if you need less than 100 ports? What if you need 50.000 ports? What do you do if you have to support low-speed edge interfaces? |
||
The examples in the following videos use switches with 10GE/40GE ports. Don't be bothered by that; the same considerations and design calculations apply to fabrics built with higher-speed switches, for example switches with 100GE server ports and 400GE uplinks. |
||
Physical Leaf-and-Spine Fabric Design | 14:55 | 2017-11-05 |
Small Fabrics and Lower-Speed Interfaces | 9:07 | 2017-11-05 |
Building Very Large Fabrics | 21:52 | 2017-11-05 |
Physical Leaf-and-Spine Fabric Design | 1.7M | 2017-11-04 |
Test your knowledge |
||
Review Questions | 172K | 2017-11-04 |
Design Assignment |
||
Build a Small Data Center Fabric | 124K | 2017-12-30 |
2:39:15 Free items Layer-3 Fabrics with Non-Redundant Server Connectivity |
||
We're starting the design part of the webinar with the simplest possible scenario – each leaf switch is a single IP subnet – and focus on routing protocol selection, route summarization, leaf-to-spine link aggregation, and core link addressing. The second half of this section describes detailed guidelines on using BGP and OSPF as the underlay routing protocols in a leaf-and-spine fabric. |
||
44:27 Overview and Design Principles |
||
Introduction to Leaf-and-Spine Designs | 4:39 | 2017-02-10 |
Layer-3 Fabric with Non-Redundant Server Connectivity | 6:52 | 2017-02-10 |
Routing Protocol Selection | 19:50 | 2017-02-10 |
Route Summarization and Link Aggregation | 6:40 | 2017-02-10 |
Core Link Addressing | 6:26 | 2017-02-10 |
Slide deck | 1.4M | 2016-03-25 |
1:04:14 Using BGP in Leaf-and-Spine Fabrics |
||
Based on the work done by Petr Lapukhov at Microsoft, every vendor talks about using BGP as the routing protocol in leaf-and-spine fabrics. Does it make sense? You'll find some of the answers in this section presented by Dinesh Dutt (Cumulus Networks). |
||
Using BGP in Leaf-and-Spine Fabrics | 10:19 | 2017-02-10 |
Simplifying BGP Configurations | 19:30 | 2017-02-10 |
Troubleshooting and Managing BGP | 8:19 | 2017-02-10 |
BGP in Data Centers - Sample Deployments | 3:15 | 2017-02-10 |
BGP in 3-tier Clos | 8:45 | 2021-04-03 |
BGP QA | 14:06 | 2021-04-03 |
Slide Deck: Operationalizing BGP in the Data Center | 1.9M | 2016-03-04 |
Slide Deck: BGP in 3-Tier Clos Topology | 890K | 2021-03-13 |
50:34 Using OSPF in Leaf-and-Spine Fabrics |
||
While everyone talks about using BGP or yet-to-be-implemented routing protocols in leaf-and-spine fabrics, OSPFv2 works just fine in some of the biggest fabrics in the world. This section describes some of the design guidelines you should use when deploying OSPF as a fabric IGP. |
||
Why Would You Use OSPF | 16:01 | 2021-04-03 |
OSPF Design | 20:02 | 2021-04-03 |
Configuration Snippets | 10:04 | 2021-04-03 |
OSPF Footnotes | 4:27 | 2021-04-03 |
OSPF in Clos Topology | 1.2M | 2021-03-13 |
Test your knowledge |
||
Review Questions | 202K | 2017-11-06 |
1:06:44 Layer-3 Fabrics with Redundant Server Connectivity |
||
After establishing the baseline in the Layer-3 fabrics with non-redundant server connectivity section we'll add complexity at the fabric edge: redundantly connected servers. |
||
Layer-3 Fabrics with Redundant Server Connectivity | 18:54 | 2016-12-13 |
Link Aggregation between Servers and Network | 5:26 | 2016-12-13 |
Active-Standby Server Connectivity | 8:18 | 2016-12-13 |
Slide deck | 1.7M | 2018-01-17 |
34:06 From the ipSpace.net Design Clinic |
||
Multi-Homed Servers | 34:06 | 2022-01-18 |
Test your knowledge |
||
Review Questions | 165K | 2017-12-31 |
1:50:15 Free items Layer-3-Only Data Centers |
||
Is it possible to build a pure layer-3 data center fabric that supports redundant server connectivity and IP address mobility? You'll find out in this section. |
||
6:53 Design Guidelines |
||
Host Routing | 6:53 | 2016-12-13 |
Slide deck | 1.3M | 2016-03-25 |
24:22 Building a Pure L3 Data Center with Cumulus Linux |
||
Building a Pure L3 Data Center with Cumulus Linux | 24:22 | 2016-12-13 |
Slide deck | 949K | 2016-03-29 |
Test your knowledge |
||
Review Questions | 157K | 2018-01-17 |
1:19:00 More to explore |
||
Layer-3-only Data Center Networks with Cumulus Linux on Software Gone Wild | ||
Enterasys Robust Data Center Interconnect Solutions | 1:19:00 | |
36:40 Free items Routing on Servers |
||
Another approach to building a pure layer-3 fabric is to extend the fabric routing protocol into the servers and announce servers' loopback IP addresses using BGP. |
||
Runinng Routing Protocols on Servers | 10:55 | 2016-12-13 |
Routing from Hosts - Deep Dive | 10:24 | 2016-12-13 |
Examples from Real World | 8:08 | 2016-12-13 |
7:13 From the ipSpace.net Design Clinic |
||
VXLAN and EVPN on Linux Hosts | 7:13 | 2022-01-18 |
Test your knowledge |
||
Review Questions | 153K | 2017-12-31 |
1:53:15 Free items Layer-2 Fabrics |
||
We're leaving the stable world of L3-only fabrics and entering the realm of large VLANs that most enterprise data centers have to deal with. We'll cover numerous design scenarios, from traditional bridging to routing on layer 2 and MAC-over-IP encapsulation. |
||
1:00:15 Design Guidelines |
||
Layer-2 Fabrics | 14:49 | 2017-03-22 |
Traditional Bridging | 10:05 | 2017-03-22 |
Routing on Layer-2 | 13:12 | 2017-03-22 |
MAC-over-IP Encapsulation | 13:25 | 2017-03-22 |
Redundant Server-to-Network Connectivity | 8:44 | 2017-03-22 |
Slide deck | 1.8M | 2016-04-01 |
53:00 Shortest Path Bridging in Avaya Fabric |
||
Avaya is one of the few data center switching vendors that still uses routing on layer 2 (SPB) technology instead of VXLAN encapsulation. In this guest presentation Roger Lapuh (Avaya) explains how SPB works and how you can use it to build layer-2 or layer-2+3 data center fabrics. |
||
Introduction to SPB and Avaya Fabric Connect | 18:25 | 2017-03-22 |
SPB Deep Dive | 18:17 | 2017-03-22 |
Building Data Center Fabrics with SPB | 16:18 | 2017-03-22 |
Slide deck | 2.1M | 2016-04-06 |
Test your knowledge |
||
Review Questions | 179K | 2017-12-31 |
Design Assignment |
||
Design a Robust Layer-2 Fabric | 236K | 2017-12-30 |
1:53:31 Free items Mixed Layer-2 + Layer-3 Fabrics |
||
Most data center fabrics have to combine elements of large VLANs and routing. In this section we'll explore the various combinations, from traditional routing on spine switches to anycast routing on leaf switches. |
||
31:05 Design Guidelines |
||
Layer-2+3 Fabrics | 6:45 | 2017-04-05 |
Routing on Spine Switches | 9:04 | 2017-04-05 |
Routing on Leaf Switches | 15:16 | 2017-04-05 |
Slide deck | 1.4M | 2016-04-20 |
1:18:52 VXLAN with BGP EVPN on Cisco Nexus OS |
||
Major data center switching vendors use VXLAN to build large layer-2 domains across IP fabrics, and EVPN control plane to build flooding trees and exchange MAC address reachability information. In this section Lukas Krattiger (guest speaker from Cisco Systems) explains how VXLAN transport and EVPN control plane work on Nexus switches. |
||
Overlays in Data Center Fabrics | 15:07 | 2017-04-05 |
Overview of VXLAN with BGP EVPN | 15:59 | 2017-04-05 |
Introduction to BGP EVPN | 15:29 | 2017-04-05 |
BGP EVPN Deep Dive | 15:39 | 2017-04-05 |
EVPN Integrated Routing and Bridging | 16:38 | 2017-04-05 |
Slide deck | 12M | 2016-04-21 |
3:34 From the ipSpace.net Design Clinic |
||
VXLAN and EVPN in Small Data Center | 3:34 | 2022-04-02 |
Test your knowledge |
||
Review Questions | 169K | 2017-12-31 |
2:16:42 Free items Stretching Leaf-and-Spine Fabrics |
||
Should you stretch a single fabric across multiple sites? Does it make sense to split a large fabric into smaller fabrics (pods)? What could you do to improve the scalability of VXLAN-based EVPN fabrics? This section contains the design guidelines and technology details you need to answer these questions. |
||
44:42 Multi-Site and Multi-Pod Fabrics Design Guidelines |
||
What Problem Are We Trying to Solve? | 13:38 | 2019-01-28 |
Physical Multi-Site Topologies | 11:54 | 2019-01-28 |
Data-, Control- and Management-Plane Failure Domains | 17:05 | 2019-01-28 |
Conclusions | 2:05 | 2019-01-28 |
Slide Deck | 2.1M | 2018-03-29 |
Related Podcasts |
||
Open-Source Hybrid Cloud Reference Architecture | ||
Related Webinars |
||
Data Center Interconnects | 4:31:00 | |
Focused exclusively on DCI issues, this webinar describes numerous designs and technologies (including local and global load balancers, VPLS, OTV and MPLS/VPN) that can be used to build active-active data centers. This webinar expands on the concepts described in the Data Center 3.0 for Networking Engineers webinar, which is thus a highly recommended prerequisite. |
||
Designing Active-Active and Disaster Recovery Data Centers | 3:37:00 | |
This webinar covers typical design scenarios encountered when building a disaster recovery data center or deploying multiple data centers in an active-active configuration. |
||
VMware NSX, Cisco ACI or Standard-Based EVPN | 6:30:00 | |
Are you building a new data center fabric? Trying to decide whether to use
VMware NSX or Cisco ACI? How about EVPN? This fast-paced webinar will help
you select the best option for your environment.
|
||
1:32:00 Using VXLAN and EVPN in Multi-Pod and Multi-Site Fabrics |
||
Introduction to Multi-Pod and Multi-Site Fabrics | 6:58 | 2019-01-28 |
Multi-Pod Fabrics | 27:04 | 2019-01-28 |
Multi-Site Fabrics | 33:19 | 2019-01-28 |
Multi-Site Packet Forwarding | 19:10 | 2019-01-28 |
Conclusions | 5:29 | 2019-01-28 |
Slide Deck | 29M | 2018-03-27 |
Related Webinars |
||
EVPN Technical Deep Dive | 11:49:00 | |
This webinar will help you grasp the EVPN fundamentals, potential use cases, and its benefits and drawbacks. The deep dive part of the webinar discussed technical details that will enable you to deploy and troubleshoot EVPN in multi-vendor environments. |
||
VXLAN Technical Deep Dive | 3:42:00 | |
VXLAN is a novel MAC-over-IP technology used to implement large-scale layer-2 multi-tenant virtual networking solutions within the VMware's vSphere ecosystem. This webinar will give you the architectural and technical details you need to design and deploy VXLAN-based virtual networks. It assumes familiarity with virtual networking concepts and VMware networking solutions. |
||
Test your knowledge |
||
Review Questions | 195K | 2018-12-17 |
Design Assignment |
||
Extend a Leaf-and-Spine Fabric | 105K | 2017-12-30 |
19:31 QoS and Load Balancing in Leaf-and-Spine Fabrics |
||
Do you need QoS in data center fabrics? How do you handle load balancing between elephant and mice flows? This section describes some of the answers. |
||
The following video was recorded in the Autumn 2016 Q&A session of the Building Next-Generation Data Center online course |
||
QoS and Load Balancing in Leaf-and-Spine Fabrics | 19:31 | 2017-11-06 |
Advanced Topics |
||
1:14:48 Whitebox Switching and Disaggregation |
||
This section describes the differences between SDN, disaggregation and whitebox switching, the major components you'll find in any disaggregated solution (from hardware abstraction layer to control-plane components) and the benefits and drawbacks of several disaggregated network operating systems. |
||
Whitebox and Disaggregation Drivers | 13:38 | 2017-09-17 |
Components | 18:19 | 2017-09-17 |
Network Operating Systems | 19:28 | 2017-09-17 |
Challenges | 13:12 | 2017-09-17 |
Questions | 10:11 | 2017-09-17 |
Additional resources |
||
Slide deck | 1.2M | 2017-05-09 |
Test your knowledge |
||
Review Questions | 167K | 2017-12-30 |
1:06:50 Large-Scale Open Data Center Fabrics |
||
In this section you'll learn how to build very large data center fabrics. The section covers:
|
||
Large Data Center Fabric Challenges | 16:54 | 2017-11-20 |
Pod Architecture and Basic Access Topologies | 6:39 | 2017-11-20 |
Single-SKU Data Center Networks | 9:44 | 2017-11-20 |
Hardware Challenges | 14:48 | 2017-11-20 |
Building Large Leaf-and-Spine Fabrics | 18:45 | 2017-11-20 |
Additional resources |
||
Slide deck | 4.5M | 2017-05-23 |
Test your knowledge |
||
Review Questions | 170K | 2017-12-30 |
56:19 Data Center Fabric Architectures |
||
As you analyze the vendor solutions, it quickly becomes obvious that every single one of them uses one of few typical control/management plane architectures, from completely decentralized model (sometimes heavily automated) to centralized control- or even data plane. This section describes most of the architectures you might encounter in real life and documents their benefits and drawbacks. |
||
Fabric Architectures - Introduction | 15:08 | 2016-06-22 |
Independent Devices | 4:42 | 2016-06-22 |
Shared Management Plane | 7:49 | 2016-06-22 |
Centralized Control Plane | 13:13 | 2016-06-22 |
Centralized Control Plane with Local Offload | 6:37 | 2016-06-22 |
Centralized Data Plane | 6:17 | 2016-06-22 |
Conclusions | 2:33 | 2016-06-22 |
Slide Deck |
||
Data Center Fabrics - Architectures | 2.3M | 2016-05-30 |
Test your knowledge |
||
Review Questions | 198K | 2017-12-30 |
1:11:40 Free items Fabrics, Buffers and Drops |
||
Buffering is a confusing and sometimes controversial topic when it comes to data center fabrics. In this short section JR Rivers discusses some of the related theory and practicalities as well as gives insight into tools that are available to networking specialists. |
||
52:20 Switch Buffering Deep Dive |
||
Switch Buffer Architectures | 28:54 | 2017-09-16 |
Small Buffer or Big Buffer Switches | 5:36 | 2017-09-16 |
Tools and Knobs | 17:50 | 2017-09-16 |
19:20 Questions and Answers |
||
Questions and Answers | 19:20 | 2017-09-16 |
Reference material |
||
Slide deck | 2.5M | 2016-11-09 |
Test your knowledge |
||
Review Questions | 192K | 2017-12-30 |
Reference material |
||
5:36:01 Reference: Data Center Fabric solutions |
||
In this section you’ll find an overview of data center fabric solutions from major data center switching vendors. The Data Center Fabrics webinar (included with this course) contains even more information. |
||
2:12:55 Arista EOS |
||
This section contains information on Arista’s data center switches and software features available in Arista EOS. |
||
Product Overview - Arista | 7:26 | 2019-01-03 |
Product Details - Arista | 10:26 | 2017-08-18 |
Software Features Release Overview - Arista | 28:42 | 2017-08-18 |
Highlighted Software Features - Arista EOS | 45:26 | 2017-08-18 |
Highlighted Software Features - Arista 7150 | 8:18 | 2014-07-22 |
OpenFlow Support - Arista | 13:36 | 2015-12-22 |
VXLAN Support - Arista | 19:01 | 2017-08-18 |
1:45:03 Cisco Nexus Switches |
||
This section covers Cisco’s Nexus series of data center switches and their Nexus-OS operating system. |
||
Product Overview - Cisco Nexus Switches | 9:00 | 2019-01-03 |
Product Details - Cisco | 3:35 | 2014-09-12 |
FabricPath, vPC and DFA | 17:37 | 2016-01-04 |
Software Release Overview - Nexus OS | 34:25 | 2017-08-21 |
Highlighted Software Features - Nexus OS | 19:45 | 2017-08-21 |
OpenFlow Support in Nexus Switches | 4:20 | 2017-08-21 |
VXLAN Support in Nexus Switches | 16:21 | 2017-08-21 |
1:38:03 Juniper QFX- and EX Series |
||
This section covers Juniper’s QFX- and EX-series of data center switches and the data center features of Junos. |
||
Product Overview - Juniper Data Center Switches | 6:04 | 2019-01-03 |
Product Details - QFX- and EX-Series Switches | 9:57 | 2016-01-04 |
Product Details - QFabric | 8:08 | 2014-10-17 |
Virtual Chassis | 10:30 | 2014-10-17 |
Junos Software Release Overview | 19:42 | 2018-11-23 |
Highlighted Software Features | 23:45 | 2018-11-23 |
OpenFlow Support on Juniper Data Center Switches | 2:54 | 2016-01-04 |
VXLAN Support on Juniper Data Center Switches | 17:03 | 2018-11-23 |
51:19:00 Relevant ipSpace.net Webinars |
||
All webinars in this section are included with your online course. Some of them are part of the core course content, others are included as reference material. |
||
Data Center Fabric Architectures | 20:27:00 | |
Leaf-and-Spine Fabric Architectures | 13:07:00 | |
Data Center Infrastructure for Networking Engineers | 10:40:00 | |
Open Networking for Large-Scale Networks | 2:22:00 | |
Networks, Buffers, and Drops | 1:12:00 | |
ExpertExpress Case Studies | 51:19:00 | |
Introduction to Virtualized Networking | 3:31:00 |