Introduction
Welcome to the comprehensive VXLAN Fabric Technology guide. This guide takes you from Layer 2 fundamentals through advanced VXLAN fabric architectures, preparing you for both implementation and interviews.
The Evolution of Data Center Networking
Part I: Layer 2 Fundamentals
Understanding the foundation before the evolution
Chapter 1: What is a VLAN?
✓ When to Use VLANs
- Segmenting broadcast domains
- Grouping devices by department/function
- Implementing security policies
- Small to medium networks (<4000 segments)
- Single data center deployments
✗ When NOT to Use VLANs
- Need more than 4094 segments
- Multi-datacenter connectivity
- Need VM mobility across locations
- Large-scale cloud environments
- Use VXLAN instead for these cases
Why VLANs Were Created
Without VLANs (Problem)
- Single broadcast domain
- All devices receive broadcasts
- Network congestion
- Security concerns
- No logical separation
With VLANs (Solution)
- Separate broadcast domains
- Traffic isolated by VLAN
- Reduced congestion
- Enhanced security
- Logical grouping achieved
VLAN Port Types
| Port Type | Description | Use Case |
|---|---|---|
| Access Port | Belongs to single VLAN | End devices (PCs, printers) |
| Trunk Port | Carries multiple VLANs | Switch-to-switch connections |
| Native VLAN | Untagged VLAN on trunk | Management, backward compatibility |
VLAN Fundamentals Diagram
Figure 1.1: VLAN network topology with port assignments and VLAN membership
802.1Q VLAN Tagging
Configuration Examples
Cisco IOS
! Create VLANs Switch(config)# vlan 10 Switch(config-vlan)# name Sales ! Configure Access Port Switch(config)# interface GigabitEthernet0/1 Switch(config-if)# switchport mode access Switch(config-if)# switchport access vlan 10 ! Configure Trunk Port Switch(config)# interface GigabitEthernet0/24 Switch(config-if)# switchport mode trunk Switch(config-if)# switchport trunk allowed vlan 10,20
Juniper Junos
# Create VLANs set vlans Sales vlan-id 10 # Configure Access Port set interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members Sales # Configure Trunk Port set interfaces ge-0/0/24 unit 0 family ethernet-switching port-mode trunk set interfaces ge-0/0/24 unit 0 family ethernet-switching vlan members [Sales Engineering]
VLAN Limitations
| Limitation | Impact | Modern Solution |
|---|---|---|
| 4,094 VLAN limit | Cannot scale for multi-tenant DCs | VXLAN (16M VNIs) |
| Spanning Tree required | Blocks redundant links | VXLAN + ECMP |
| Layer 2 boundary | VLANs don't cross routers | VXLAN overlay |
Chapter 2: How the CAM Table is Built
The Learning Process
MAC Address Learning Animation
Frame arrives on Port 1 with Source MAC: AA:AA:AA:AA:AA:AA
Switch learns: MAC AA:AA -> Port 1
Step-by-Step Process
- Frame Reception: Frame arrives on a switch port
- Source MAC Learning: Switch records source MAC + ingress port
- Destination Lookup: Switch checks if destination MAC is in table
- Forward or Flood: Known = forward to port; Unknown = flood to all ports in VLAN
- Timer Management: Entries age out (default 300 seconds)
CAM Table Learning Process
Figure 2.1: CAM table learning process showing frame arrival, MAC learning, and flooding
CAM Table Example
| MAC Address | Port | VLAN | Timer |
|---|---|---|---|
| AA:AA:AA:AA:AA:AA | Port 1 | 10 | 300s |
| BB:BB:BB:BB:BB:BB | Port 2 | 10 | 245s |
| CC:CC:CC:CC:CC:CC | Port 3 | 20 | 180s |
Verification Commands
! Cisco show mac address-table show mac address-table vlan 10 show mac address-table count ! Juniper show ethernet-switching table show ethernet-switching table vlan Sales
Chapter 3: First Hop Redundancy Protocols
FHRP Comparison
| Protocol | Full Name | Standard | Virtual MAC |
|---|---|---|---|
| HSRP | Hot Standby Router Protocol | Cisco Proprietary | 0000.0c07.acXX |
| VRRP | Virtual Router Redundancy Protocol | RFC 5798 | 0000.5e00.01XX |
| GLBP | Gateway Load Balancing Protocol | Cisco Proprietary | 0007.b400.XXYY |
Part II: Evolution to VXLAN
From traditional VLANs to modern overlay networks
Chapter 4: VXLAN Fundamentals
✓ When to Use VXLAN
- Large-scale environments (>4000 segments)
- Multi-datacenter deployments
- VM mobility requirements
- Cloud and SDN environments
- Need for multi-tenancy
- Active-Active redundancy requirements
✗ When NOT to Use VXLAN
- Small networks with <100 segments
- Legacy hardware that doesn't support it
- Simple flat network requirements
- No Layer 3 infrastructure available
- MTU constraints that can't be resolved
Key VXLAN Components
| Component | Description | Role |
|---|---|---|
| VNI | VXLAN Network Identifier | 24-bit ID (16 million unique values) |
| VTEP | VXLAN Tunnel Endpoint | Encapsulates/decapsulates frames |
| NVE | Network Virtualization Edge | Logical interface for VXLAN |
| Underlay | Physical IP network | Transports encapsulated packets |
| Overlay | Virtual Layer 2 network | Logical network seen by endpoints |
VXLAN Encapsulation
VXLAN Encapsulation Format
Figure 4.1: VXLAN encapsulation showing packet header structure and comparison with VLAN
Chapter 5: VLAN vs VXLAN Comparison
| Feature | VLAN | VXLAN |
|---|---|---|
| Segment ID Size | 12 bits (4,094) | 24 bits (16,777,216) |
| Transport | Layer 2 | Layer 3 (UDP) |
| Spanning Tree | Required | Not needed |
| Redundancy | Active/Standby (STP) | Active/Active (ECMP) |
| Scalability | Limited | Massive |
| Control Plane | None (data plane) | BGP EVPN (optional) |
VLAN Architecture (3-Tier)
- STP blocks 50% of links
- Oversubscription at aggregation
- Variable latency
- Complex scaling
VXLAN Architecture (Leaf-Spine)
- All links active (ECMP)
- Predictable 2-hop latency
- Non-blocking
- Linear scaling
Part III: VXLAN Fabric Architecture
Understanding Clos, Underlay, and Overlay
Chapter 6: Clos Architecture
Leaf-Spine Clos Architecture
VTEP
VTEP
VTEP
VTEP
Leaf-Spine Clos Architecture with Interface Labels
Figure 6.1: Clos leaf-spine architecture with VTEPs and interface numbering
Clos Properties
| Property | Description |
|---|---|
| Non-blocking | Full bandwidth between any endpoints |
| Predictable | Consistent 2-hop path (leaf-spine-leaf) |
| ECMP | Equal-Cost Multi-Path load balancing |
| Scalable | Add spines for bandwidth, leaves for ports |
Clos vs MAC Addresses
| Aspect | Clos Architecture | MAC Addresses |
|---|---|---|
| Purpose | Network topology design | Device identification |
| Layer | Physical + Layer 3 | Layer 2 |
| Function | Traffic path selection | Frame forwarding |
| Learning | Static (configured) | Dynamic (CAM table) |
Chapter 7-8: Underlay and Overlay Networks
| Aspect | Underlay | Overlay |
|---|---|---|
| Definition | Physical IP infrastructure | Virtual network on top |
| Layer | Layer 3 (IP) | Layer 2 (logical) |
| Protocol | OSPF, IS-IS, BGP | VXLAN |
| Addressing | VTEP IP addresses | Endpoint MAC addresses |
| Purpose | Transport VXLAN packets | Provide tenant connectivity |
Underlay vs Overlay Networks
Figure 7.1: Relationship between underlay physical network and overlay virtual network
Part IV: Control Plane - BGP EVPN
The brain of the VXLAN fabric
Chapter 9-10: BGP EVPN and Route Types
EVPN Route Types
| Type | Name | Purpose |
|---|---|---|
| Type-1 | Ethernet Auto-Discovery | Multi-homing, fast convergence |
| Type-2 | MAC/IP Advertisement | MAC and IP reachability (most used) |
| Type-3 | Inclusive Multicast | BUM traffic handling |
| Type-4 | Ethernet Segment | Designated Forwarder election |
| Type-5 | IP Prefix | External IP routing |
BGP EVPN Configuration (Cisco NX-OS)
! Enable BGP EVPN
nv overlay evpn
router bgp 65000
router-id 10.0.1.1
neighbor 10.0.0.1 remote-as 65000
update-source loopback0
address-family l2vpn evpn
send-community extended
evpn
vni 10000 l2
rd auto
route-target import auto
route-target export auto
BGP EVPN Control Plane
Figure 9.1: BGP EVPN control plane with route reflector and route type distribution
Part V: Traffic Flow in VXLAN
Understanding how packets traverse the fabric
Chapter 11-13: Intra-VXLAN, Inter-VXLAN, and BUM Traffic
Traffic Types in VXLAN
| Traffic Type | Description | VNI Used |
|---|---|---|
| Intra-VXLAN (Same VNI) | Hosts in same subnet | L2VNI (e.g., 10000) |
| Inter-VXLAN (Different VNI) | Hosts in different subnets | L3VNI (e.g., 50000) |
| BUM Traffic | Broadcast, Unknown unicast, Multicast | L2VNI with replication |
BUM Handling Methods
Multicast Underlay
Map VNIs to multicast groups. Requires PIM in underlay.
- Efficient delivery
- Requires multicast infrastructure
Ingress Replication
Source VTEP sends unicast copy to each remote VTEP.
- No multicast needed
- More CPU at source
BGP EVPN (Recommended)
EVPN Type-3 routes advertise VTEP membership.
- Scalable and dynamic
- Most common today
VXLAN Traffic Flow (Same VNI)
Figure 11.1: VXLAN traffic flow showing encapsulation, routing through spines, and decapsulation
Part VI: Multicast in VXLAN
Multicast group strategies and considerations
Chapter 14-15: Multicast Groups and VNIs
Multicast Group Assignment Strategies
| Strategy | Description | When to Use |
|---|---|---|
| One Group per VNI | VNI 10000 -> 239.1.1.1 VNI 10001 -> 239.1.1.2 |
Small deployments (<100 VNIs) |
| Shared Groups | VNI 10000-10099 -> 239.1.1.1 VNI 10100-10199 -> 239.1.1.2 |
Large deployments (100+ VNIs) |
| Ingress Replication | No multicast groups | No multicast in underlay / BGP EVPN |
Part VII: Vendor Implementations
Cisco DNA vs Juniper Campus Fabric
Chapter 16-17: Cisco DNA and Juniper Campus Fabric
Comparison Table
| Aspect | Cisco SD-Access | Juniper Campus Fabric |
|---|---|---|
| Control Plane | LISP | BGP EVPN |
| Data Plane | VXLAN-GPO | VXLAN |
| Management | DNA Center (mandatory) | Mist Cloud / Junos Space |
| Underlay | IS-IS (auto) | eBGP or OSPF |
| Segmentation | SGT (TrustSec) | Micro-segmentation |
| Hardware | Catalyst 9000 | QFX/EX Series |
Cisco DNA Center Communication
| API/Protocol | Use Case |
|---|---|
| REST APIs | Automation, integration, provisioning |
| NETCONF/YANG | Configuration management |
| SSH/CLI | Device access |
| SNMP | Monitoring, polling |
Cisco SD-Access vs Juniper Campus Fabric
Figure 16.1: Side-by-side comparison of Cisco SD-Access and Juniper Campus Fabric architectures
Part VIII: External Connectivity
Fusion routers and traffic leaving the fabric
Chapter 18-19: Fusion Router and External Traffic
VNI Extension Options
Option 1: VXLAN to Fusion
Fusion router acts as VTEP and participates in BGP EVPN.
- Full VXLAN integration
- Seamless routing
Option 2: VLAN Handoff
Border leaf terminates VXLAN and trunks VLANs to fusion.
- Legacy compatibility
- Simpler fusion router
Automation Capabilities
| Task | Can Automate? | Method |
|---|---|---|
| VNI Creation | Yes | DNA Center / Ansible |
| VNI-to-VLAN Mapping | Yes | Templates |
| BGP EVPN Peering | Partial | ZTP + Templates |
| External Routing | Manual | Security review needed |
Part IX: Hands-On Labs
Practice configurations for Cisco and Juniper
Lab Exercises
Lab 1: Basic VXLAN Configuration (Cisco)
Objective: Configure basic VXLAN with multicast BUM handling
! Enable features
feature nv overlay
feature vn-segment-vlan-based
! Create VLAN and map to VNI
vlan 100
name Production
vn-segment 10000
! NVE Interface
interface nve1
no shutdown
source-interface loopback0
member vni 10000
mcast-group 239.1.1.1
Lab 2: BGP EVPN Setup (Cisco)
Objective: Add BGP EVPN control plane
! Enable BGP EVPN
nv overlay evpn
router bgp 65000
neighbor 10.0.0.1 remote-as 65000
address-family l2vpn evpn
send-community extended
evpn
vni 10000 l2
rd auto
route-target import auto
route-target export auto
Lab 3: VXLAN with Juniper
Objective: Configure EVPN-VXLAN on Junos
# VLAN and VNI set vlans Production vlan-id 100 set vlans Production vxlan vni 10000 # VTEP source set switch-options vtep-source-interface lo0.0 set switch-options route-distinguisher 10.0.1.1:1 set switch-options vrf-target target:65000:1 # EVPN set protocols evpn encapsulation vxlan set protocols evpn extended-vni-list 10000
Lab 4: Verification Commands
! Cisco show nve peers show nve vni show bgp l2vpn evpn summary show bgp l2vpn evpn route-type 2 ! Juniper show ethernet-switching vxlan-tunnel-end-point remote show evpn database show route table bgp.evpn.0
Part X: Interview Preparation
Questions from basic to consultant level
Interview Questions & Answers
Basic Level
A VLAN (Virtual Local Area Network) is a logical grouping of network devices that allows them to communicate as if they were on the same physical network segment. VLANs create separate broadcast domains within a switch.
Key points:
- VLAN IDs: 1-4094
- 802.1Q standard for tagging
- Access ports (single VLAN) vs Trunk ports (multiple VLANs)
Through source MAC learning:
- Frame arrives on a port
- Switch records source MAC + port
- Timer starts (default 300 seconds)
- Entry refreshed on each frame
- Aged out if timer expires
For forwarding: Known MAC = specific port; Unknown = flood
VXLAN (Virtual Extensible LAN) creates Layer 2 overlay networks over Layer 3 infrastructure.
Why developed:
- VLANs limited to 4,094; VXLAN has 16 million VNIs
- STP blocks links; VXLAN uses ECMP
- VLANs can't cross L3; VXLAN can
- Cloud scale and multi-tenancy requirements
Intermediate Level
BGP EVPN distributes MAC and IP reachability information across VXLAN fabrics.
Route Types:
- Type-1: Ethernet Auto-Discovery (multi-homing)
- Type-2: MAC/IP Advertisement (most common)
- Type-3: Inclusive Multicast (BUM handling)
- Type-4: Ethernet Segment (DF election)
- Type-5: IP Prefix (external routing)
Inter-VXLAN routing using Symmetric IRB:
- Source sends to anycast gateway on local leaf
- Leaf routes in VRF, determines destination subnet
- Encapsulates with L3VNI for tenant VRF
- Routes through spines to destination leaf
- Destination leaf decapsulates L3VNI
- Routes to destination L2VNI
- Forwards to destination host
Key: L2VNI for bridging, L3VNI for routing between subnets
Advanced/Consultant Level
Key Differences:
- Control Plane: Cisco uses LISP; Juniper uses BGP EVPN
- Management: DNA Center is mandatory for Cisco; Juniper is flexible
- Segmentation: Cisco uses SGT/TrustSec; Juniper uses micro-segmentation
- Underlay: Cisco auto-provisions IS-IS; Juniper uses eBGP or OSPF
Cisco is more prescriptive; Juniper is more modular.
Systematic approach:
- Underlay: Check OSPF/BGP neighbors, ping VTEP IPs, verify MTU (9000+)
- VXLAN Config: Verify NVE interface, VNI mappings
- Control Plane: Check BGP EVPN neighbors and routes
- NVE Peers: Verify peer state and VNIs
- MAC Learning: Check local and remote MAC entries
Common issues: MTU too small, underlay not advertising loopbacks, NVE source mismatch
Key Design Decisions:
- Spine Layer: 4 spines with 48x 100G ports each
- Leaf Layer: 13+ leaves (800 endpoints per leaf)
- Underlay: eBGP (unique AS per leaf)
- Overlay: VXLAN with BGP EVPN, ingress replication
- Gateway: Anycast gateway on all leaves
- HA: vPC/MC-LAG at leaf, ECMP across spines
- External: Dual border leaves with firewall integration
Glossary
| Term | Definition |
|---|---|
| ARP | Address Resolution Protocol - maps IP to MAC |
| BGP | Border Gateway Protocol - routing protocol |
| BUM | Broadcast, Unknown unicast, Multicast traffic |
| CAM | Content Addressable Memory (MAC table) |
| Clos | Multi-stage non-blocking network topology |
| ECMP | Equal-Cost Multi-Path load balancing |
| EVPN | Ethernet VPN - BGP extension for L2/L3 |
| IRB | Integrated Routing and Bridging |
| VNI | VXLAN Network Identifier |
| VTEP | VXLAN Tunnel Endpoint |
| VRF | Virtual Routing and Forwarding |