This guide provides step-by-step instructions to configure Spine Leaf Topology using Asymmetric IRB EVPN in SONiC, covering VLAN setup, IP address configuration, VxLAN creation, and BGP establishment for EVPN.
Spine-Leaf Architecture
The spine-leaf, or leaf-spine, architecture is a two-layer network topology consisting of spine and leaf switches. This design enhances data center networks by minimizing latency and hop count while boosting overall network efficiency.
Unlike traditional three-tier architectures, the spine-leaf topology employs a full-mesh network structure, making it particularly well-suited for contemporary data centers that predominantly handle east-west traffic. East-west traffic refers to the flow of data packets between servers within the data center, as opposed to north-south traffic, which typically involves communication between servers and external networks.
By IETF, In a Spine-Leaf topology, normally a leaf node connects to a number of spine nodes. Data traffic going from one leaf node to another leaf node needs to pass through one of the spine nodes. Also, the decision to choose one of the spine nodes is usually part of equal cost multi-path (ECMP) load sharing. The spine nodes can be considered as gateway devices to reach destinations on other leaf nodes. In this type of topology, the spine nodes have to know the topology and routing information of the entire network, but the leaf nodes only need to know how to reach the gateway devices to which are the spine nodes they are uplinked to.
Three-tier architectures typically rely on the Spanning Tree Protocol (STP).
In contrast, the spine-leaf architecture features only two layers—spine and leaf switches—forming a simplified two-layer topology. This design facilitates direct connections through the spine layer, ensuring single-hop connectivity between leaf switches and thereby reducing latency and bottlenecks.
The spine-leaf architecture is optimized for east-west traffic and does not utilize STP. While STP provides redundant paths between switches, it allows only one path to be active at a time, potentially leading to oversubscription. By eliminating the use of STP, the spine-leaf architecture further reduces latency and avoids bottlenecks.
Asymmetric IRB EVPN
In the asymmetric IRB model, routing and bridging occur at the VXLAN tunnel ingress, with the packet after the routing action being VXLAN bridged to the destination VTEP.
The egress VTEP removes the VXLAN header and forwards the packet onto the L2 domain based on the VNI to VLAN mapping.
The ingress VTEP is configured with all destination virtual networks, and has the ARP entries and MAC addresses for all destination hosts in its hardware tables.
Testbed & SONiC Version
- GNS3
- SONiC ( 202305 )
Topology
Steps to Configure Asymmetric IRB EVPN
- Setup VLAN Environment
- Configure IP addresses
- Create VxLAN
- Establish BGP environment for EVPN
Note: The above steps are only for Leaf switches. For Spine switches there is no need to create VLANs and VxLAN.
Configurations on LEAF-1:
Step 1. Setup VLAN environment as per topology.
admin@sonic:~$ sudo config vlan add 10
admin@sonic:~$ sudo config vlan add 20
admin@sonic:~$ sudo config vlan member add -u 10 Ethernet8
admin@sonic:~$ sudo config vlan member add -u 20 Ethernet12
Step 2. Configure IP addresses.
admin@sonic:~$ sudo config interface ip add Ethernet0 192.168.11.1/24
admin@sonic:~$ sudo config interface ip add Ethernet4 192.168.12.1/24
admin@sonic:~$ sudo config interface ip add Loopback10 1.1.1.1/32
admin@sonic:~$ sudo config interface ip add Vlan10 192.168.10.254/24
admin@sonic:~$ sudo config interface ip add Vlan20 192.168.17.254/24
Below is the status of the VLAN table.
admin@sonic:~$ show vlan brief
+———–+——————-+————+—————-+————-+———————–+
| VLAN ID | IP Address | Ports | Port Tagging | Proxy ARP | DHCP Helper Address |
+===========+===================+============+================+=============+=======================+
| 10 | 192.168.10.254/24 | Ethernet8 | untagged | disabled | |
+———–+——————-+————+—————-+————-+———————–+
| 20 | 192.168.17.254/24 | Ethernet12 | untagged | disabled | |
+———–+——————-+————+—————-+————-+———————–+
Step 3. Create VxLAN.
admin@sonic:~$ sudo config vxlan add vtep 1.1.1.1
admin@sonic:~$ sudo config vxlan evpn_nvo add nvo vtep
admin@sonic:~$ sudo config vxlan map add vtep 10 10
admin@sonic:~$ sudo config vxlan map add vtep 20 20
Step 4. Establish a BGP environment for EVPN.
sonic# configure
sonic(config)# no router bgp 65100
sonic(config)# router bgp 65000
sonic(config-router)# bgp router-id 1.1.1.1
sonic(config-router)# bgp bestpath as-path multipath-relax
sonic(config-router)# neighbor 192.168.11.2 remote-as 65000
sonic(config-router)# neighbor 192.168.12.2 remote-as 65000
sonic(config-router)# neighbor 192.168.13.1 remote-as 65000
sonic(config-router)# neighbor 192.168.15.1 remote-as 65000
sonic(config-router)# neighbor 192.168.14.1 remote-as 65000
sonic(config-router)# neighbor 192.168.16.1 remote-as 65000
sonic(config-router)# neighbor 192.168.13.1 update-source 192.168.11.1
sonic(config-router)# neighbor 192.168.15.1 update-source 192.168.12.1
sonic(config-router)# neighbor 192.168.14.1 update-source 192.168.11.1
sonic(config-router)# neighbor 192.168.16.1 update-source 192.168.12.1
sonic(config-router)# address-family ipv4
sonic(config-router-af)# network 1.1.1.1/32
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn
sonic(config-router-af)# neighbor 192.168.13.1 activate
sonic(config-router-af)# neighbor 192.168.15.1 activate
sonic(config-router-af)# neighbor 192.168.14.1 activate
sonic(config-router-af)# neighbor 192.168.16.1 activate
sonic(config-router-af)# advertise-all-vni
sonic(config-router-af)# end
sonic# wr
Configurations on LEAF-2:
Step 1. Setup VLAN environment as per topology.
admin@sonic:~$ sudo config vlan add 10
admin@sonic:~$ sudo config vlan add 20
admin@sonic:~$ sudo config vlan member add -u 10 Ethernet8
admin@sonic:~$ sudo config vlan member add -u 20 Ethernet12
Step 2. Configure IP addresses.
admin@sonic:~$ sudo config interface ip add Ethernet0 192.168.13.1/24
admin@sonic:~$ sudo config interface ip add Ethernet4 192.168.15.1/24
admin@sonic:~$ sudo config interface ip add Loopback10 2.2.2.2/32
admin@sonic:~$ sudo config interface ip add Vlan10 192.168.10.253/24
admin@sonic:~$ sudo config interface ip add Vlan20 192.168.17.253/24
Step 3. Create VxLAN.
admin@sonic:~$ sudo config vxlan add vtep 2.2.2.2
admin@sonic:~$ sudo config vxlan evpn_nvo add nvo vtep
admin@sonic:~$ sudo config vxlan map add vtep 10 10
admin@sonic:~$ sudo config vxlan map add vtep 20 20
Step 4. Establish a BGP environment for EVPN.
sonic# configure
sonic(config)# no router bgp 65100
sonic(config)# router bgp 65000
sonic(config-router)# bgp router-id 2.2.2.2
sonic(config-router)# bgp bestpath as-path multipath-relax
sonic(config-router)# neighbor 192.168.13.2 remote-as 65000
sonic(config-router)# neighbor 192.168.15.2 remote-as 65000
sonic(config-router)# neighbor 192.168.11.1 remote-as 65000
sonic(config-router)# neighbor 192.168.12.1 remote-as 65000
sonic(config-router)# neighbor 192.168.14.1 remote-as 65000
sonic(config-router)# neighbor 192.168.16.1 remote-as 65000
sonic(config-router)# neighbor 192.168.11.1 update-source 192.168.13.1
sonic(config-router)# neighbor 192.168.12.1 update-source 192.168.15.1
sonic(config-router)# neighbor 192.168.14.1 update-source 192.168.13.1
sonic(config-router)# neighbor 192.168.16.1 update-source 192.168.15.1
sonic(config-router)# address-family ipv4
sonic(config-router-af)# network 2.2.2.2/32
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn
sonic(config-router-af)# neighbor 192.168.11.1 activate
sonic(config-router-af)# neighbor 192.168.12.1 activate
sonic(config-router-af)# neighbor 192.168.14.1 activate
sonic(config-router-af)# neighbor 192.168.16.1 activate
sonic(config-router-af)# advertise-all-vni
sonic(config-router-af)# end
sonic# wr
Configurations on LEAF-3:
Step 1. Setup VLAN environment as per topology.
admin@sonic:~$ sudo config vlan add 10
admin@sonic:~$ sudo config vlan add 20
admin@sonic:~$ sudo config vlan member add -u 10 Ethernet8
admin@sonic:~$ sudo config vlan member add -u 20 Ethernet12
Step 2. Configure IP addresses.
admin@sonic:~$ sudo config interface ip add Ethernet0 192.168.14.1/24
admin@sonic:~$ sudo config interface ip add Ethernet4 192.168.16.1/24
admin@sonic:~$ sudo config interface ip add Loopback10 3.3.3.3/32
admin@sonic:~$ sudo config interface ip add Vlan10 192.168.10.252/24
admin@sonic:~$ sudo config interface ip add Vlan20 192.168.17.252/24
Step 3. Create VxLAN.
admin@sonic:~$ sudo config vxlan add vtep 3.3.3.3
admin@sonic:~$ sudo config vxlan evpn_nvo add nvo vtep
admin@sonic:~$ sudo config vxlan map add vtep 10 10
admin@sonic:~$ sudo config vxlan map add vtep 20 20
Step 4. Establish a BGP environment for EVPN.
sonic# configure
sonic(config)# no router bgp 65100
sonic(config)# router bgp 65000
sonic(config-router)# bgp router-id 3.3.3.3
sonic(config-router)# bgp bestpath as-path multipath-relax
sonic(config-router)# neighbor 192.168.14.2 remote-as 65000
sonic(config-router)# neighbor 192.168.16.2 remote-as 65000
sonic(config-router)# neighbor 192.168.11.1 remote-as 65000
sonic(config-router)# neighbor 192.168.12.1 remote-as 65000
sonic(config-router)# neighbor 192.168.13.1 remote-as 65000
sonic(config-router)# neighbor 192.168.15.1 remote-as 65000
sonic(config-router)# neighbor 192.168.11.1 update-source 192.168.14.1
sonic(config-router)# neighbor 192.168.12.1 update-source 192.168.16.1
sonic(config-router)# neighbor 192.168.13.1 update-source 192.168.14.1
sonic(config-router)# neighbor 192.168.15.1 update-source 192.168.16.1
sonic(config-router)# address-family ipv4
sonic(config-router-af)# network 3.3.3.3/32
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn
sonic(config-router-af)# neighbor 192.168.11.1 activate
sonic(config-router-af)# neighbor 192.168.12.1 activate
sonic(config-router-af)# neighbor 192.168.13.1 activate
sonic(config-router-af)# neighbor 192.168.15.1 activate
sonic(config-router-af)# advertise-all-vni
sonic(config-router-af)# end
sonic# wr
Configurations on SPINE-1:
The primary role of spine switches is to serve as route reflectors (server) for the leaf switches, which act as their clients in this network architecture.
Step 1. Configure IP addresses.
admin@sonic:~$ sudo config interface ip add Ethernet0 192.168.11.2/24
admin@sonic:~$ sudo config interface ip add Ethernet4 192.168.13.2/24
admin@sonic:~$ sudo config interface ip add Ethernet8 192.168.14.2/24
Step 2. Establish a BGP environment for EVPN.
sonic# configure
sonic(config)# no router bgp 65100
sonic(config)# router bgp 65000
sonic(config-router)# bgp bestpath as-path multipath-relax
sonic(config-router)# neighbor 192.168.11.1 remote-as 65000
sonic(config-router)# neighbor 192.168.13.1 remote-as 65000
sonic(config-router)# neighbor 192.168.14.1 remote-as 65000
sonic(config-router)# neighbor 192.168.11.1 route-reflector-client
sonic(config-router)# neighbor 192.168.13.1 route-reflector-client
sonic(config-router)# neighbor 192.168.14.1 route-reflector-client
sonic(config-router)# address-family ipv4
sonic(config-router-af)# network 192.168.11.0/24
sonic(config-router-af)# network 192.168.13.0/24
sonic(config-router-af)# network 192.168.14.0/24
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn
sonic(config-router-af)# neighbor 192.168.11.1 route-reflector-client
sonic(config-router-af)# neighbor 192.168.13.1 route-reflector-client
sonic(config-router-af)# neighbor 192.168.14.1 route-reflector-client
sonic(config-router-af)# end
sonic# wr
Configurations on SPINE-2:
Step 1. Configure IP addresses.
admin@sonic:~$ sudo config interface ip add Ethernet0 192.168.12.2/24
admin@sonic:~$ sudo config interface ip add Ethernet4 192.168.15.2/24
admin@sonic:~$ sudo config interface ip add Ethernet8 192.168.16.2/24
Step 2. Establish a BGP environment for EVPN.
sonic# configure
sonic(config)# no router bgp 65100
sonic(config)# router bgp 65000
sonic(config-router)# bgp bestpath as-path multipath-relax
sonic(config-router)# neighbor 192.168.12.1 remote-as 65000
sonic(config-router)# neighbor 192.168.15.1 remote-as 65000
sonic(config-router)# neighbor 192.168.16.1 remote-as 65000
sonic(config-router)# neighbor 192.168.12.1 route-reflector-client
sonic(config-router)# neighbor 192.168.15.1 route-reflector-client
sonic(config-router)# neighbor 192.168.16.1 route-reflector-client
sonic(config-router)# address-family ipv4
sonic(config-router-af)# network 192.168.12.0/24
sonic(config-router-af)# network 192.168.15.0/24
sonic(config-router-af)# network 192.168.16.0/24
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn
sonic(config-router-af)# neighbor 192.168.12.1 route-reflector-client
sonic(config-router-af)# neighbor 192.168.15.1 route-reflector-client
sonic(config-router-af)# neighbor 192.168.16.1 route-reflector-client
sonic(config-router-af)# end
sonic# wr
Results
Below is the result of LEAF-1 switch which shows that remotevtep like “2.2.2.2” and “3.3.3.3” has been learned through EVPN and the source IP is “1.1.1.1”.
The figure below shows the MAC address of host PC6. The switch LEAF-1 will learn this MAC address as it is Asymmetric IRB EVPN.
Fours Hosts are attached on LEAF-2 and LEAF-3 switches. In Asymmetric IRB EVPN, the MAC address of the destination host is learned. The figure below shows that destination MAC addresses of four Hosts have been learned.
The figure below shows the BGP summary of LEAF-1.
The result below shows that host PC1 successfully received responses from PC2 and PC3.
The result below shows that host PC6 successfully received responses from PC1 and PC3.
References
About Hardware Nation:
Hardware Nation is a professional services company that accelerates network transformation through an open networking approach, enabling freedom of choice, flexibility, and cost efficiency. Our seasoned experts have worked on projects for some of the world’s leading organizations, leveraging a hybrid cloud-first and AI-enabled approach. We help our customers navigate the ecosystem, drawing on decades of experience. Our deployments are powered by leading white box and OEM network, compute, and storage vendors. Our expertise encompasses a wide range of industries and use cases, including enterprise, cloud, data center, AI, 5G/ISP infrastructure, and edge IT.
Humza Atlaf
Network Engineer
Humza is a network engineer at Hardware Nation Labs, where his passion for Open Networking drives his work. With a blend of deep expertise and innovative approaches, he designs robust, scalable networks of the future. His practical experience includes configuring and deploying a range of protocols such as LACP, VLANs, MPLS, and VRRP. At his previous role, he was part of a SONiC testing team, further honing his skills in network setup and troubleshooting. Humza is also adept at network analysis with tools like Wireshark, enhancing his ability to manage complex network environments.
Alex Cronin
Co-Founder and Solutions Architect
Alex Cronin is a seasoned Solutions Architect with over 15 years of experience in networking and disaggregated infrastructure. His career is defined by aligning enterprise technology with business needs across diverse market segments, from emerging startups to Fortune 500 companies. He has worked on digital infrastructure projects covering network design and software solutions for data center operators, service providers, and enterprises. He is continuously collaborating with Hardware Nation Labs R&D to explore and pioneer the latest advancements in open networking and is assessing the applicability of AI/ML technology across enterprise, data center, and service provider infrastructures.
