Explore the differences between Asymmetric and Symmetric Integrated Routing and Bridging (IRB) functionality, as tested using SONiC. Discover how these approaches impact data center efficiency and network performance, with insights into their setup, testing methods, and comparative advantages.
Integrated Routing and Bridging
In the traditional setup of data centers, a central router manages the transfer of data between different subnets. This means data has to travel across the network to this central point and then back out to its destination. However, in a large multi-tenant data center environment this method can lead to inefficient use of bandwidth and sub-optimal forwarding.
To address these issues, a proposed solution called Integrated Routing and Bridging in EVPN was introduced. IRB plays a critical role in connecting and facilitating communication between different network segments within a single device, typically a router or a layer 3 switch. IRB seamlessly integrates both routing and bridging functionalities, providing a versatile solution for managing traffic between VLANs and improving the overall efficiency of network communications.
By integrating routing and bridging functionality directly onto the VTEP, the routing operation can occur as close to the end host as possible. By doing this, data can be routed much closer to where it starts and ends, which means it can get to its destination more efficiently.
There are two forwarding models within EVPN for the Integrated Routing and Bridging (IRB) functionality to achieve inter-subnet routing:
- Asymmetric IRB
- Symmetric IRB
Asymmetric IRB
In the asymmetric IRB model, routing and bridging occur at the VXLAN tunnel ingress, with the packet after the routing action being VXLAN bridged to the destination VTEP. The egress VTEP removes the VXLAN header and forwards the packet onto the L2 domain based on the VNI to VLAN mapping. The ingress VTEP is configured with all destination virtual networks, and has the ARP entries and MAC addresses for all destination hosts in its hardware tables.
Testing in SONiC
For better understanding, Asymmetric Integrated Routing and Bridging (IRB) is tested on SONiC (202311) in GNS3.
Considering the above figure, Host A wants to communicate with Host B:
1. Since Host B is on a different subnet, Host A sends the frame in VLAN 10 to its gateway which is VTEP 1.
2. VTEP 1 recognizes that the destination address is on another subnet (VLAN 20). It Looks up the routing table and routes it to the Orange VNI (VLAN 20).
3. VTEP 1 tunnels the traffic in the Orange VNI (VNI 2000) to VTEP 2.
4. VTEP 2 removes the VXLAN header from the frame, looks up the MAC table and bridges the frame to Host B.
5. Host B wants to send a reply to Host A. Since they are in different subnets, Host B sends the frame in VLAN 20 to its gateway which is VTEP 2.
6. VTEP 2 looks up the routing table and routes the traffic to the Green VNI (VLAN 10).
7. VTEP 2 tunnels the traffic in the Green VNI (VNI 1000) to VTEP 1.
8. VTEP 1 removes the VXLAN header from the frame, looks up the MAC table, and bridges the traffic to Host A.
The figure below shows the result of switch SONiC-1 and indicates that the MAC address of the destination host is learned
For better understanding, packets are captured with Wireshark. The figure below shows that when Host A sends the traffic to Host B then VTEP1 tunnels the traffic in the VNI 2000 to VTEP2.
The figure below shows that when Host B sends the reply to Host A then VTEP2 tunnels the traffic in the VNI 1000 to VTEP1.
Issue
All the VLANs and VNIs must be configured on all VTEPs even if the VTEPs do not have clients on all VLANs. This will increase the ARP cache, and CAM table size which will result in the control plane scaling issue.
Symmetric IRB
In symmetric IRB routing, both ingress and egress VTEPs perform IRB routing and bridging. The ingress VTEP routes packets to an egress VTEP MAC address in an intermediate virtual-network VNI. In this approach, all routed VXLAN traffic is directed through a dedicated transit VNI known as the L3VNI. This allows bidirectional traffic to traverse on the same VNI in both directions.
Using the L3 VNI associated with each tenant VRF, an ingress VTEP routes all traffic for the prefix to an egress VTEP on the L3 VNI. The egress VTEP routes from the L3 VNI to the destination virtual network or bridge domain. The L3 VNI does not have to be associated with an IP address; routing is set up in the data plane using the egress VTEP’s MAC address. This behavior is known as IP-VRF to IP-VRF interface-less routing. The ingress VTEP does not have to be configured with every destination virtual network; it must have the ARP and MAC addresses only to the egress VTEP, not to each host connected to the VTEP. For this reason, symmetric IRB routing allows the overlay network to scale larger than Asymmetric Routing.
Considering the above figure Host A wants to communicate with Host B:
- Because the destination is in a different subnet from Host A, Host A sends the frame to its gateway, VTEP 1 in VLAN 10.
- VTEP 1 looks up the routing table and routes the traffic to the L3VNI and destination VTEP 2.
- VTEP 1 encapsulates traffic with the VXLAN header using the L3VNI 3000 and VTEP 2’s IP and MAC address.
- VTEP 2 decapsulates the traffic and routes traffic to the destination VLAN.
- VTEP 2 bridges traffic towards Host B.
- The return traffic is routed similarly using the same L3VNI.
The figure below shows the result of switch SONiC-2 and indicates that the MAC address of the destination VTEP is learned.
For better understanding, packets are captured with Wireshark. The figure below shows that when Host A sends the traffic to Host B then VTEP1 tunnels the traffic in the L3VNI 3000 to VTEP2.
The figure below shows that when Host B sends the reply to Host A then VTEP2 tunnels the traffic in the same L3VNI 3000 to VTEP1.
Symmetric and Asymmetric IRB Feature Comparison
References
About Hardware Nation:
Hardware Nation is a professional services company that accelerates network transformation through a disaggregated, open approach, enabling freedom of choice, flexibility, and cost efficiency. Our seasoned experts have worked on projects for some of the world’s leading organizations, leveraging a hybrid cloud-first and AI-enabled approach. We help our customers navigate the ecosystem, drawing on decades of experience. Our deployments are powered by leading white box and OEM network, compute, and storage vendors. Our expertise encompasses a wide range of industries and use cases, including enterprise, cloud, data center, AI, 5G/ISP infrastructure, and edge IT.
Humza Atlaf
Network Engineer
Humza is a network engineer at Hardware Nation Labs, where his enthusiasm for Open Networking drives his work. With a blend of deep expertise and innovative approaches, he designs robust, scalable networks of the future. His practical experience includes configuring and deploying a range of protocols such as LACP, VLANs, MPLS, and VRRP. At his previous role, he was part of a SONiC testing team, further honing his skills in network setup and troubleshooting. Humza is also adept at network analysis with tools like Wireshark, enhancing his ability to manage complex network environments.
