Cumulus Linux was built on the idea that operators could buy the switch from one company and the operating system from another. Before the NVIDIA acquisition, a network team could run the same Debian-based software on switches built with Broadcom or Mellanox silicon and manage them with the Linux tools it already used on its servers.
NVIDIA has continued to develop Cumulus since the acquisition in 2020. Release 5.18 added support for a 128-port 800GbE Spectrum-6 switch. Current releases run only on NVIDIA Spectrum silicon, and the last branch for Broadcom switches reached end of life in December 2025. Operators with a supported Spectrum switch can upgrade the software. Those running Cumulus on Broadcom hardware need a new switch to run a supported release. Some teams have kept Cumulus on existing switches and chosen SONiC for new ones.
Why network teams put Linux on the switch
In 2012, Cumulus Networks created ONIE, a boot environment that lets a bare-metal switch install any compatible operating system, and the Open Compute Project adopted it a year later. Hardware makers could now ship switches without a bundled OS, and in January 2014 Dell agreed to offer Cumulus on its S6000 and S4810 top-of-rack switches.
On a Cumulus switch, front-panel ports appear as Linux interfaces named swp1, swp2 and so on, and engineers work in a Bash shell. When Network World tested Cumulus in 2015, there was no single running configuration to inspect. Settings lived in separate files, with ports, bonds and VLANs in /etc/network/interfaces and routing in its own file. Troubleshooting meant knowing which file held a setting and which service applied it.
That same layout made automation easier at Namex, the Italian internet exchange. In 2021, its engineers described moving their peering fabric to Cumulus 4.2 on Mellanox switches. They found that removing a configuration section was simply a matter of rewriting and reloading a file, with no old commands to parse. They validated the design in a hosted Cumulus simulation, brought the switches up with zero-touch provisioning and let Ansible build the whole fabric in a few minutes. They tested the traffic filters on the physical switches.
How a route reaches the ASIC
Early versions of Cumulus used Quagga, an open source routing suite. In April 2017, Cumulus and several other companies launched FRRouting under the Linux Foundation as Quagga’s successor, and Cumulus Linux 3.4 replaced Quagga with FRR. SONiC uses FRR too.
On Cumulus, BGP hands its best path to Zebra, FRR’s route manager, which compares it with routes from other sources and installs the selected route in the Linux kernel. A daemon called switchd tracks changes to the kernel routing table and programs the ASIC. The ip route command shows the kernel’s routing table. To confirm that a route was installed in hardware, an engineer also needs to check the ASIC’s forwarding table.
In SONiC’s standard Redis-based design, fpmsyncd takes forwarding updates from Zebra and writes them to the application database. Orchagent turns those updates into Switch Abstraction Interface (SAI) operations, and syncd calls the vendor’s SAI implementation to program the ASIC.
Teams moving from Cumulus to SONiC can often reuse their BGP policies. How they apply those policies depends on whether the SONiC distribution generates FRR settings from its configuration database or reads native FRR files. Port settings, breakouts and buffer profiles sit outside FRR entirely and have to be checked on the target platform.
What changed under NVIDIA
Cumulus became part of NVIDIA in June 2020, a month after the deal was announced. NVIDIA said Cumulus then had more than 2,000 customers on 130 hardware platforms, and that its Spectrum switches had shipped with both Cumulus and SONiC since 2016. An older NVIDIA datasheet reported that 34% of Fortune 50 companies used Cumulus. Today, the SN6600-LD supports both Cumulus and SONiC. The SN6800-LD supports Cumulus but not SONiC.
EVPN multihoming, the standards-based alternative to MLAG, appeared as early access in Cumulus 4.2.0 and became generally available in 4.2.1, on supported Mellanox Spectrum ASICs only. GRNET, the Greek research and education network, ran Cumulus 4.3 on Broadcom-based leaves, so it had to use MLAG and its peer links for active-active server connections. Its white paper explained that NVIDIA’s strategy “is to provide some Cumulus features only on the Mellanox chipset.”
In July 2021, NVIDIA released Cumulus 4.4 for Spectrum switches only, leaving Broadcom switches on 3.7 or 4.3, depending on the model. With 4.4, operators no longer needed a key to enable switchd, but they still needed a Cumulus license for each Spectrum switch. The release also introduced NVUE, a command-line interface and REST API that use the same configuration model. Five months later, Cumulus 5.0 made NVUE the default and retired NCLU.
Cumulus Linux milestones
Dates are in chronological order. Each milestone links to its source. Future dates reflect published support schedules.
Moving to Cumulus 5
Upgrading a Spectrum switch from Cumulus 4.x to 5.0 required a fresh install through ONIE. Operators had to back up the configuration before the install and restore it afterward.
Within 5.x, package upgrades and optimized image upgrades can use a warm reboot to keep traffic forwarding on supported platforms and upgrade paths. NVIDIA requires full BGP graceful restart and half-resource mode to be configured in advance. Enabling either setting is itself disruptive. In 5.18, in-service upgrades are not supported on Spectrum-6 or in EVPN multihoming and EVPN MLAG configurations.
NVUE generates files such as /etc/network/interfaces from its own configuration model. Applying a configuration through NVUE can overwrite edits made to those files by hand or with Ansible, Chef or Puppet. In Cumulus 5.18, NVUE saves its configuration automatically by default. With auto-save enabled, manual edits can also be lost when the switch reboots after an upgrade or when an administrator changes the cumulus account password with Linux passwd. Teams can still manage Linux and FRR files directly. If they also use NVUE, they can configure it to ignore files managed by another tool.
Starting with 5.13 in 2025, NVIDIA stopped releasing new standalone Cumulus VX images and directed users to its hosted Air platform. Hostinger, for example, had used local virtual switches to test network changes before deployment. In March 2026, Ivan Pepelnjak dropped Cumulus from his online BGP labs, citing unmaintained containers, the shift to NVUE and the end of freely downloadable Vagrant images.
Running Cumulus alongside SONiC
The metal-stack project added SONiC in July 2023 so it could keep the Broadcom switches it used to manage bare-metal infrastructure. Its team had been “very satisfied” with Cumulus, but Cumulus 5 would not run on its Broadcom switches. Because both systems use FRR, it reused its existing leaf routing configuration, but wrote a new Ansible role to provision SONiC switches. Its playbooks detect which OS a switch runs. The project has since deprecated Cumulus support.
Vinted, the online secondhand marketplace, chose SONiC for new switches. At BalticNOG in September 2025, a Vinted engineer described a BGP unnumbered Clos fabric spanning nearly 200 network devices across three locations. The team had originally chosen Cumulus for its support for multiple hardware vendors. It now builds its own community SONiC image and tests against both operating systems.
In a 2023 Hardware Nation lab, two virtual switches, one running Cumulus and the other SONiC, exchanged BGP EVPN routes and carried traffic across VXLAN. With EVPN multihoming, some peers advertise Ethernet auto-discovery routes per Ethernet segment but not per EVPN instance. Cumulus can use the per-segment route without waiting for a per-instance route.
Both peers in a Cumulus MLAG pair must be the same switch model and run the same Cumulus release in normal operation. A switch moved to SONiC can no longer serve as a peer in that pair.
Support on Spectrum today
NVIDIA adds new features and switch models to its mainline releases. Each long-term support (LTS) branch receives three years of security and critical fixes. Support for 5.11 LTS is scheduled to end a year later than support for 5.15.
Cumulus Linux release support
| Branch | Track | End of life |
|---|---|---|
| 4.3.x (from 4.3.1) | Final Broadcom branch | December 2025 |
| 5.15 | Mainline | November 2026 |
| 5.9 | LTS | April 2027 |
| 5.16 | Mainline | April 2027 |
| 5.17 | Mainline | July 2027 |
| 5.18 | Mainline | August 2027 |
| 5.11 | LTS | November 2027 |
Selected releases, ordered by end-of-life month. LTS = long-term support. Source: NVIDIA support policy (November 3, 2026).
On the Spectrum-6 SN6600-LD, Cumulus 5.18 does not support 1x800G port breakout, warm boot or Precision Time Protocol (PTP). Before moving to a later release, operators must install 5.18.2 or 5.18.3 to get the required firmware update.
Where Cumulus runs today
IDC ranked NVIDIA the largest branded vendor of data center Ethernet switches by revenue in the second quarter of 2026, with $2.5 billion, or 20.4% of the worldwide segment. That figure measures switch sales, not Cumulus installations. Meta uses its own switch software, FBOSS, on both versions of its 102.4 Tb/s IceCube blade: one uses NVIDIA Spectrum-6 silicon and the other uses Broadcom Tomahawk 6C.
Jetstream2, the NSF-funded research cloud, runs Cumulus on six SN4600 spines and 37 SN2700 leaves in its primary cluster network. Its predecessor used MLAG, and the team estimated that keeping that design would have required about 96 additional links between switches. Instead, it extended BGP to the servers.
Each host runs FRR with its own private ASN and advertises its IPv4 loopback address, while every physical 100GbE interface carries only an IPv6 link-local address. BGP unnumbered lets those sessions form without an IPv4 subnet on every link, and a GPU or storage host attached to two leaves uses ECMP across both, keeping a route through either one if the other fails.
Texas A&M’s VISION supercomputer uses 12 Cumulus switches for in-band and out-of-band Ethernet. Its 54 NVIDIA Quantum switches carry compute and storage traffic on two separate 400 Gb/s InfiniBand fabrics.
For GPU clusters that use Ethernet, NVIDIA validates specific combinations of Cumulus and ConnectX SuperNIC firmware as part of Spectrum-X. It validated Cumulus 5.18.1 for reference architecture 2.3.1 and Cumulus 5.18.2 and 5.18.3 for version 2.3.2. Adaptive routing can send packets from the same RoCEv2 flow over different uplinks as congestion changes. When packets arrive out of order, the receiving SuperNIC writes the data to the correct locations in memory.
Where Cumulus fits now
Cumulus Linux is NVIDIA’s flagship network operating system for Spectrum Ethernet switches and is available preinstalled. Teams that already run Linux and FRR can use familiar tools to manage those switches. For GPU clusters, NVIDIA validates the switch, NIC firmware and Cumulus release together.
Choosing Cumulus also means choosing Spectrum silicon. SONiC runs on switches built with Broadcom Tomahawk, Cisco Silicon One, Marvell Teralynx 10 and Xsight X2, among others. A team moving from Cumulus to one of those switches would need to qualify the new operating system and check which parts of its automation it could reuse.
One of the original reasons to choose Cumulus was the freedom to change switches and keep the operating system. For operators whose Broadcom switches still meet their needs, the trade-off is between buying Spectrum hardware to run a supported Cumulus release and qualifying another operating system for the switches they already own.
Alex Cronin
Alex Cronin is the founder of Hardware Nation and Nanites AI. He is a graduate of Georgia Tech with 20 years of experience in networking and data center infrastructure across enterprises, service providers and hyperscalers. His work spans traditional and open networking, production SONiC deployments, multivendor integration and direct collaboration with manufacturers and engineering teams. Hardware Nation has served organizations including NASA, Lockheed Martin, Accenture, Stanford University and Georgia Tech.
Alex leads engineering teams and contributes directly to architecture, code, integration and evaluation. At Nanites AI, he has led nearly two years of development of an agentic NetOps platform for network troubleshooting. He also develops AI automation for product research and leads business operations at Hardware Nation.
Nanites AI has been cited in Gartner’s Market Guide for Agentic NetOps Software, covered by InformationWeek and discussed by Cisco DevNet. It was selected as one of 12 finalists from more than 400 applicants in the 2025 T Challenge, organized by T-Mobile and Deutsche Telekom. Alex has presented his work at Deutsche Telekom in Bonn, CableLabs and Network X Americas.
