Certainty in Scale with Spectrum

Certainty, Scale, Spectrum…? Sweet alliteration Chris, but what are you trying to say?

I’m glad you asked!

Let’s sift backwards through the significantly soft sounds and see what sentience I can supply:

Spectrum?

No, not a band of colors – the Spectrum I’m talking about here is Silicon. Specifically, it’s the ASIC from Mellanox Technologies that powers their Ethernet switches. Mellanox is most famous for their InfiniBand products. Don’t let that stop you from considering their robust Ethernet offerings though.

They describe themselves as “a leading supplier of end-to-end Ethernet and InfiniBand intelligent interconnect solutions and services for servers, storage, and hyper-converged infrastructure.” More succinctly, as they recently informed us at NFD17, Mellanox focuses on connecting servers and storage. They do this with purpose-built NICs, cables, and switches.

But why am I writing a post about an ASIC?

In the past, many network engineers paid little attention to which ASIC powered their devices. That’s because they didn’t really have a choice. Traditional, vertically integrated network gear bundles the hardware and software together. Even datasheets for these devices mix hardware and software features. So we compare monolithic switches and routers as a package. There is no real need to dig into the capabilities of specific integrated circuits when you aren’t allowed to swap them out.

Disaggregation gives us the ability to make these choices. Choices that were previously off-limits. This newfound freedom comes with the responsibility to understand the individual components with more clarity.

Scale?

The future of networking certainly seems tied up in the trends of disaggregation and virtualization. Separating hardware and software in this way allows us to optimize each component for the specific job we ask it to do. We need Ethernet switches to process packets with enough scale to support all the layers above. Any hardware limitation cascades upwards, creating constraints for the NOS, the underlay network, the overlay networks, and any services supported by those networks. Even a full-blown SDN needs robust hardware to run on. This makes ASIC scale a fundamental need for building any modern network.

It seems that Mellanox gets this. In the presentation included below, focused on VxLAN EVPN, they highlight a couple of numbers that impressed me:

• 750+ VTEPs (VxLAN Tunnel Endpoints)
• 100k VxLAN Tunnels

That’s real scale. It gives me confidence that these switches won’t get in the way as I grow and change my network.

Certainty?

Beyond just scale, what we really need from our switches is predictable performance: Certainty.

This is where the Mellanox Spectrum shines. Take a look at these graphs from a 2016 Tolly report. They compare it with the Broadcom Tomahawk ASIC on several key metrics:

Wow. Even if we ignore the comparison, it seems obvious that Spectrum switches can provide the certainty we demand.

The full report, linked above, is worth reading before you decide on your next switch purchase.

More:

Learn more about Mellanox Spectrum and why you might want to use it for the underlay of your next VxLAN EVPN network. Check out the full NFD17 presentation: