Intel Xeon CPU Max Flips the Script on Memory Bandwidth
Published: 07-16-2024
Whether you’re using a high-end workstation or an office PC clunker, your memory is connected to the CPU using a memory bus that’s basically using the same design. No matter how many channels you have, or how fast the memory gets, there are limits to how much bandwidth is possible with this architecture.
Which is why Intel has gone back to the drawing board with their Xeon CPU Max series of processors.
What Makes CPU Max Different?
The big story here is that Intel has done something never before attempted on an x86 CPU. They’ve included 64GB of HBM (High-Bandwidth Memory) right on the CPU package. This allows for far more memory bandwidth than regular DDR can manage.
This makes these chips more like dedicated GPU or AI accelerators. You might remember HBM memory from AMD’s brief experiment with it on some of their GPUs a few years ago. Unfortunately AMD had problems scaling their HBM implementation past 4GB, but clearly Intel’s HBM2e technology has surpassed that in leaps and bounds with 64GB of the stuff crammed into a CPU die.
What’s So Special About Having Lots of Bandwidth?
There are some types of high-performance computing tasks that benefit from having lots of memory close to the CPU.
By Intel’s reckoning, AI tasks on Xeon Max are almost four times compared to the previous generation Xeon chips. These CPUs are pretty much the same thing as the Xeon Platinum+ CPUs of the same generation, so the differentiating factor really is the HBM.
It doesn’t have to be AI-based work of course. Any workload that’s currently limited by bandwidth rather than CPU performance will see a dramatic increase using HBM. Under the assumption that you can fit all the relevant data in 64GB!
Who Should Get Xeon Max?
You may have noticed that we sell many workstations with far more RAM than 64GB! Many high-end professional workstations have memory allocations in the hundreds or even thousands of GB.
So obviously if your data does not fit within the 64GB of HBM these CPUs currently have, you won’t get all or any of the benefit of the extra bandwidth. This is one of the reasons that AI is such a good current candidate, because the model has to be entirely resident in RAM to work efficiently.
Talk To Us
While Xeon CPU Max isn’t for everyone, if it sounds like this might be the right breakthrough in CPU architecture for your needs, talk to us about the best approach to build a workstation based on Intel’s fascinating high-bandwidth memory technology. Together with the expert system builders at Titan Computers, we can figure out if Xeon Max should be at the heart of your next project.
