Core-to-Core Latency

As the core count of modern CPUs is growing, we are reaching a time when the time to access each core from a different core is no longer a constant. Even before the advent of heterogeneous SoC designs, processors built on large rings or meshes can have different latencies to access the nearest core compared to the furthest core. This rings true especially in multi-socket server environments.

But modern CPUs, even desktop and consumer CPUs, can have variable access latency to get to another core. For example, in the first generation Threadripper CPUs, we had four chips on the package, each with 8 threads, and each with a different core-to-core latency depending on if it was on-die or off-die. This gets more complex with products like Lakefield, which has two different communication buses depending on which core is talking to which.

If you are a regular reader of AnandTech’s CPU reviews, you will recognize our Core-to-Core latency test. It’s a great way to show exactly how groups of cores are laid out on the silicon. This is a custom in-house test, and we know there are competing tests out there, but we feel ours is the most accurate to how quick an access between two cores can happen.


(Click on the image to enlarge)

Analyzing core-to-core latencies on the AMD Ryzen Threadripper 7980X (64C/128T), our test is limited to probing the first 64 threads, although scaling out to 128 threads would be identical. Each CCD on the Threadripper 7980X has 8 x Zen 4 cores, with 32 MB of L3 cache. Looking at the latency range within the CCD, we can see inner latencies between 7 and 20 ns, which increase to 89 and 96 ns as each core communicates within the CCX.


A visual render of the AMD Ryzen Threadripper 7980X with 8 x CCDs and IOD

Given we've reviewed the AMD Ryzen 9 7950X, which has the same Zen 4 cores and the same CCD complex approach to communicating between the cores, we see relatively similar latencies in both Threadripper 7000 and Ryzen 7000. A quad-channel DDR5 memory controller integrated within the large IOD and using PCIe 5.0 lanes as the primary pathway is important in enhancing the Infinity Fabric interconnect to reduce latencies and help counteract any penalties.

SPEC2017 Multi-Threaded Results Threadripper 7000 vs. Threadripper 3000: Generational Improvements
Comments Locked

66 Comments

View All Comments

  • tamalero - Tuesday, December 5, 2023 - link

    One of the most common issues of AMD is sketchy performance and stability with non samsung chips.
    If you ram had hynix or similar chips..it would usually not post at the advertised max speed.

    As soon I moved to Samsung chips in all my AMD builds, all problems went away.
  • JRF68 - Tuesday, December 12, 2023 - link

    Just unlucky. Should RMA the mb. Asus imho, is still the best for HEDT whether AMD or Intel.
  • clsmithj - Sunday, December 31, 2023 - link

    I have a 3960X Threadripper on a MSI TRX40 Creator, and a 2990WX Threadripper on a MSI X399 MEG Creation board.
    Both system use 4 sticks of Kingston 64GB (16GB x 4) DDR4 3200 dual rank memory (HyperX Predaor/Fury) that worked fine at their C16 3200 MT XMP rate.
    I used DRAM Calc app and managed to get the RAM of my 3960X to run stable at gear 1 C16 3800 MT with UCLK=MCLK set 1900Mhz.
    The ZEN2 CPU is definitely capable of this high memory OC. I've watch Buildzoid clips of him showing how with ZEN2 TR it can go even higher than 3800.
    My ZEN+ 2990WX I have running at Gear 1 C16 DDR4-3466 speed, and I checked ZenTimings that revealed its UCLK=MCLK is at 1733Mhz.
    The high memory clocking is doable you do need a good motherboard, but I always observed the TRX40 boards having a fast amount of decent boards with high amount of VRM phases.
    It was the X399 boards that only had a total of two good boards, which was either the MSI MEG Creation or the ASUS ROG Zenith Extreme.
  • thestryker - Monday, November 20, 2023 - link

    Where have you seen JEDEC compliant 6400 RDIMMs? The highest I've seen is 5600.
  • Rοb - Wednesday, November 29, 2023 - link

    Here: https://www.anandtech.com/show/18988/teamgroup-unv... and https://www.anandtech.com/show/21129/micron-introd...
  • demu - Monday, November 20, 2023 - link

    I was able to run my 3960X with G.Skill Trident Neo 4x16GB 3600CL16 (B-die) at 3600CL14 and/or 3766 CL16 (Asus ROG Strix TRX40-E Gaming) for weeks without any problems.

    Broke down the processor about two months ago. Touched accidentally USB-A -port with an USB-C -cable and the processor no longer booted (or booted and went trough all but the last POST test and then stuck - tried two different MB) :(
    Put all components into a new 5800X3D system and got 4x16GB memory to work @3600CL16.

    Now considering the new 7960X.
  • Makaveli - Monday, November 20, 2023 - link

    So looks like the difference between your system Demu and Teamswitcher is you paired it with high-quality memory.

    Which is no surprise and it's why I never go cheap on memory.
  • meacupla - Monday, November 20, 2023 - link

    It's not about cheap or expensive, it's if you got Samsung B-die or not.

    I have G.Skill F4-3600C18-16GVK, and it only works at 3200 CL18. Why? because I was told it would be Samsung B-die, but it turned out to be SK Hynix, and the AM4 platform doesn't like working with anything other than Samsung B-die.
  • Makaveli - Monday, November 20, 2023 - link

    I have B-die and that is what I'm talking about when I said don't buy cheap memory. B-die = expensive but you get what you pay for and it just works. And you should have done your own research I would have caught the memory was Hynix before purchase.
  • meacupla - Monday, November 20, 2023 - link

    That's the thing though, I did do my research.

Log in

Don't have an account? Sign up now