Bulldozer for Servers: Testing AMD's "Interlagos" Opteron 6200 Seriesby Johan De Gelas on November 15, 2011 5:09 PM EST
Introducing AMD's Opteron 6200 Series
When virtualization started to get popular (ca. 2005-2007), there was a fear that this might slow the server market down. Now several years later, the server market has rarely disappointed and continues to grow. For example, IDC reported a 12% increase in revenue when comparing Q1 2010 and Q1 2011. The server market in total accounted for $12 billion revenue and almost two million shipments in Q1 2011, and while the best desktop CPUs generally sell for $300, server chips typically start at $500 and can reach prices of over $3000. With the high-end desktop market shrinking to become a niche for hardcore enthusiasts--helped by the fact that moderate systems from several years back continue to run most tasks well--the enterprise market is very attractive.
Unfortunately for AMD, their share of the lucrative server market has fallen to a very low percentage (4.9%) according IDC's report early this year (some report 6-7%). It is time for something new and better from AMD, and it seems that the Bulldozer architecture is AMD's most server-centric CPU architecture ever. We quote Chuck Moore, Chief Architect AMD:
By having the shared architecture, reducing the size and sharing things that aren’t commonly used in their peak capacity in server workloads, “Bulldozer” is actually very well aligned with server workloads now and on into the future. In fact, a great deal of the trade-offs in Bulldozer were made on behalf of servers, and not just one type of workload, but a diversity of workloads.
This alginment with server workloads can also be found in the specs:
|L1 Instructions||8x 64 KB 2-way||12x 64 KB 2-way||6x 32 KB 4-way|
|L1 Data||16x 16 KB 4-way||12x 64 KB 2-way||6x 32 KB 4-way|
|L2 Cache||4x 2MB||12x 0.5MB||6x 256 KB|
|L3 Cache||2x 8MB||2x 6MB||12MB|
|IMC Clock Speed||2GHz||1.8GHz||2GHz|
|Interconnect||4x HT 3.1 (6.4 GT/s)||4x HT 3.1 (6.4 GT/s)||2x QPI (4.8-6.4 GT/s)|
The new Opteron has loads of cache, faster access to memory and more threads than ever. Of course, a good product is more than a well designed microarchitecture with impressive specs on paper. The actual SKUs have to be attractively priced, reach decent clock speeds, and above all offer a good performance/watt ratio. Let us take a look at AMD's newest Opterons and how they are positioned versus Intel's competing Xeons.
|AMD vs. Intel 2-socket SKU Comparison|
|High Performance||High Performance|
|High clock / budget||High clock / budget|
|Power Optimized||Power Optimized|
The specifications (16 threads, 32MB of cache) and AMD's promises that Interlagos would outperform Magny-cours by a large margin created the impression that the Interlagos Opteron would give the current top Xeons a hard time. However, the newest Opteron cannot reach higher clock speeds than the current Opteron (6276 at 2.3GHz), and AMD positions the Opteron 6276 2.3GHz as an alternative to the Xeon E5649 at 2.53GHz. As the latter has a lower TDP, it is clear that the newest Opteron has to outperform this Xeon by a decent margin. In fact most server buyers expect a price/performance bonus from AMD, so the Opteron 6276 needs to perform roughly at the level of the X5650 to gain the interest of IT customers.
Judging from the current positioning, the high-end is a lost cause for now. First, AMD needs a 140W TDP chip to compete with the slower parts of Intel's high-end armada. Second, Sandy Bridge EP is coming out in the next quarter--we've already seen the desktop Sandy Bridge-E launch, and adding two more cores (four more threads) for the server version will only increase the performance potential. The Sandy Bridge cores have proven to be faster than Westmere cores, and the new Xeon E5 will have eight of them. Clock speeds will be a bit lower (2.0-2.5GHz), but we can safely assume that the new Xeon E5 will outperform its older brother by a noticeable margin and make it even harder for the new Opteron to compete in the higher end of the 2P market.
At the low-end, we see some interesting offerings from AMD. Our impression is that the 6212 at 2.6-2.9GHz is very likely to offer a better performance per dollar ratio than the low-end Xeons E560x that lack Hyper-Threading and turbo support.
Okay, we've done enough analyzing of paper specs; let's get to the hardware and the benchmarks. Before we do that, we'll elaborate a bit on what a server centric architecture should look like. What makes server applications tick?
Post Your CommentPlease log in or sign up to comment.
View All Comments
JohanAnandtech - Thursday, November 17, 2011 - link
1) Niagara is NOT a CMT. It is interleaved multipthreading with SMT on top.
I haven't studied the latest Niagaras but the T1 was a fine grained mult-threaded CPU. It switched like a gatling gun between threads, and could not execute two threads at the same time.
Penti - Thursday, November 17, 2011 - linkSPARC T2 and onwards has additional ALU/AGU resources for a half physical two thread (four logically) solution per core with shared scheduler/pipeline if I remember correctly. That's not when CMT entered the picture according to SUN and Sun engineers any way. They regard the T1 as CMT as it's chip level. It's not just a CMP-chip any how. SMT is just running multiple threads on the cpus, CMP is working the same as SMP on separate sockets. It is not the same as AMDs solution however.
Phylyp - Tuesday, November 15, 2011 - linkFirstly, this was a very good article, with a lot of information, especially the bits about the differences between server and desktop workloads.
Secondly, it does seem that you need to tune either the software (power management settings) or the chip (CMT) to get the best results from the processor. So, what advise is AMD offering its customers in terms of this tuning? I wouldn't want to pony up hundreds of dollars to have to then search the web for little titbits like switching off CMT in certain cases, or enabling High-performance power management.
Thirdly, why is the BIOS reporting 32 MB of L2 cache instead of 8 MB?
mino - Wednesday, November 16, 2011 - linkNo need for tuning - turbo is OS-independent (unless OS power management explicitly disables it aka Windows).
Just disable the power management on the OS level (= high performance fro Windows) and you are good to go.
JohanAnandtech - Thursday, November 17, 2011 - linkThe BIOS is simply wrong. It should have read 16 MB (2 orochi dies of 8 MB L3)
gamoniac - Tuesday, November 15, 2011 - linkThanks, Johan. I run HyperV on Windows Server 2008 R2 SP1 on Phonem II X6 (my workstation) and have noticed the same CPU issue. I previously fixed it by disabling AMD's Cool'n'Quiet BIOS setting. After switching to high performance increase my overall power usage by 9 watts but corrected the CPU capping issue you mentioned.
Yet another excellent article from AnandTech. Well done. This is how I don't mind spending 1 hour of my precious evening time.
mczak - Tuesday, November 15, 2011 - linkL1 data and instruction cache are swapped (instruction is 8x64kB 2-way data is 16x16kB 4-way)
L2 is 8x2MB 16-way
JohanAnandtech - Thursday, November 17, 2011 - linkfixed. My apologies.
hechacker1 - Tuesday, November 15, 2011 - linkCurious if those syscalls for virtualization were improved at all. I remember Intel touting they improved the latency each generation.
I'm guessing it's worse considering the increased general cache latency? I'm not sure how the latency, or syscall, is related if at all.
Just curious as when I do lots of compiling in a guest VM (Gentoo doing lots of checking of packages and hardware capabilities each compile) it tends to spend the majority of time in the kernel context.
hechacker1 - Tuesday, November 15, 2011 - linkJust also wanted to add: Before I had a VT-x enabled chip, it was unbearably slow to compile software in a guest VM. I remember measuring latencies of seconds for some operations.
After getting an i7 920 with VT-x, it considerably improved, and most operations are in the hundred or so millisecond range (measured with latencytop).
I'm not sure how the latests chips fare.