Intel's Latest Advancements in AI and Cloud Computing: A New Era for High-Performance Computing

Intel has been making significant strides in recent years to establish itself as a leader in high-performance computing, particularly in the realm of artificial intelligence. The company's latest advancements in this field are centered around its Xeon 6 processor, which is designed to tackle complex AI workloads with ease. But what exactly does this mean for Intel and its customers? Let's dive deeper into the details.

One of the most significant developments in Intel's latest offerings is its focus on AI inferencing performance. The company has recognized that AI processing power is a critical component of many modern applications, from autonomous vehicles to medical imaging. By investing heavily in AI research and development, Intel aims to provide customers with the tools they need to build high-performance AI systems.

But how does this translate into real-world benefits for customers? According to Intel, its latest processors are capable of delivering significant performance gains over existing solutions. For example, when compared to Nvidia GPUs, Intel's Xeon 6 processor can deliver impressive results without sacrificing power efficiency. This is a critical advantage in the datacenter, where energy costs and cooling requirements can be substantial.

Intel has also made significant strides in improving its PCIe connectivity and networking capabilities. The company's latest processors feature 200 Gbps of bandwidth and two channels for high-speed data transfers. This makes them well-suited for demanding applications that require low latency and high throughput.

But what about the role of GPUs in AI workloads? While Intel has traditionally focused on CPU-based solutions, the company recognizes that GPUs can play a critical role in certain applications. In fact, Intel's AMX technology allows it to offload certain tasks from the GPU, freeing up resources for more complex processing tasks.

One interesting development is IBM's announcement of its use of Gudy for its cloud offerings. According to IBM, this technology provides secure isolation and performance advantages that make it an attractive solution for cloud-based applications. While details are still scarce, it's clear that Intel sees significant potential in this space.

Intel's focus on AI inferencing performance also raises the question of how CPUs will be used in the future. While GPUs have traditionally dominated the field of deep learning, CPUs are well-suited for general-purpose compute tasks. However, as AI workloads become increasingly complex, it's likely that we'll see a convergence of CPU and GPU capabilities.

In terms of specific use cases, one area where Intel's latest processors shine is in cloud computing applications. For example, the company's Xeon 6 processor has been shown to deliver impressive performance gains when paired with its own Optane storage solutions. This makes it an attractive option for customers looking to build high-performance cloud infrastructure.

Another interesting development is Intel's new socket and format, which promises significant power efficiency and performance advantages. According to Intel, this technology enables the company to deliver "one of the strongest performance products in years." While details are still scarce, it's clear that Intel sees significant potential in this space.

Of course, no discussion of high-performance computing would be complete without mentioning AMD's upcoming chip launch. According to sources, AMD is working on a new generation of processors that promises significant performance gains over existing solutions. Whether or not these claims hold up to scrutiny remains to be seen, but it's clear that the competition in this space will only continue to intensify.

In conclusion, Intel's latest advancements in AI and cloud computing are promising developments that promise significant benefits for customers. From its focus on AI inferencing performance to its improvements in PCIe connectivity and networking capabilities, Intel is well-positioned to tackle some of the toughest challenges in high-performance computing. As we move forward into an era where AI workloads become increasingly complex, it's clear that Intel will be playing a critical role.

The author recently had the opportunity to get hands-on with Intel's Xeon 6 processor and Optane storage solutions. The results were impressive, with the system delivering exceptional performance gains over existing solutions. Whether or not this translates into real-world benefits for customers remains to be seen, but one thing is clear: Intel is on the right track.

For those interested in exploring high-performance computing applications, the author recommends taking a closer look at Intel's Optane storage solutions. These innovative technologies promise significant performance gains and power efficiency advantages over existing storage solutions. As cloud computing continues to grow in popularity, these solutions will be critical for customers looking to build high-performance infrastructure.

Finally, the author wants to hear from readers! What types of workloads are you interested in running on Intel's Xeon 6 processor or other high-performance computing platforms? Are there any specific applications or use cases that you think would benefit from improved performance or power efficiency? The author is always eager to hear from readers and explore new opportunities for high-performance computing.

"WEBVTTKind: captionsLanguage: enoh no Intel's in fire just they're on fire they're in dire Financial Straits the sky is falling the stock prices is the worst that it's been in 30 years Intel doesn't have the products that the market wants yeah it's it's overblown those are Wall Street Bros manipulating things I'm not sure the Wall Street is connected in any way to reality anymore we had everything before us we have nothing before us now but until xon 6 is launching Andy 3 and I'm just back from a visit to Intel in Portland boy do I have some thoughts okay okay let's chat Intel zon 6 for server and Intel gy 3 for AI are the crown jewels of the most ambitious engineering we've seen from Intel in a decade and these products are entering availability shortly you know Intel's saying we're bringing things to Market a little ahead of schedule it's pretty wild it's a result of more than 3 years of NOS to the grindstone engineering with a completely different approach to product than Intel has taken in an age an Epoch if you will just over a month ago I reviewed Sierra Forest May the forest to be with you and with this product I was raised you know my eyebrow hyperscalers really sort of want this it's it's 144 cores and an eight memory Channel platform that's the Zeon 6700 platform but it was only available with eor and my eyebrow was raised because it was actually good but this this is what's coming now this is the 12 channel configuration the 6900 series zons those are peores 128 of them 500 watts per socket this is the performance Crown Jewel the other one was the density and performance per watt Crown Jewel and memory memory is one big differentiator that Intel sees itself having in the market here Intel is moving up the supported memory speeds a lot for one dim per Channel configurations they also support something called Multiplex rank dims Multiplex 8800 what an incredible Leap Forward in speed it's really like two dims on one PCB with some slightly different clock geometry I mean 8800 sounds amazing but a multiplex really just 24,400 dims on one physical package it's about 30% faster real world scenario so I think of it like two dims on one PCB that can clock together clock independently 8,800 megga transfers it's little different Intel offers another route to high memory capacities and that's cxl some benchmarking that showed off with ilm and a system that had about 1 and a half terabytes of ddr5 and 3/4 of a terab of ddr4 that was only 3% slower than a system fully outfitted with 2.25 GB of ddr5 and you got to have a smart memory controller to pull that off that's not much of a performance loss at all that's it's actually it's a kind of mind-blowing that's without Numa in there either I mean if your OS is smart you can use cxl devices uh as a memory tier but only 3% performance loss on ilm is impressive without having to resort to any of that and that is thousands and thousands of dollars off the cost one system so imagine the savings across a data center it's a lot to be excited about with the new platform launches and yes Intel is doing pees for the 6700 socket as well in case 128 courses Overkill now one thing that was absent from the road mapap and that was single socket configurations is Intel going to offer a discount in single socket configurations because 128 P cord single socket that's that's what a lot of people need at this point I mean they don't need two sockets where you could get 128 performance cores and just to be clear what's launching today is the Intel zeeon 6900 P that's P cor so right now you have the 6700 platform with ecores as a choice and the 6900 platform with peores as a choice but coming q1 of 2024 is ecores on the 6900 platform and PES on the 6700 platform up to 8 Sierra Forest cores per socket in that 12 memory Channel platform and I sort of called around I mean the 6700 e has been out for a little while I've got hands on one I've done real world workloads very impressive but you know it doesn't matter what I say if customers aren't buying it and from what I hear from my contacts inside of large system manufacturers there are actually a lot of customers interested in this platform or have been satisfied with their test workloads to the extent that these partners that build motherboards and systems around these process processors uh didn't build enough so that's encouraging it's very very encouraging to see folks say wow Intel is shattering the way that they used to do their work in Zeon and server space they're thinking about stuff in terms of new paradigms and these products could not get here fast enough ecores for efficiency P cores for performance also in Intel's press material hex mode what is hex mode hex mode is making the entire CPU act as if it is one Numa node now some softwares not really designed to deal with that 128 P cores one Newman node it's Madness but Intel has hex mode that they call for that but you can also still do subn Numa clustering subn Numa clustering is a common Zeon feature subn Numa clustering you turn that on and you can get three Numa domains because there's three compute tiles on the P core 6900 and yeah it gives you three Numa domains 128 doesn't divide evenly into three can't wait to go handson now as for gouty 3 and the AI side of it AI is where the money is it's looking promising and tells focusing on inferencing performance and focusing on corporate wins where they can get them with certain kinds of workloads a corporate workloads where the customer doesn't know or care about which flavor of linear algebra is happening under the hood such as with IBM the End customer doesn't care about Cuda is not looking for Cuda compatibility they just want to run their AI stuff and IBM was here at Intel's event in Portland and you know they really complemented Intel's process and working with them hand in hand to come up with the perfect AI Cloud product Intel and Ai and IBM in the cloud well yeah and that's an opportunity for Intel developers to learn see fact is Intel still has more software developers on staff than just about any other company in the space and processor development and everything else and Intel software devs are their shot I mean it's Intel shot at differentiating itself competitively I mean Intel's devs are basically going to be responsible for carrying Intel into a new era of competition now don't get me wrong Zeon 6 is hugely important in the AI workload for gluing the AI stuff together I mean zon 6 is definitely not going away it's just that in a line go up kind of a world the types of silicon that can do AI is uh more closely connected to the money spigot so that's more gouty 3 than zon 6 it looks pretty easy to deploy a 32 node gudy system for basically any business it's a pretty standard layout that doesn't require proprietary interconnect technology and Intel's plan is that they can leverage their software team and software stack to run both an internal large language model as well as a customer service large language model and be cryptographically assured that no data can leak from one of those workloads into the other which is a bit of a problem right now with AI workloads and of course that is a gouty based solution but all of this is running locally or maybe at the edge but I have a feeling the promise functionality here around the secure isolation is part of what IBM has in mind to use gudy for their Cloud offerings for IBM customers one of the interesting things that Intel showed in their AI workload slides was that just using Xeon the new Zeon 6 as a platform to carry Nvidia gpus will uplift the performance of the Nvidia gpus because the gpus don't spend all their time just GPU they have to be fed by data that's pcie that's memory that's interconnect speed and if you look at what Intel has done for their G3 architecture those internal changes to zeeon in terms of being the heart of this machine will feed other platforms just as well so by building a cohesive solution they're not able to sort of ignore what other customers will require in terms of pcie connectivity in terms of cxl compatibility in terms of networking platform 200 gbits and two channels for their own intern internal gy 3 Translate to to real world Network and dma transfers and so that looks good no matter what sort of AI platform you're building with Zeon CPUs at the heart of it CPUs are not enough but while CPUs by themselves are not enough if the CPU is not good enough it will also be a little bit of a bottleneck I mean the joke is you could hang an h100 off of a TI 92 and probably sell that on the internet but the reality is you're still leaving some performance on the table if you do that yes TI 92 the graphing calculator now let's be clear here Intel is focusing on AI inferencing performance they're not giving up anything on the AI Al some of those Wall Street Bros have widely misreported that Intel is giving up on AI Intel cannot give up on AI because that's where the money spigot is and gudy 3 focusing on inferencing I think is a good move for Intel 2x close to 2x a little bit better than 2x the h100 inferencing performance is huge if that were in the market 6 months ago or even 3 months ago or even a month ago then Intel would have tons of wins I think if the software stack and everything else is ready to go me I want to go Hands-On with this and see how it actually Stacks up but the slides look promising as well as what IBM was saying because IBM has kicked the tires on this and it does look pretty interesting from what they said for the other performance intels showing their performance slides looks pretty good whether you're running General compute or AI but again what's going to matter is Hands-On your specific workload the specific things you want to see you want your specific stuff to run fast and as a competitive differentiator Intel does have AMX and Intel is able to do certain types of AI workloads actually on the CPU and have reasonable performance if you are uh what Intel calls sort of at the frontier with what you're doing for your AI workloads chances are you're not even in GPU territory yet you're working on Prim you're working with other sorts of experimental libraries and chances are that's going to be a CPU bound application at least until you figure it out at least until you sit down and start optimizing for what can actually run on a GP CPU and a CPU is still the best place for really general purpose compute now I'll soon be Hands-On with an Intel Xeon 6 with peores in the new socket and format but so far it looks like it's going to be a much stronger performance product than Intel has had in years the very high-end but still very power efficient and that looks like it's going to be well positioned to complement Intel xon 6 Sierra Forest with the ecores at the same time AMD has got a chip launch coming up soon I mean it's got to be soon right the next 5 to 10 years is going to be unlike anything the world has ever seen and I'm here for it and I'm pretty excited and what this level one if you want to run a workload or let me know what kind of workloads I should be running on these systems what you want to see I mean I just got done with garage it's a plug-in drop in S3 replacement S3 compatible and that runs like a dream on the Zeon 6 Sierra Forest ecor system 6700 so if you want to replace your own you know S3 with something that's self-hosted or use it as a as a backup Target or whatever but it's you know self-hosted distributed in your own cloud self-managed whatever that's a pretty reasonable Solution that's a video for another day I'm W it's level one I'm signing out and you can find me in the level one forumsoh no Intel's in fire just they're on fire they're in dire Financial Straits the sky is falling the stock prices is the worst that it's been in 30 years Intel doesn't have the products that the market wants yeah it's it's overblown those are Wall Street Bros manipulating things I'm not sure the Wall Street is connected in any way to reality anymore we had everything before us we have nothing before us now but until xon 6 is launching Andy 3 and I'm just back from a visit to Intel in Portland boy do I have some thoughts okay okay let's chat Intel zon 6 for server and Intel gy 3 for AI are the crown jewels of the most ambitious engineering we've seen from Intel in a decade and these products are entering availability shortly you know Intel's saying we're bringing things to Market a little ahead of schedule it's pretty wild it's a result of more than 3 years of NOS to the grindstone engineering with a completely different approach to product than Intel has taken in an age an Epoch if you will just over a month ago I reviewed Sierra Forest May the forest to be with you and with this product I was raised you know my eyebrow hyperscalers really sort of want this it's it's 144 cores and an eight memory Channel platform that's the Zeon 6700 platform but it was only available with eor and my eyebrow was raised because it was actually good but this this is what's coming now this is the 12 channel configuration the 6900 series zons those are peores 128 of them 500 watts per socket this is the performance Crown Jewel the other one was the density and performance per watt Crown Jewel and memory memory is one big differentiator that Intel sees itself having in the market here Intel is moving up the supported memory speeds a lot for one dim per Channel configurations they also support something called Multiplex rank dims Multiplex 8800 what an incredible Leap Forward in speed it's really like two dims on one PCB with some slightly different clock geometry I mean 8800 sounds amazing but a multiplex really just 24,400 dims on one physical package it's about 30% faster real world scenario so I think of it like two dims on one PCB that can clock together clock independently 8,800 megga transfers it's little different Intel offers another route to high memory capacities and that's cxl some benchmarking that showed off with ilm and a system that had about 1 and a half terabytes of ddr5 and 3/4 of a terab of ddr4 that was only 3% slower than a system fully outfitted with 2.25 GB of ddr5 and you got to have a smart memory controller to pull that off that's not much of a performance loss at all that's it's actually it's a kind of mind-blowing that's without Numa in there either I mean if your OS is smart you can use cxl devices uh as a memory tier but only 3% performance loss on ilm is impressive without having to resort to any of that and that is thousands and thousands of dollars off the cost one system so imagine the savings across a data center it's a lot to be excited about with the new platform launches and yes Intel is doing pees for the 6700 socket as well in case 128 courses Overkill now one thing that was absent from the road mapap and that was single socket configurations is Intel going to offer a discount in single socket configurations because 128 P cord single socket that's that's what a lot of people need at this point I mean they don't need two sockets where you could get 128 performance cores and just to be clear what's launching today is the Intel zeeon 6900 P that's P cor so right now you have the 6700 platform with ecores as a choice and the 6900 platform with peores as a choice but coming q1 of 2024 is ecores on the 6900 platform and PES on the 6700 platform up to 8 Sierra Forest cores per socket in that 12 memory Channel platform and I sort of called around I mean the 6700 e has been out for a little while I've got hands on one I've done real world workloads very impressive but you know it doesn't matter what I say if customers aren't buying it and from what I hear from my contacts inside of large system manufacturers there are actually a lot of customers interested in this platform or have been satisfied with their test workloads to the extent that these partners that build motherboards and systems around these process processors uh didn't build enough so that's encouraging it's very very encouraging to see folks say wow Intel is shattering the way that they used to do their work in Zeon and server space they're thinking about stuff in terms of new paradigms and these products could not get here fast enough ecores for efficiency P cores for performance also in Intel's press material hex mode what is hex mode hex mode is making the entire CPU act as if it is one Numa node now some softwares not really designed to deal with that 128 P cores one Newman node it's Madness but Intel has hex mode that they call for that but you can also still do subn Numa clustering subn Numa clustering is a common Zeon feature subn Numa clustering you turn that on and you can get three Numa domains because there's three compute tiles on the P core 6900 and yeah it gives you three Numa domains 128 doesn't divide evenly into three can't wait to go handson now as for gouty 3 and the AI side of it AI is where the money is it's looking promising and tells focusing on inferencing performance and focusing on corporate wins where they can get them with certain kinds of workloads a corporate workloads where the customer doesn't know or care about which flavor of linear algebra is happening under the hood such as with IBM the End customer doesn't care about Cuda is not looking for Cuda compatibility they just want to run their AI stuff and IBM was here at Intel's event in Portland and you know they really complemented Intel's process and working with them hand in hand to come up with the perfect AI Cloud product Intel and Ai and IBM in the cloud well yeah and that's an opportunity for Intel developers to learn see fact is Intel still has more software developers on staff than just about any other company in the space and processor development and everything else and Intel software devs are their shot I mean it's Intel shot at differentiating itself competitively I mean Intel's devs are basically going to be responsible for carrying Intel into a new era of competition now don't get me wrong Zeon 6 is hugely important in the AI workload for gluing the AI stuff together I mean zon 6 is definitely not going away it's just that in a line go up kind of a world the types of silicon that can do AI is uh more closely connected to the money spigot so that's more gouty 3 than zon 6 it looks pretty easy to deploy a 32 node gudy system for basically any business it's a pretty standard layout that doesn't require proprietary interconnect technology and Intel's plan is that they can leverage their software team and software stack to run both an internal large language model as well as a customer service large language model and be cryptographically assured that no data can leak from one of those workloads into the other which is a bit of a problem right now with AI workloads and of course that is a gouty based solution but all of this is running locally or maybe at the edge but I have a feeling the promise functionality here around the secure isolation is part of what IBM has in mind to use gudy for their Cloud offerings for IBM customers one of the interesting things that Intel showed in their AI workload slides was that just using Xeon the new Zeon 6 as a platform to carry Nvidia gpus will uplift the performance of the Nvidia gpus because the gpus don't spend all their time just GPU they have to be fed by data that's pcie that's memory that's interconnect speed and if you look at what Intel has done for their G3 architecture those internal changes to zeeon in terms of being the heart of this machine will feed other platforms just as well so by building a cohesive solution they're not able to sort of ignore what other customers will require in terms of pcie connectivity in terms of cxl compatibility in terms of networking platform 200 gbits and two channels for their own intern internal gy 3 Translate to to real world Network and dma transfers and so that looks good no matter what sort of AI platform you're building with Zeon CPUs at the heart of it CPUs are not enough but while CPUs by themselves are not enough if the CPU is not good enough it will also be a little bit of a bottleneck I mean the joke is you could hang an h100 off of a TI 92 and probably sell that on the internet but the reality is you're still leaving some performance on the table if you do that yes TI 92 the graphing calculator now let's be clear here Intel is focusing on AI inferencing performance they're not giving up anything on the AI Al some of those Wall Street Bros have widely misreported that Intel is giving up on AI Intel cannot give up on AI because that's where the money spigot is and gudy 3 focusing on inferencing I think is a good move for Intel 2x close to 2x a little bit better than 2x the h100 inferencing performance is huge if that were in the market 6 months ago or even 3 months ago or even a month ago then Intel would have tons of wins I think if the software stack and everything else is ready to go me I want to go Hands-On with this and see how it actually Stacks up but the slides look promising as well as what IBM was saying because IBM has kicked the tires on this and it does look pretty interesting from what they said for the other performance intels showing their performance slides looks pretty good whether you're running General compute or AI but again what's going to matter is Hands-On your specific workload the specific things you want to see you want your specific stuff to run fast and as a competitive differentiator Intel does have AMX and Intel is able to do certain types of AI workloads actually on the CPU and have reasonable performance if you are uh what Intel calls sort of at the frontier with what you're doing for your AI workloads chances are you're not even in GPU territory yet you're working on Prim you're working with other sorts of experimental libraries and chances are that's going to be a CPU bound application at least until you figure it out at least until you sit down and start optimizing for what can actually run on a GP CPU and a CPU is still the best place for really general purpose compute now I'll soon be Hands-On with an Intel Xeon 6 with peores in the new socket and format but so far it looks like it's going to be a much stronger performance product than Intel has had in years the very high-end but still very power efficient and that looks like it's going to be well positioned to complement Intel xon 6 Sierra Forest with the ecores at the same time AMD has got a chip launch coming up soon I mean it's got to be soon right the next 5 to 10 years is going to be unlike anything the world has ever seen and I'm here for it and I'm pretty excited and what this level one if you want to run a workload or let me know what kind of workloads I should be running on these systems what you want to see I mean I just got done with garage it's a plug-in drop in S3 replacement S3 compatible and that runs like a dream on the Zeon 6 Sierra Forest ecor system 6700 so if you want to replace your own you know S3 with something that's self-hosted or use it as a as a backup Target or whatever but it's you know self-hosted distributed in your own cloud self-managed whatever that's a pretty reasonable Solution that's a video for another day I'm W it's level one I'm signing out and you can find me in the level one forums\n"