A Conversation with Gudy 3: The Next Generation of Large Language Models

So, let's see what our LLM has to say. As we can see, it's a standard text answer, but it's also multimodal, which means we have this great visual of the chest X-ray here. I'm not good at reading X-rays, so I'll spare you my typing skills and do a little cut and pasting. The nice thing about this multimodal LLM is that we can ask it questions to further illustrate what's going on here.

This LLM is actually going to analyze the image and tell us more about this hazy opacity, such as it is. You can see here that it says it's down in the lower left. Again, just a great example of how multimodal LLMs are delivering incredible results. We're not just talking about price; we're also talking about performance and efficiency.

Gudy 3 Architecture: The Future of Large Language Models

Gudy 3 architecture is the only ML perf Benchmark alternative to H100s for LLM training and inference. And Gudy 3 only makes it stronger. We're projected to deliver 40% faster time to train than H100s and 1.5x versus H20s. Faster inference means that we can process information more quickly, which is essential for many applications.

Gudy 3 is expected to deliver 2x the performance per dollar compared to H100s. This means that our customers will get more bang for their buck. We're highly scalable and use open industry standards like Ethernet. We also support all of the expected open-source frameworks like PyTorch, which is great news for developers.

The Ecosystem Behind Gudy 3

We have hundreds of thousands of models available on Hugging Face for Gudy, and with our Developer Cloud, you can experience Gudy capabilities firsthand. Easily accessible and readily available, but of course, this is just the beginning. The entire ecosystem is lining up behind Gudy 3.

Launching Zeon 6 with ecores: The Future of Data Centers

We're launching Zeon 6 with ecores today, which we see as an essential upgrade for modern data centers. High core count, high density, exceptional performance per watt – it's all here. And this is our first product on Intel 3. We're continuing our march back to process technology competitiveness and leadership.

Next Year: The Future of Data Centers

We'll be bringing the second generation of Zeon 6 with ecores later this year, which will feature a whopping 288 cores. This will enable a stunning 6:1 consolidation ratio, better than anything we've seen in the industry. We're not stopping there; we'll also see even more innovation and advancements.

The Impact of Gudy 3 and Zeon 6

So, what does this mean for you? If just 500 data centers were upgraded with what we just saw, this would power almost 1.4 million Taiwanese households for a year or 3.7 million cars off the road for a year. By bringing the second generation of Zeon 6 with ecores later this year, we'll be able to deliver an even more significant impact on sustainability and performance.

Conclusion

Gudy 3 is here, and it's changing the game. With its exceptional performance per dollar and throughput, this LLM is set to revolutionize many industries. And as for Zeon 6 with ecores, this is just the beginning of a new era in data center technology. We're committed to innovation and sustainability, and we can't wait to see what the future holds.

The Power of Gudy 3

I'd like you to fill this rack right with the equivalent compute capability of the Gen 2 using Gen 6. Give me a minute or two, I'll make it happen. Okay, get with it, come on! Hop to it, buddy!

This is important to think about, especially when it comes to data centers. Every data center provider I know today is being crushed by how they upgrade and expand their footprint. The space the flexibility – for high-performance computing they have more demands for AI in the data center. Having a processor with 144 cores versus 28 cores for Gen 2 gives them the ability to both condense as well as attack these new workloads with performance and efficiency that was never seen before.

That's all from today, folks. We hope you enjoyed this conversation with Gudy 3 and Zeon 6 with ecores. Stay tuned for more updates on the latest advancements in large language models and data center technology!

"WEBVTTKind: captionsLanguage: enas we've launched the core Ultra with meteor Lake it also introduced this next generation of chiplet based design and lunar lake is the next step forward and I'm happy to announce it today lunar lake is a revolutionary design it's new IP blocks for CPU GPU and npu it'll power the largest number of NextGen AIP PCS in the industry we already have over 80 designs with 20 OEM that will start shipping and volume in Q3 you know first it starts with a great CPU and with that this is our next Generation lion Cove processor that has significant IPC improvements and delivers that performance while also delivering dramatic power efficiency gains as well so it's delivering core Ultra performance at nearly half the power that we had in meteor Lake which was already a great chip you know the GPU is also a huge step forward it's based on our next Generation C2 IP and it delivers 50% more Graphics of performance and literally we've taken a discrete graphics card and we've shoved it into this amazing chip called lunar Lake alongside this we're delivering strong AI compute performance with our enhanced npu up to 48 tops of performance and as you heard satcha talk about our collaboration with Microsoft and co-pilot Plus and along with 300 other isvs incredible software support more applications than anyone else now some say that the npu is the only thing that you need and simply put that's not true you know and now having engaged with hundreds of isvs most of them are taking advantage of CPU GPU and npu performance in fact our new she2 you know GPU is an incredible on device AI performance engine only 30% of the isvs we've engaged with are only using the npu the GPU and the CPU and combination deliver extraordinary performance the GPU 67 tops with our XMS performance 3 and 1/2x the gains over prior generation and since there's been some talk about this other exolite uh chip coming out and its superiority to the x86 I just want to put that to bed right now ain't true you know lunar Lake Running in our Labs today outperforms the ex Elite on the CPU on the GPU and on AI performance delivering a stunning 120 tops of total platform performance and it's compatible so you don't need any of those compatibility issues you know this is x86 at its finest every Enterprise every customer every historical driver and capability simply works this is a no-brainer everyone should upgrade and you know the final nail in the coffin of this discussion is some say the x86 can't win on power efficiency lunar Lake busts this myth as well this radical new s so architecture and design delivers unprecedented power efficiency up to 40% lower s so performance than meteor Lake which was already very good customers are looking for high performance cost effective Genai training and inference Solutions and they started to turn to Alternatives like gouty you know they want choice they want open open software and Hardware Solutions and time tomarket Solutions at dramatically lower tcos and that's why we're seeing customers like neighbor airel Bosch infosis and Seeker turning to gouty 2 and we're putting these pieces together we're standardizing through the open source community in the Linux Foundation we've created the open platform for Enterprise AI to make Zeon and gouty a standardized AI solution for workloads like rag so let me start with maybe a quick medical query okay so this is Zeon and gouty working together on a medical query so it's a lot of private confidential on-prem data being combined with a open source llm exactly okay very cool all right so let's see what our llm has to say so you can see like a typical llm we're getting you know the text answer here standard but it's a multimodal llm so we also have this great visual here of the chest x-ray okay I'm not good at reading x-rays so what does this say I'm not great either but the nice thing about um and I'm I'm going to spare you my uh my typing skills I'm going to do a little cut and pasting here the nice thing about this uh multimodal llm is we can actually ask it questions to further further illustrate what's going on here so this llm is actually going to Analyze This image and tell us a little bit more about this uh hazy opacity such as it is so you can see here it's saying it's down here in the lower left so once again just a great example of multimodal L and as you see you know gouty is not just winning on price it's also delivering incredible TCO and incredible performance and that performance is only getting better with gudi 3 gudy 3 architecture is the only ml perf Benchmark alternative to h100s for llm training and inferencing and gudy 3 only makes it stronger you know we're projected to deliver 40% faster time to train than h100s and 1.5x versus h20s and you know faster inferencing than h100s and delivering that 2.3x performance per dollar and throughput versus h100s and in training you know G 3 is expected to deliver 2x the performance per dollar you know and this idea is simply music to our customers ear spend less and get more it's highly scalable uses open industry standards like ethernet which we'll talk more about in a second and we're also supporting all of the expected open- Source Frameworks like pytorch VM you know hundreds of thousands of models are now available on hugging face for gudy and with our developer Cloud you can experience gouty capabilities firsthand easily accessible and readily available but of course with this the entire ecosystem is lining up behind gudy 3 and it's my pleasure today to show you the wall of gouty 3 today we're launching Zeon 6 with ecores and we see this as an essential upgrade for the modern data center a high core count high density exceptional performance per watt you know it's also important to note that this is our first product on Intel 3 and Intel 3 is the third of our five nodes in four years as we continue our March back to process technology competitiveness and Leadership next year I'd like you to fill this rack right with the equivalent compute capability of the Gen 2 using Gen 6 okay give me a minute or two I'll make it happen Okay get with it come on Hop to it buddy and you know it's important to think about you know the data centers you know every data center provider I know today is being crushed by how they upgrade how they expand their footprint and the space the flexibility you know for high performance Computing they have more demands for AI in the data center and having a processor with 144 cores versus 28 cores for Gen 2 gives them the ability to both condense as well as to attack these new workloads as well with performance and efficiency that was never seen before so Chuck are you done I'm done I wanted a few more reps but you said equivalent I even put a little bit more okay so let me get it that rack has become this you and what you just saw was eor delivering this distinct Advantage for cloud native and hyperscale workloads 4.2x in media trans code 2.6x performance per watt and from a sustainability perspective this is just gamechanging you know uh a 3:1 rack consolidation over a 4year cycle just one 200 rack data center would save 80k megawatts per uh uh megawatt hours of energy and Zeon is everywhere so imagine the benefits that this could have across the thousands and tens of thousands of data centers in fact if just 500 data centers were upgraded with what we just saw this would power almost 1.4 million Taiwan households for a year 3.7 million cars off the road for a year or Power taipe 101 for 500 years and by the way this will only get better and uh you know if 144 cores is good well let's put two of them together and let's have 288 cores so later this year we'll be Dr bringing the second generation of our Zeon 6 with ecores a whopping 288 cores and this will enable a stunning 6:1 consolidation ratio better claim than anything we've seen in the industryas we've launched the core Ultra with meteor Lake it also introduced this next generation of chiplet based design and lunar lake is the next step forward and I'm happy to announce it today lunar lake is a revolutionary design it's new IP blocks for CPU GPU and npu it'll power the largest number of NextGen AIP PCS in the industry we already have over 80 designs with 20 OEM that will start shipping and volume in Q3 you know first it starts with a great CPU and with that this is our next Generation lion Cove processor that has significant IPC improvements and delivers that performance while also delivering dramatic power efficiency gains as well so it's delivering core Ultra performance at nearly half the power that we had in meteor Lake which was already a great chip you know the GPU is also a huge step forward it's based on our next Generation C2 IP and it delivers 50% more Graphics of performance and literally we've taken a discrete graphics card and we've shoved it into this amazing chip called lunar Lake alongside this we're delivering strong AI compute performance with our enhanced npu up to 48 tops of performance and as you heard satcha talk about our collaboration with Microsoft and co-pilot Plus and along with 300 other isvs incredible software support more applications than anyone else now some say that the npu is the only thing that you need and simply put that's not true you know and now having engaged with hundreds of isvs most of them are taking advantage of CPU GPU and npu performance in fact our new she2 you know GPU is an incredible on device AI performance engine only 30% of the isvs we've engaged with are only using the npu the GPU and the CPU and combination deliver extraordinary performance the GPU 67 tops with our XMS performance 3 and 1/2x the gains over prior generation and since there's been some talk about this other exolite uh chip coming out and its superiority to the x86 I just want to put that to bed right now ain't true you know lunar Lake Running in our Labs today outperforms the ex Elite on the CPU on the GPU and on AI performance delivering a stunning 120 tops of total platform performance and it's compatible so you don't need any of those compatibility issues you know this is x86 at its finest every Enterprise every customer every historical driver and capability simply works this is a no-brainer everyone should upgrade and you know the final nail in the coffin of this discussion is some say the x86 can't win on power efficiency lunar Lake busts this myth as well this radical new s so architecture and design delivers unprecedented power efficiency up to 40% lower s so performance than meteor Lake which was already very good customers are looking for high performance cost effective Genai training and inference Solutions and they started to turn to Alternatives like gouty you know they want choice they want open open software and Hardware Solutions and time tomarket Solutions at dramatically lower tcos and that's why we're seeing customers like neighbor airel Bosch infosis and Seeker turning to gouty 2 and we're putting these pieces together we're standardizing through the open source community in the Linux Foundation we've created the open platform for Enterprise AI to make Zeon and gouty a standardized AI solution for workloads like rag so let me start with maybe a quick medical query okay so this is Zeon and gouty working together on a medical query so it's a lot of private confidential on-prem data being combined with a open source llm exactly okay very cool all right so let's see what our llm has to say so you can see like a typical llm we're getting you know the text answer here standard but it's a multimodal llm so we also have this great visual here of the chest x-ray okay I'm not good at reading x-rays so what does this say I'm not great either but the nice thing about um and I'm I'm going to spare you my uh my typing skills I'm going to do a little cut and pasting here the nice thing about this uh multimodal llm is we can actually ask it questions to further further illustrate what's going on here so this llm is actually going to Analyze This image and tell us a little bit more about this uh hazy opacity such as it is so you can see here it's saying it's down here in the lower left so once again just a great example of multimodal L and as you see you know gouty is not just winning on price it's also delivering incredible TCO and incredible performance and that performance is only getting better with gudi 3 gudy 3 architecture is the only ml perf Benchmark alternative to h100s for llm training and inferencing and gudy 3 only makes it stronger you know we're projected to deliver 40% faster time to train than h100s and 1.5x versus h20s and you know faster inferencing than h100s and delivering that 2.3x performance per dollar and throughput versus h100s and in training you know G 3 is expected to deliver 2x the performance per dollar you know and this idea is simply music to our customers ear spend less and get more it's highly scalable uses open industry standards like ethernet which we'll talk more about in a second and we're also supporting all of the expected open- Source Frameworks like pytorch VM you know hundreds of thousands of models are now available on hugging face for gudy and with our developer Cloud you can experience gouty capabilities firsthand easily accessible and readily available but of course with this the entire ecosystem is lining up behind gudy 3 and it's my pleasure today to show you the wall of gouty 3 today we're launching Zeon 6 with ecores and we see this as an essential upgrade for the modern data center a high core count high density exceptional performance per watt you know it's also important to note that this is our first product on Intel 3 and Intel 3 is the third of our five nodes in four years as we continue our March back to process technology competitiveness and Leadership next year I'd like you to fill this rack right with the equivalent compute capability of the Gen 2 using Gen 6 okay give me a minute or two I'll make it happen Okay get with it come on Hop to it buddy and you know it's important to think about you know the data centers you know every data center provider I know today is being crushed by how they upgrade how they expand their footprint and the space the flexibility you know for high performance Computing they have more demands for AI in the data center and having a processor with 144 cores versus 28 cores for Gen 2 gives them the ability to both condense as well as to attack these new workloads as well with performance and efficiency that was never seen before so Chuck are you done I'm done I wanted a few more reps but you said equivalent I even put a little bit more okay so let me get it that rack has become this you and what you just saw was eor delivering this distinct Advantage for cloud native and hyperscale workloads 4.2x in media trans code 2.6x performance per watt and from a sustainability perspective this is just gamechanging you know uh a 3:1 rack consolidation over a 4year cycle just one 200 rack data center would save 80k megawatts per uh uh megawatt hours of energy and Zeon is everywhere so imagine the benefits that this could have across the thousands and tens of thousands of data centers in fact if just 500 data centers were upgraded with what we just saw this would power almost 1.4 million Taiwan households for a year 3.7 million cars off the road for a year or Power taipe 101 for 500 years and by the way this will only get better and uh you know if 144 cores is good well let's put two of them together and let's have 288 cores so later this year we'll be Dr bringing the second generation of our Zeon 6 with ecores a whopping 288 cores and this will enable a stunning 6:1 consolidation ratio better claim than anything we've seen in the industry\n"