The Evolution of DLSS: A New Era in AI-Powered Upscaling

Nvidia's Deep Learning Super Sampling (DLSS) technology has come a long way since its introduction, and with each new iteration, it's getting closer to rivaling traditional upscaling methods. In this article, we'll explore the changes made in DLSS 2.0 and 2.1, and how they've improved image quality and performance.

With DLSS 2.0, Nvidia has moved away from training AI models on individual games, instead using a generalized dataset to improve accuracy. The new version now uses 16K reference images to enhance clarity, and incorporates motion vector data from the game engine to predict where moving objects will be in the next frame. This improvement has resulted in more accurate upscaling, making it a valuable tool for developers looking to create stunning visuals.

But what about performance? With DLSS 2.0, Nvidia has added an ultra-performance mode that offers 8K resolution at 60 FPS, as well as VR support and the ability to upscale games that dynamically alter their own resolution based on scene complexity. This feature is a game-changer for developers who want to create immersive experiences without sacrificing performance.

One of the most significant changes in DLSS 2.0 is the added feature of 4K tier for image quality comparisons. With this setting, you can see exactly what's happening behind the scenes, including corner cameras, which provide valuable insights into the rendering process. The 4K tier also offers a more detailed comparison of upscaling algorithms, making it easier to evaluate their performance.

So, what does it cost to have these improvements? To find out, we ran LDAC latency tests on our 3090 system using both 4K native and DLSS performance mode. The results showed that we gained about an extra millisecond of click-to-photon latency in average, which is a relatively small price to pay for the increased frame rate.

In performance mode, which uses 1080p textures to fill in at 4K, it's an extremely compelling story. We've got more than double the original frame rate in both Control and Cyberpunk compared to native 4K on our 3060, with Metro Exodus: Enhanced Edition seeing slightly worse results but still impressive nonetheless.

But there's a bigger question mark surrounding DLSS: why are we accepting anything other than traditional brute-force pixel-perfect approaches to driving more FPS? The answer lies in the limitations of GPU manufacturers and the increasing resolution of displays. With RTX 2000 series, only about 10% of the die area was allocated to RTX and other AI features like DLSS, leaving the rest for other tasks.

Despite these limitations, DLSS has become an essential tool for developers who want to create stunning visuals without sacrificing performance. And with the rumored Nintendo Switch Pro on the horizon, it's clear that DLSS will play a huge role in allowing this console to achieve high-resolution gaming in a power-efficient design.

Another exciting development is LG's new generation OLED Evo panel, which offers brighter and punchier images with all the traditional benefits of OLED, including perfect blacks and per-pixel illumination. This panel manages to look like a gallery display while feeling like a gaming monitor with support for adaptive refresh rates, HDR, up to 120 Hertz 4K over HDMI 2.1, and lightning-fast pixel response times for excellent image clarity.

But what about gamers who don't need a card that can drive this panel? Nvidia's DLSS technology is here to help, enabling users to enjoy stunning visuals without sacrificing performance. And with features like Game Optimizer, which optimizes the picture by genre of game, and Motion Pro, which aims to reduce motion blur for games that don't allow it to be turned off, DLSS has become an indispensable tool for gamers who want the best possible experience.

So, are you enabling DLSS in games that support it? Do you still notice the weirdness that can occur when using this technology? Let us know in the comments. And if you enjoyed this video, maybe check out our other content as well.

"WEBVTTKind: captionsLanguage: en- Free performance?Wow!How much does it cost?It's free?Nothing in life is free.But Nvidia's DLSS is really close.Look at these images side by side.One of them is runningat native 4K resolution,and one of them is an imposterrunning at just a quarter ofthat, then upscaling to 4K.Can't tell which is which?You're not alone.- I think this game is goingto be really hard to tell.- It's pretty hard to tell the difference.- Not a lot of difference.- The best part is that theGPU in our upscaled machineonly costs a quarter as much.Let's talk about how thistechnology works thenand see who from the LTT teamwas ultimately able totell the difference.Our test was generously sponsored by LG,who also provided their new G1 TVfeaturing a second gen OLED Evo panel.This is as good as it gets for gaming TVs,and will give ourparticipants their best shotat picking out the imposter 4K GPU.(beeping)(upbeat music)There are many obvious benefitsto a premium graphics card.Typically they have enoughperformance headroomto enable advanced visual features,like more realistic hair rendering,real-time dynamic lightingand higher quality textures.Of course, you can get all of those thingsout of a mid-range GPU,but if you can't renderenough frames per second,they come at the costof animation smoothnessand input responsiveness.That's why medium settings exist.Of course, if you don't liketurning the details down,another great way to get more FPSis to drop down the resolutionthat your GPU is rendering at.But take it too far, and thedownsides of stair-steppingor aliasing and chunky looking texturesstart to outweigh the smoothness benefits.That's where DLSS comes in.It combines the visual benefitsof high end game settingswith the performance benefitsof lower resolution rendering,using a process that I call magic.I'm just kidding.This is magic.Lttstore.com.DLSS, that's not magic.And it wasn't alwaysactually even very good.In the first iteration,Nvidia would generateboth low resolution framesand then the same scene again,but super sampled a perfect frame,then they would train their AIto compare them pixel bypixel and fill in the gaps,all of this in real time onthe RTX series tensor cores.Very cool, but very flawed.Version 1.0 lacked temporal awareness,which made it basically impossibleto accurately upscale moving objects.And it had to be painstakingly implementedgame by game, scene by scene,which led to all manner of distractingand even comical results.Oh yeah, and the performanceuplift wasn't that great.But with version 2.0 and higher,the results are pretty shocking.In our double blind test,subjects played threegames on our LG G1 OLED TV,Cyberpunk 2077, Control andMetro Exodus: Enhanced Edition.- So I'm just trying totell which one is which?- Yeah.- Okay.- Avert your gaze.Let's do like this.- This is gonna be really embarrassingif I don't get it right.- Our two gaming PCs had an RTX3090 for native 4K renderingand an RTX 3060 using DLSSwith otherwise identical specs.There was no time limitand our participantscould switch between theinputs as often as they liked.- The sign is just completely,it's like the qualityis turned way up, so.Okay, what would that mean?So on DLSS you get to turn the details upand still run a higher frame rate.So whereas a native, you would have-To maintain the frame rate,you turn the details down.So if this one was DLSS,they would have thedetails turned up higher.- Yeah, like, ifyou weren't analyzing this.- I would not notice the difference.It would be the same.- God, there is no different at all.Maybe I'm just looking at the wrong thing.Let me go somewhere else.- Between the reflections over hereand the character's hair textures,I'm pretty sure this onehere is the 4K native.The definition between the strands of hairis a little bit less over here.It seems to be almost like,I don't want to say blurred,kind of more like it's painted on,whereas over here,the definition seems tobe more self-shadowed.So there seems to be more actual like,pixel work going in.Also, if you look over here,there's a little bit of detailin this reflection here,but on this one, wedon't have that detail.So I'm pretty sure this is DLSS.- Yeah, this is DLSS.See this, this lamppost right here?- Yeah.There's a little bit of kindof artifact when you move.When you're still,you cannot tell,.But when you move slowly,you can see just a little bit of kind of,it's like a fuzziness.Boy is it subtle, though.- While Anthony and Iboth correctly identifiedthe DLSS machine every time,even in performance mode,which renders at just 1080p and upscales,we both relied on our technical knowledgerather than trying to subjectively pickthe better looking one,and we both agreed thatwithout specifically looking for clues,almost no gamer wouldnotice the differencein the course of normal gameplay.And this is backed up bythe rest of our results.Interestingly, Andy,who's a bit more into theartistic photography scene,ended up actually preferringthe slightly softer lookof DLSS in some cases.If you want to see what we saw,the screen capture from both systemswill be uploaded with corner camerasas a behind-the-scenes on Floatplane.I'd recommend the 4K tier forimage quality comparisons.So what did Nvidia changeto make it this good?Well, with DLSS 2.0 and up,the AI is no longer trainedon each game individually,but rather on a generalized dataset.It now uses 16K referenceimages to improve clarity,and it now takes motion vectordata from the game engineto predict where movingobjects are going to bein the next frame.That improves accuracy.The smaller updates havemostly added featureswith 2.1 adding an 8Kultra performance mode,VR support, and theability to upscale gamesthat dynamically altertheir own resolution,depending on scene complexity,and DLSS 2.2 was mostly focused onperformance improvements and bug fixes.Of course, as I said before,nothing in life is free.So what's it costing us?Well, to find out, weran LDAC latency testson our 3090 system using both 4K nativeand DLSS performance mode.And what we found was that we gotabout an extra one millisecondof click to photon latency on average.And considering that youcan be doubling your FPSin many cases, whileDLSS is clearly addingsome processing time,to me that looks likea pretty good trade-off for the increasein frames per second.Also, as we played through these gamesto find good comparisonscenarios for our testers,we found some more obviousexamples of poor upscaling.But that's actually about it.In performance mode, which isusually using 1080p texturesto fill in at 4K, it's anextremely compelling story.We've got more than doublethe original frame ratein both Control and Cyberpunkcompared to native 4K on our 3060,and while Metro Exodus: Enhanced Editionsaw slightly worse results,they're still very impressive.But we haven't answeredthe biggest question here.For years, hackycorner-cutting in GPU driverswas considered cheating.So why are we accepting anythingother than the traditionalbrute force pixel-perfectapproach to driving more FPS?The answer is that wedon't really have a choice.Developers keep buildingmore and more realistic looking gamesand manufacturers likeLG, wonderful OLED TVs,thanks for sponsoring the video,have pushed the resolutionsof their displays so highthat GPU makers couldn'tpossibly hope to keep upunless you want your GPU totake up half of your caseand double as a space heaterfor your two bedroom apartment.And given that the RTX 2000series only had an estimated10% of its die area allocated to RTXand other AI features like DLSS,it actually seems likea pretty good trade-offas long as the imagequality is up to snuff.And it's useful for more than just savinga buck on your GPU.If you have an enthusiast graphics card,who's to say you don't wanteven more FPS for free-ish?Also, I'm expectingDLSS to play a huge rolein allowing the rumoredNintendo Switch Proto achieve high resolution gamingin a super power-efficient design.Of course, when and even if that's comingis a big question mark right now.What isn't a question markis that I hope to beenjoying the maybe Switch Proon one of these new G1 OLEDs from LG.LG's next generation OLED Evo panelsoffer brighter and punchier imageswith all the traditional benefits of OLED,like perfect blacks andper-pixel illumination.This thing manages to looklike a gallery display,while feeling like a gaming monitorwith support for adaptive refresh rates,HDR, up to 120 Hertz 4K over HDMI 2.1,and of course, lightningfast pixel response timesfor excellent image clarity.It really is the perfect TV for 4K gamingif you've got a card that can drive it.But then, hey, maybe thanks to DLSS,you don't even need acard that can drive it.Haha, got 'em.Oh, LG also wanted me tomention Game Optimizer,which optimizes the pictureby genre of game that you're playing,and Motion Pro, whichaims to reduce motion blurfor games that don'tallow you to turn it off.So are you enabling DLSSin games that support itor do you still notice theweirdness that can occur?I mean, hey, I could tell,but it was a lot harderthan I would have thoughtwe'd ever get to based onmy experience with DLSS 1.0.So let us know in the comments.And if you enjoyed this video,maybe check out our \"Is RTX atotal waste of money?\" video.Those results might surprise you too.- I mean, if you're gonnaspend $2,000 on a 3060, anyway,(laughing)Ugh, I wish things weren't so bad.\n"