**The First VR Test: Understanding CPU Performance in Virtual Reality**

As we continue to explore the world of virtual reality (VR), it's essential to understand the role of Central Processing Units (CPUs) in delivering smooth and seamless experiences. In this article, we'll dive into our first-ever VR test, which compares the performance of AMD's Ryzen 7 7700X and Intel Core i7-1700 CPUs in various games.

**The Importance of Overclocking**

One of the key aspects to consider when testing CPUs in VR is overclocking. Can overclocking improve performance, or is it a waste of time? To answer this question, we conducted an experiment where we overclocked both CPUs and tested their performance with the same game settings. We then repeated the test without overclocking to ensure a fair comparison. The results showed that overclocking does indeed help to increase frame rates, but the difference may not be as significant as expected.

**The FPS Output Chart**

Next, we created an FPS output chart to visualize the performance of both CPUs in different games. We tested three games: Everest, a high-quality demo with photogrammetry; the Rift game "Boneworks"; and another Rift game "Skyrim VR". The results showed that the AMD Ryzen 7 7700X outperformed the Intel Core i7-1700 in two of the three games, while the gap was smaller in the remaining game. However, when we looked at the average frame rate chart, we found that there was little to no difference between the two CPUs.

**The Frame Time Chart**

We also created a frame time chart to show how each CPU performed in terms of delivering frames within a certain timeframe. The results were largely inconclusive, with both CPUs producing similar frame times. This suggests that neither CPU has a significant advantage over the other when it comes to delivering consistent frame rates.

**The Everest Test**

In our first VR test, we chose to use the Everest demo as a tech demo, rather than an actual game. The demo featured high-quality visuals stitched together with photogrammetry, using more than 30,000 photos of the actual Mountain to build its environment. We tested both CPUs in this demo and found that they produced similar frame times, although the Intel Core i7-1700 had a slight edge.

**Conclusion**

So, what do our results tell us? In short, neither CPU has a significant advantage over the other when it comes to delivering smooth VR experiences. The differences we saw were largely imperceptible and may not be noticeable in real-world usage. However, this test does give us valuable insights into how CPUs perform in VR environments, which is essential for developers looking to optimize their games for these types of applications.

**Limitations and Future Plans**

There are several limitations to our first VR test that we must acknowledge. For example, the games we tested were not optimized specifically for VR, which may affect the results. Additionally, the headsets used in this test were locked at 90Hz, which may not reflect real-world usage scenarios.

Moving forward, we plan to conduct more tests with different games and settings to further understand CPU performance in VR. We'll also be testing other CPUs, such as the AMD Ryzen 5 5600X and Intel Core i5-11600K, to see how they perform in various environments.

**Join the Discussion**

We're always looking for feedback from our readers, so if you have any thoughts on this test or would like to suggest games for us to test next, please let us know. You can do this by visiting our Patreon page and contributing directly to our channel. Your support is essential in helping us continue to produce high-quality content for the gaming community.

**Get Your Gear Here**

Finally, if you're interested in buying any of the gear we used in this test, including the headsets, games, and CPUs, be sure to check out our affiliate links below. By supporting our channel through these links, you'll not only be helping us continue to produce content but also getting access to some amazing gaming gear.

* Patreonsus: Gamers Nexus

* Merchandise: Gamers Nexus Store

* Affiliate Links:

+ NVIDIA GeForce RTX 3080 Ti Hybrid Graphics Card

+ Oculus Rift S Headset

+ AMD Ryzen 7 7700X CPU

"WEBVTTKind: captionsLanguage: enour first in-depth foray into the VR benchmarking scene is focused on testing the AMD r7700 and the Intel i77700 K CPUs on both the HTC Vive and the Oculus Rift headsets we're plotting delivered frame times dropped and synthesized frames warp misses frame rate overall and plain back Hardware capture of the benchmarks in this set of tests VR benchmarking is still new and challenging to perform so we've limited the tests to five games and for configurations but even just running those 20 total configs take significantly longer than benchmarking an entire GPU suite and must be preceded with an explanation of VR testing before getting to that this coverage is brought to you by the 1080c which has a new MSRP that is lowered with the launch of the 1080 TI series The 1080 and 1060 SC cards come with for Honor or Ghost Recon wild lands which you can choose at checkout learn more at the link in the description below interpreting the data in these charts to today isn't necessarily obvious this is a different type of chart and the data has a different significance than our normal test with traditional benchmarking so part of this is going to be explaining what we're actually looking at the first game Benchmark that shows up on the screen will explain what some of the numbers are and the lines and the bars and all that and then going forward we'll be dumping a lot of the charts from the games following the first one which is dirt rally just because if we put every single chart in this video there would be 50 charts it would take an hour to go through them one minute at a time so instead we're going to put only the most pertinent data sets in this video the rest as always will be linked in the article in the description below which will have pretty much every chart you could possibly want including individual breakouts of the two CPUs of the two CPUs overclocked hardware and software monitoring and then the comparative analysis which is what we're mostly focusing on today in the video as an intro to VR benchmarking our setup is as follows we're using Hardware capture of the headsets to intercept the footage and send it to another machine this is done in a way that does not impact per performance of the benchmarking system and that's important since it's just splitting the data a splitter box sits between the capture machine and the gaming machine and that feeds into a $2,000 SC hd4 card from Vision which is capable of accepting the high bandwidth from VR headsets and also splitting things we then use Virtual dub to capture the playback in the headset and on the gaming machine and run color overlays that bake into the output from the FCAT VR Suite this can be later extracted to analyze delivered and dropped frames on a hardware level and note also that you need a very fast SSD or raid ssds in order to keep up with the data because one minute of testing can easily equal a 50 GB file we finally feed that file into our own compression script which is creating the files that you see in this video and those are compressed down to a level of hundreds of megabytes rather than tens of gigabytes on the gaming machine we use FCAT VR to intercept frame time delivered frame rate drop frames and warp Miss data of note for today frame times represent the time in milliseconds between each frame delivery just like always VR headsets range from roughly 11 to 13 milliseconds for the time required to deliver a frame before encountering some sort of drop frame or warp miss or other unpleasant action and the extra 2 milliseconds there at the end is really just an approximation because the rift can do some funny things with its run time to help lengthen the amount of time before the frame is required which is generally 11 milliseconds to hit that 90 refresh but there is some stretching room after that the captured files look something like what's on screen now at which point we feed these into a spreadsheet and into FCAT VR filter the data and output the data you'll see in the test later there are thousands of individual data points for each test which is one minute long and then the top row is several Vari variables La as well so there's plenty of data there to analyze and interpret the hard part is figuring out what to do with all that data once once we've captured it we have a previous video explaining drop frames and warp misses and what those are if you're curious about those two specific names as it pertains to VR testing and in that video we also talk about the VR pipelines that goes through this 11 millisecond window where do you hit the run time when do you need to have the frame ready and dispatched and things like that so that's in a previous content piece as for the benchmarking itself for today VR test execution is a big challenge so VR testing in general is brand new we've been working on this for a few months with FCAT VR early iterations but even with that experience it's not a perfect setup so a few things here to note first of all there's a major human element with VR testing that cannot really be easily resolved so it's not only do we have to as usually execute test passes which are fairly identical more or less from one pass to the next but we have to do that while in VR with a headset where you've got head movement uh so the level of accuracy diminishes compared to a standard Benchmark where you're just using a keyboard and mouse and that's something we acknowledge and basically uh use error bars in the graphs and the bar graphs to show a margin of error which right now is a bit wider than I want it to be we're working on sort of tightening the variance between tests but there's only so much we can do before you run into issues with just VR in general not being a a very friendly platform to Benchmark compared to normal benchmarking so another example of this outside of the human element there's also a randomized element with the games a lot of the VR games right now are King of the Hill style stand in the middle you attack enemies that spawn around you those enemies are normally randomly generated and depending on your performance one run to the next it could be that you see more enemies or fewer enemies or whatever than in the previous past that's another challenge now fortunately we have enough data from again the months of working on this to know what the variance is and what the margin of error is one pass to the next with the games and so we have that margin of error bar on the charts it's not too wide it's plus or minus 1.5% more or less but that will give you an idea of where the numbers fall or where they could fall at one past the next the gamees benchmarks for today include the Oculus Rift version of dirt rally Oculus Rift version of elite dangerous and then we have the HTC Vive running raw data Arizona sunshine and Everest if you're curious about the test platform used and the specs of the machine you can check the article Linked In the description below but the most important element is the GPU and that was our GTX 1080ti hybrid model that we built ourselves for the first test we're starting with the Oculus Rift and then moving on to the Vive our Rift games include again dirt and Elite we're testing dirt rally configured the high settings with the advanced blending option enabled this first set of charts will contain all the data we have while subsequent games will only contain head-to-head data so they will be simplified and faster to get through for all of it if you want it for all the games check the article below let's start with the R7 1700 CPU at stock settings only the left axis on this chart shows frame times and milliseconds lower is better here and we have a rough 11 millisecond window to deliver the frame and the rift sometimes it can go up a bit maybe around 13 as your max the magenta line represents the hardware capture while the red line represents the software capture the hardware capture cannot see what's going on at a software level and so only validates findings by illustrating dropped frames never delivered to the headset the software line is more of what we're interested in the lower third of the chart meanwhile is an interval plot this one helps us visualize delivered synthesized frames dropped frames and delivered new frames with everything explained now we can start the data analysis for this chart with the 1700 stock we can see that the 1700 encounters a few dropped frames or frames that were synthesized by updating head tracking and position without a full update to the scene right now we're playing video playback of the hardware capture for this run the experience overall as seen in the playback is smooth and the dropped and synthetic frames go unnoticed during use we don't have enough to detect as a human here that we can detect it with tools and we'll talk about this more going forward because again as a reminder if you're at 60 seconds for a test 90 Hertz that means you have 5,400 intervals so a couple drop frames is not a big deal we'll show average frame times and unconstrained FPS at the end of this charted section next the r7700 overclocked shows mostly the same performance with one pretty bad drop at the end of the capture but overall nothing too critical the time to deliver frames is now shorter with the 3.9 GHz overclock but there's no major difference in user experience this next chart shows the 1700 stock and overclocked at the same time where we can observe that the red line representing the overclocked values is consistently faster in frame delivery than the yellow line or the stock 1700 we are generally around 10 milliseconds for both devices but we'll show that value more explicitly in a moment and now here's Intel's i77700 kcpu with stock settings Intel CPU has a few drops but just like the R7 chip these are not appreciable to the end user frame times are closer to the 8 to 9 millisecond Mark with the hardware frame time spikes validating the software measurements exactly at the same moment we're playing some gameplay of the Intel i7 7700k Benchmark now where you'll notice that user experience is smooth and without any significant or noticeable hitches both CPUs can deliver a smooth experience which is subjectively to the human eye the same but let's look at how they compare objectively here's a chart showing the 1700 and 7700k head-to-head in stock frequencies because we have standardized this Benchmark with the same head movements we can see that the frame time spikes generally align between both CPUs Intel is faster overall in frame time delivery in this test with each CPU dropping fewer than 20 frames throughout the entire run it's not until drop frames are significantly greater on one config than the other that we'd actually noticed them so for all intents and purposes The Experience on each CPU is the same that said Intel is faster and its frame times on this particular title and experiences shorter spikes when the going gets tough but let's look at a few more charts first one more frame time chart and then we'll look at the bar graphs this one shows the two CPUs overclocked with Intel at 4.9 and AMD at 3.9 GHz both overclocks are achievable on the majority of the respective chips the drop frames are similar again with the experience again being effectively equivalent AMD is closer to the 11 to 13 millisecond cutof window but still within bounds and so the experience is the same interestingly Intel loses ground and overclocking compared to the stock benchmarks and we'll see that this trend repeats throughout the tests let's get a bar graph on the screen for better illustration this chart plots delivered fps to the headset which is the most important metric then drop frames as the second most important metric and then unconstrained FPS as a calculation unconstrained FPS is an imperfect prediction of how many frames would have been delivered per second if the hmd did not have a fixed update interval of 90 HZ since they are fixed at 90 HZ really the most important item is delivered FPS the unconstrained value is calculated by taking 1,000 milliseconds and dividing it by the average frame time which is done in the new FCAT VR tool automatically the two hard metrics are again delivered FPS which we can validate with an effectively infallible Hardware capture setup and drop frames also validated by Hardware capture drout frames are an absolute measure in total frame count over the test period which is 60 seconds at 90 htz a 60c pass will produce 5,400 refresh intervals on the headset final note we currently have a test variance of roughly plus or minus 1.5% in this chart for dirt rally we immediately see that both the 1700 and 7700 K are capable of delivering 90 fps to the headset that's what we want and we next see that the drop frames have a range of eight going from 3 to 11 drop frames per test pass as we saw in our our previous charts the drop frames are not clustered tightly enough to be noticeable by the user in the absolute worst case the r7700 encounters 11 drop frames over its 5400 refresh intervals to put that into perspective that yields a 0.2% drop frames for the test period a user would not notice this particularly when the drop frames are spaced out over the entire period we next see that unconstrained FPS lands around 135 to 137 for the 7700 K while the r7700 is in the 105 to 115 FPS range again this is an extrapolation we're still seeing 90 on either device through the hmd and let's now move to the average frame time chart here we see where that number is calculated this chart's scale is set to 12 M seconds at which point you'll probably start encountering drops warp misses or reprojection issues the 7700k stock and overclocked CPUs perform effectively equally at 7.3 to 7.4 milliseconds the r7700 shows a bigger gain from the the overclock just outside tolerances landing at 8.7 milliseconds from 9.54 milliseconds comparatively the 7700k stock experiences a 22% reduction in average frame time over the 1700 stock and about a 14.8% reduction versus the overclocked r7700 let's move on to the next game finally the next tested game is Elite dangerous which has Hardware capture footage on the screen now we're going to simplify the frame time charts here and only show comparative data if you want all of the charts check the article linked below Elite has some issues with the Vive so we're using the rift this was configured with VR High settings in the game and played in the VR training level this first chart shows the 1700 versus the 7700 K both had stock frequencies the r7700 tends to run slower in average frame time delivery sometimes running against the limit before we start encountering the Rifts 11 to 13 millisecond refresh interval again note that Oculus does some things in run time to stretch that 11 millisecond refresh out a little bit hence the extra buffer at the end there as Illustrated in the interval plot at the bottom the r7700 and the i77700 K are dropping and synthesizing a similar amount of frames you can see that with the green yellow and red colors on the lower two charts neither of these is appreciably worse than the other as a user again this experience is equal within the confines of human perception and with what the headsets allow that said the difference is statistically significant you could make an argument that the extra Headroom is valuable but we have not yet found a scenario where we begin encountering noticeable jarring or stuttering on either the 1700 or the 7700 K this chart shows that overclocking the r7700 tends to close the gap versus Intel which is also now overclocked on the frame time and interval plot graph that's the same as we saw in dirt rally it appears that overclocks don't benefit Intel quite as much as they're benefiting AMD here likely because of the r7's lower starting frequency we're seeing a range of six for the drop frames from 11 to 17 in the best and worst cases and the i7 7700 K OC and 7700k stock perform equally when looking at the bar graphs that's within variance and further illustrates that VR benchmarking is not yet repeatable enough to analyze with tight margins and the hard statements as to what data we're seeing even in the worst case of the 17 drop frames we're still at 0.3% of all frames delivered as dropped whereas 11 drop frames would be 0.2% completely imperceptible to the user the dropped frames are also dispersed enough to not matter with regard to actual delivered frames we're at 90 FPS for all four tests Intel holds a lead in unconstrained FPS as it has shorter frame latencies overall as shown in the next chart that we're on now for average frame times we're at roughly 8.6 millisecs between the two Intel tests and at 9.9 and 10.4 for the AMD tests 10.4 is starting to push the limmit of what we're comfortable with but still delivers a smooth experience in this game the overclock Keeps Us reasonably distance from the 11 to 13 millisecond Mark and again there's no perceptible difference between these SKS given that they all hit 90 FPS without any significant drop frame counts that are noticeable on the screen now our test passes for the next game raw data we're moving on to the HTC Vive using raw data as a VR specific title that's shown some promise this is another Cane of the Hill title as you can see from the footage we've got on the screen screen and these first results are with the game configured to high settings with zero mrss this Frame time plot shows the 1700 versus the 7700 K both stock neither CPU struggles here with the 1700 generally being faster in frame delivery that said both are below the 6 millisecond line on average and the interval plot below shows that intal encounter zero dropped or synthesized frames MD encounters a few but they are not significant enough to be perceived by the wearer this seems to be a trend between these two CPUs cuz because they're both pretty good at what they're doing here these two for all intents and purposes are again effectively equivalent in this game here's the overclock data same story here Intel drops no frames but tends to run average frame times a bit over the 1700 let's just move straight to the bar graphs delivered frame rate post 90 FPS for all devices with AMD dropping six frames in each pass and Intel dropping zero this is again not really noticeable but it's worth plotting because we can measure it Intel CPU is again showing limited gains from over clocking with both line items performing effectively identically in unconstrained frame rate and that's both 7700 K the same is true for AMD this time where overclocking doesn't really change the fact that we're around 229 on constrained FPS looking at average frame times we see that the 1700 CPUs stick around 4.36 milliseconds with Intel around 4.96 milliseconds looking back to the video capture of the game which we can play on the screen we see that both of these are so distant from an 11 millisecond refresh interval as to be effectively equivalent in visual output you could buy either CPU and experience and identical experience in the headset for this raw data pass we did on high based on our initial testing more charts in the article if you want to see the rest of raw data's data we're now testing using Arizona Sunshine which is another King of the Hill title on the Vive and the rift we're using the Vive here and testing with Advanced CPU features enabled and very high texture quality as you've seen while introducing this game is a zombie shooter game and this test is conducted during the first swarm wave here's the first chart both CPUs perform at an equal frame rate particularly when considering the certainty of test variant in this game the r7700 encounters more synth and drop frames this time but we've still not encountered a warp Miss which is important we have not seen a single warp Miss on any of these tests without any warp misses between these drop frames or without heavier drop frame saturation in the interval plot it remains fair to say that the experience between the two CPUs is really not that much different visually objectively Intel does have fewer drop frames in this title we're just facing a question of when does that become noticeable to the end user and since VR testing is still new it's kind of hard to say we need more experience in the game to really make a definitive statement on that overclocking helps the r7700 as seen here and posts an effective zero change for the 7700k again and then finally let's move on to the next chart here's the FPS output the r7700 drops 100 frames over 5400 intervals or 1.85% that's the most out of everything we've seen so far but still not a big deal and that's reflect Ed in the delivered frame bar as well where we're seeing 87.5 FPS average versus 90 FPS delivered frames to the headset the overclock gets us down to 38 drops frames but the gap between 257 FPS and 262 FPS on the R7 CPUs shows that again this test is imperfect this is precisely why we added those error bars 87 versus 89 FPS is also imperceptibly different and the unconstrained frame rate does not post a statistically significant difference either though objectively Intel is better in this title at least with regard to drop frames very quickly here's the average frame time chart that supports that statement again no statistical significance in the differences between frame times with this test configuration they're basically all the same finally Everest is more of a tech demo than the game that's the one we're showing now it's got some highquality visuals that are stitched together with photogrametry last we spoke to the Everest devs the demo used more than 30,000 photos of the actual Mountain to build its environment we're using kumu icefall for the test course with settings configured so that the bars are two ticks down from Mac settings and so that they're all equal length in the graphics menu which is really not a great graphics menu here's the frame time chart between the 1700 and 7700k stock it's also the most boring chart we've looked at thus far the interval plot shows really no activity other than mostly successful frame deliveries frame times look about the same looking at the FPS chart quickly we see the same thing these numbers are more or less equal average frame time chart finally blasting through these reinforces that everything is running around 6 milliseconds and two CPUs produce an equal experience in this game with neither superior to the other for the most part there's no real significant appreciable difference between these two CPUs in these games and normally that would be kind of a boring thing but because VR testing is so new this data is good to have because we can actually start building an understanding as a community as to what is a good experience in VR and as we move toward testing things like r5s and i5s well probably start seeing more drop frames and limitations in VR with CPUs which of course VR being such a high resolution thing is almost natively a GPU bottleneck but by controlling the settings in the games and lowering them down to things like high with a 1080ti hybrid we're able to eliminate a good amount of that and still show some of the CPU limitations it's just that the r s and the i7 do fine in this they don't really have an issue in general for the two Rift games we tested Intel tended to do better but it was not in a way that really mattered ultimately and the same was true when AMD did better in the one or two games where it posted Superior frame times whether that was a big or not swing uh the difference to a user was the same so it's an interesting challenge to test these things the headsets are locked to 90 Herz your frame rate's locked to 90 FPS and unconstrained metrics are really interesting and cool to have be but it's more for GPU testing where you might have six gpus on the chart seeing unconstrained FPS in theory helps tell you what the GPU is capable of in a more familiar metric beyond the 90fps limit which is effectively a locked vsync setting still it's an imperfect metric because it's trying to extrapolate something that you're not going to see now unconstrained FPS does kind of tell you hey this GPU or this CPU can prepare more frames for delivery So in theory going forward it might indicate that that particular product performs better but but it's really too early to say if that's how it's going to play out we'll see so overall first VR test let us know what you think we have some more plans probably for the R5 and maybe the I5 as always you can go to patreon.com gamersus help us out directly with this in-depth testing thank you for watching subscribe for more Gamers nexus.net for the article version of this or if you're a fan store. Gamers nexus.net for the shirts I guess the natural progression of a YouTube channel is that the end slowly widens as you add more and more things to plug so that's it for this one thanks for watching I'll see you all next timeour first in-depth foray into the VR benchmarking scene is focused on testing the AMD r7700 and the Intel i77700 K CPUs on both the HTC Vive and the Oculus Rift headsets we're plotting delivered frame times dropped and synthesized frames warp misses frame rate overall and plain back Hardware capture of the benchmarks in this set of tests VR benchmarking is still new and challenging to perform so we've limited the tests to five games and for configurations but even just running those 20 total configs take significantly longer than benchmarking an entire GPU suite and must be preceded with an explanation of VR testing before getting to that this coverage is brought to you by the 1080c which has a new MSRP that is lowered with the launch of the 1080 TI series The 1080 and 1060 SC cards come with for Honor or Ghost Recon wild lands which you can choose at checkout learn more at the link in the description below interpreting the data in these charts to today isn't necessarily obvious this is a different type of chart and the data has a different significance than our normal test with traditional benchmarking so part of this is going to be explaining what we're actually looking at the first game Benchmark that shows up on the screen will explain what some of the numbers are and the lines and the bars and all that and then going forward we'll be dumping a lot of the charts from the games following the first one which is dirt rally just because if we put every single chart in this video there would be 50 charts it would take an hour to go through them one minute at a time so instead we're going to put only the most pertinent data sets in this video the rest as always will be linked in the article in the description below which will have pretty much every chart you could possibly want including individual breakouts of the two CPUs of the two CPUs overclocked hardware and software monitoring and then the comparative analysis which is what we're mostly focusing on today in the video as an intro to VR benchmarking our setup is as follows we're using Hardware capture of the headsets to intercept the footage and send it to another machine this is done in a way that does not impact per performance of the benchmarking system and that's important since it's just splitting the data a splitter box sits between the capture machine and the gaming machine and that feeds into a $2,000 SC hd4 card from Vision which is capable of accepting the high bandwidth from VR headsets and also splitting things we then use Virtual dub to capture the playback in the headset and on the gaming machine and run color overlays that bake into the output from the FCAT VR Suite this can be later extracted to analyze delivered and dropped frames on a hardware level and note also that you need a very fast SSD or raid ssds in order to keep up with the data because one minute of testing can easily equal a 50 GB file we finally feed that file into our own compression script which is creating the files that you see in this video and those are compressed down to a level of hundreds of megabytes rather than tens of gigabytes on the gaming machine we use FCAT VR to intercept frame time delivered frame rate drop frames and warp Miss data of note for today frame times represent the time in milliseconds between each frame delivery just like always VR headsets range from roughly 11 to 13 milliseconds for the time required to deliver a frame before encountering some sort of drop frame or warp miss or other unpleasant action and the extra 2 milliseconds there at the end is really just an approximation because the rift can do some funny things with its run time to help lengthen the amount of time before the frame is required which is generally 11 milliseconds to hit that 90 refresh but there is some stretching room after that the captured files look something like what's on screen now at which point we feed these into a spreadsheet and into FCAT VR filter the data and output the data you'll see in the test later there are thousands of individual data points for each test which is one minute long and then the top row is several Vari variables La as well so there's plenty of data there to analyze and interpret the hard part is figuring out what to do with all that data once once we've captured it we have a previous video explaining drop frames and warp misses and what those are if you're curious about those two specific names as it pertains to VR testing and in that video we also talk about the VR pipelines that goes through this 11 millisecond window where do you hit the run time when do you need to have the frame ready and dispatched and things like that so that's in a previous content piece as for the benchmarking itself for today VR test execution is a big challenge so VR testing in general is brand new we've been working on this for a few months with FCAT VR early iterations but even with that experience it's not a perfect setup so a few things here to note first of all there's a major human element with VR testing that cannot really be easily resolved so it's not only do we have to as usually execute test passes which are fairly identical more or less from one pass to the next but we have to do that while in VR with a headset where you've got head movement uh so the level of accuracy diminishes compared to a standard Benchmark where you're just using a keyboard and mouse and that's something we acknowledge and basically uh use error bars in the graphs and the bar graphs to show a margin of error which right now is a bit wider than I want it to be we're working on sort of tightening the variance between tests but there's only so much we can do before you run into issues with just VR in general not being a a very friendly platform to Benchmark compared to normal benchmarking so another example of this outside of the human element there's also a randomized element with the games a lot of the VR games right now are King of the Hill style stand in the middle you attack enemies that spawn around you those enemies are normally randomly generated and depending on your performance one run to the next it could be that you see more enemies or fewer enemies or whatever than in the previous past that's another challenge now fortunately we have enough data from again the months of working on this to know what the variance is and what the margin of error is one pass to the next with the games and so we have that margin of error bar on the charts it's not too wide it's plus or minus 1.5% more or less but that will give you an idea of where the numbers fall or where they could fall at one past the next the gamees benchmarks for today include the Oculus Rift version of dirt rally Oculus Rift version of elite dangerous and then we have the HTC Vive running raw data Arizona sunshine and Everest if you're curious about the test platform used and the specs of the machine you can check the article Linked In the description below but the most important element is the GPU and that was our GTX 1080ti hybrid model that we built ourselves for the first test we're starting with the Oculus Rift and then moving on to the Vive our Rift games include again dirt and Elite we're testing dirt rally configured the high settings with the advanced blending option enabled this first set of charts will contain all the data we have while subsequent games will only contain head-to-head data so they will be simplified and faster to get through for all of it if you want it for all the games check the article below let's start with the R7 1700 CPU at stock settings only the left axis on this chart shows frame times and milliseconds lower is better here and we have a rough 11 millisecond window to deliver the frame and the rift sometimes it can go up a bit maybe around 13 as your max the magenta line represents the hardware capture while the red line represents the software capture the hardware capture cannot see what's going on at a software level and so only validates findings by illustrating dropped frames never delivered to the headset the software line is more of what we're interested in the lower third of the chart meanwhile is an interval plot this one helps us visualize delivered synthesized frames dropped frames and delivered new frames with everything explained now we can start the data analysis for this chart with the 1700 stock we can see that the 1700 encounters a few dropped frames or frames that were synthesized by updating head tracking and position without a full update to the scene right now we're playing video playback of the hardware capture for this run the experience overall as seen in the playback is smooth and the dropped and synthetic frames go unnoticed during use we don't have enough to detect as a human here that we can detect it with tools and we'll talk about this more going forward because again as a reminder if you're at 60 seconds for a test 90 Hertz that means you have 5,400 intervals so a couple drop frames is not a big deal we'll show average frame times and unconstrained FPS at the end of this charted section next the r7700 overclocked shows mostly the same performance with one pretty bad drop at the end of the capture but overall nothing too critical the time to deliver frames is now shorter with the 3.9 GHz overclock but there's no major difference in user experience this next chart shows the 1700 stock and overclocked at the same time where we can observe that the red line representing the overclocked values is consistently faster in frame delivery than the yellow line or the stock 1700 we are generally around 10 milliseconds for both devices but we'll show that value more explicitly in a moment and now here's Intel's i77700 kcpu with stock settings Intel CPU has a few drops but just like the R7 chip these are not appreciable to the end user frame times are closer to the 8 to 9 millisecond Mark with the hardware frame time spikes validating the software measurements exactly at the same moment we're playing some gameplay of the Intel i7 7700k Benchmark now where you'll notice that user experience is smooth and without any significant or noticeable hitches both CPUs can deliver a smooth experience which is subjectively to the human eye the same but let's look at how they compare objectively here's a chart showing the 1700 and 7700k head-to-head in stock frequencies because we have standardized this Benchmark with the same head movements we can see that the frame time spikes generally align between both CPUs Intel is faster overall in frame time delivery in this test with each CPU dropping fewer than 20 frames throughout the entire run it's not until drop frames are significantly greater on one config than the other that we'd actually noticed them so for all intents and purposes The Experience on each CPU is the same that said Intel is faster and its frame times on this particular title and experiences shorter spikes when the going gets tough but let's look at a few more charts first one more frame time chart and then we'll look at the bar graphs this one shows the two CPUs overclocked with Intel at 4.9 and AMD at 3.9 GHz both overclocks are achievable on the majority of the respective chips the drop frames are similar again with the experience again being effectively equivalent AMD is closer to the 11 to 13 millisecond cutof window but still within bounds and so the experience is the same interestingly Intel loses ground and overclocking compared to the stock benchmarks and we'll see that this trend repeats throughout the tests let's get a bar graph on the screen for better illustration this chart plots delivered fps to the headset which is the most important metric then drop frames as the second most important metric and then unconstrained FPS as a calculation unconstrained FPS is an imperfect prediction of how many frames would have been delivered per second if the hmd did not have a fixed update interval of 90 HZ since they are fixed at 90 HZ really the most important item is delivered FPS the unconstrained value is calculated by taking 1,000 milliseconds and dividing it by the average frame time which is done in the new FCAT VR tool automatically the two hard metrics are again delivered FPS which we can validate with an effectively infallible Hardware capture setup and drop frames also validated by Hardware capture drout frames are an absolute measure in total frame count over the test period which is 60 seconds at 90 htz a 60c pass will produce 5,400 refresh intervals on the headset final note we currently have a test variance of roughly plus or minus 1.5% in this chart for dirt rally we immediately see that both the 1700 and 7700 K are capable of delivering 90 fps to the headset that's what we want and we next see that the drop frames have a range of eight going from 3 to 11 drop frames per test pass as we saw in our our previous charts the drop frames are not clustered tightly enough to be noticeable by the user in the absolute worst case the r7700 encounters 11 drop frames over its 5400 refresh intervals to put that into perspective that yields a 0.2% drop frames for the test period a user would not notice this particularly when the drop frames are spaced out over the entire period we next see that unconstrained FPS lands around 135 to 137 for the 7700 K while the r7700 is in the 105 to 115 FPS range again this is an extrapolation we're still seeing 90 on either device through the hmd and let's now move to the average frame time chart here we see where that number is calculated this chart's scale is set to 12 M seconds at which point you'll probably start encountering drops warp misses or reprojection issues the 7700k stock and overclocked CPUs perform effectively equally at 7.3 to 7.4 milliseconds the r7700 shows a bigger gain from the the overclock just outside tolerances landing at 8.7 milliseconds from 9.54 milliseconds comparatively the 7700k stock experiences a 22% reduction in average frame time over the 1700 stock and about a 14.8% reduction versus the overclocked r7700 let's move on to the next game finally the next tested game is Elite dangerous which has Hardware capture footage on the screen now we're going to simplify the frame time charts here and only show comparative data if you want all of the charts check the article linked below Elite has some issues with the Vive so we're using the rift this was configured with VR High settings in the game and played in the VR training level this first chart shows the 1700 versus the 7700 K both had stock frequencies the r7700 tends to run slower in average frame time delivery sometimes running against the limit before we start encountering the Rifts 11 to 13 millisecond refresh interval again note that Oculus does some things in run time to stretch that 11 millisecond refresh out a little bit hence the extra buffer at the end there as Illustrated in the interval plot at the bottom the r7700 and the i77700 K are dropping and synthesizing a similar amount of frames you can see that with the green yellow and red colors on the lower two charts neither of these is appreciably worse than the other as a user again this experience is equal within the confines of human perception and with what the headsets allow that said the difference is statistically significant you could make an argument that the extra Headroom is valuable but we have not yet found a scenario where we begin encountering noticeable jarring or stuttering on either the 1700 or the 7700 K this chart shows that overclocking the r7700 tends to close the gap versus Intel which is also now overclocked on the frame time and interval plot graph that's the same as we saw in dirt rally it appears that overclocks don't benefit Intel quite as much as they're benefiting AMD here likely because of the r7's lower starting frequency we're seeing a range of six for the drop frames from 11 to 17 in the best and worst cases and the i7 7700 K OC and 7700k stock perform equally when looking at the bar graphs that's within variance and further illustrates that VR benchmarking is not yet repeatable enough to analyze with tight margins and the hard statements as to what data we're seeing even in the worst case of the 17 drop frames we're still at 0.3% of all frames delivered as dropped whereas 11 drop frames would be 0.2% completely imperceptible to the user the dropped frames are also dispersed enough to not matter with regard to actual delivered frames we're at 90 FPS for all four tests Intel holds a lead in unconstrained FPS as it has shorter frame latencies overall as shown in the next chart that we're on now for average frame times we're at roughly 8.6 millisecs between the two Intel tests and at 9.9 and 10.4 for the AMD tests 10.4 is starting to push the limmit of what we're comfortable with but still delivers a smooth experience in this game the overclock Keeps Us reasonably distance from the 11 to 13 millisecond Mark and again there's no perceptible difference between these SKS given that they all hit 90 FPS without any significant drop frame counts that are noticeable on the screen now our test passes for the next game raw data we're moving on to the HTC Vive using raw data as a VR specific title that's shown some promise this is another Cane of the Hill title as you can see from the footage we've got on the screen screen and these first results are with the game configured to high settings with zero mrss this Frame time plot shows the 1700 versus the 7700 K both stock neither CPU struggles here with the 1700 generally being faster in frame delivery that said both are below the 6 millisecond line on average and the interval plot below shows that intal encounter zero dropped or synthesized frames MD encounters a few but they are not significant enough to be perceived by the wearer this seems to be a trend between these two CPUs cuz because they're both pretty good at what they're doing here these two for all intents and purposes are again effectively equivalent in this game here's the overclock data same story here Intel drops no frames but tends to run average frame times a bit over the 1700 let's just move straight to the bar graphs delivered frame rate post 90 FPS for all devices with AMD dropping six frames in each pass and Intel dropping zero this is again not really noticeable but it's worth plotting because we can measure it Intel CPU is again showing limited gains from over clocking with both line items performing effectively identically in unconstrained frame rate and that's both 7700 K the same is true for AMD this time where overclocking doesn't really change the fact that we're around 229 on constrained FPS looking at average frame times we see that the 1700 CPUs stick around 4.36 milliseconds with Intel around 4.96 milliseconds looking back to the video capture of the game which we can play on the screen we see that both of these are so distant from an 11 millisecond refresh interval as to be effectively equivalent in visual output you could buy either CPU and experience and identical experience in the headset for this raw data pass we did on high based on our initial testing more charts in the article if you want to see the rest of raw data's data we're now testing using Arizona Sunshine which is another King of the Hill title on the Vive and the rift we're using the Vive here and testing with Advanced CPU features enabled and very high texture quality as you've seen while introducing this game is a zombie shooter game and this test is conducted during the first swarm wave here's the first chart both CPUs perform at an equal frame rate particularly when considering the certainty of test variant in this game the r7700 encounters more synth and drop frames this time but we've still not encountered a warp Miss which is important we have not seen a single warp Miss on any of these tests without any warp misses between these drop frames or without heavier drop frame saturation in the interval plot it remains fair to say that the experience between the two CPUs is really not that much different visually objectively Intel does have fewer drop frames in this title we're just facing a question of when does that become noticeable to the end user and since VR testing is still new it's kind of hard to say we need more experience in the game to really make a definitive statement on that overclocking helps the r7700 as seen here and posts an effective zero change for the 7700k again and then finally let's move on to the next chart here's the FPS output the r7700 drops 100 frames over 5400 intervals or 1.85% that's the most out of everything we've seen so far but still not a big deal and that's reflect Ed in the delivered frame bar as well where we're seeing 87.5 FPS average versus 90 FPS delivered frames to the headset the overclock gets us down to 38 drops frames but the gap between 257 FPS and 262 FPS on the R7 CPUs shows that again this test is imperfect this is precisely why we added those error bars 87 versus 89 FPS is also imperceptibly different and the unconstrained frame rate does not post a statistically significant difference either though objectively Intel is better in this title at least with regard to drop frames very quickly here's the average frame time chart that supports that statement again no statistical significance in the differences between frame times with this test configuration they're basically all the same finally Everest is more of a tech demo than the game that's the one we're showing now it's got some highquality visuals that are stitched together with photogrametry last we spoke to the Everest devs the demo used more than 30,000 photos of the actual Mountain to build its environment we're using kumu icefall for the test course with settings configured so that the bars are two ticks down from Mac settings and so that they're all equal length in the graphics menu which is really not a great graphics menu here's the frame time chart between the 1700 and 7700k stock it's also the most boring chart we've looked at thus far the interval plot shows really no activity other than mostly successful frame deliveries frame times look about the same looking at the FPS chart quickly we see the same thing these numbers are more or less equal average frame time chart finally blasting through these reinforces that everything is running around 6 milliseconds and two CPUs produce an equal experience in this game with neither superior to the other for the most part there's no real significant appreciable difference between these two CPUs in these games and normally that would be kind of a boring thing but because VR testing is so new this data is good to have because we can actually start building an understanding as a community as to what is a good experience in VR and as we move toward testing things like r5s and i5s well probably start seeing more drop frames and limitations in VR with CPUs which of course VR being such a high resolution thing is almost natively a GPU bottleneck but by controlling the settings in the games and lowering them down to things like high with a 1080ti hybrid we're able to eliminate a good amount of that and still show some of the CPU limitations it's just that the r s and the i7 do fine in this they don't really have an issue in general for the two Rift games we tested Intel tended to do better but it was not in a way that really mattered ultimately and the same was true when AMD did better in the one or two games where it posted Superior frame times whether that was a big or not swing uh the difference to a user was the same so it's an interesting challenge to test these things the headsets are locked to 90 Herz your frame rate's locked to 90 FPS and unconstrained metrics are really interesting and cool to have be but it's more for GPU testing where you might have six gpus on the chart seeing unconstrained FPS in theory helps tell you what the GPU is capable of in a more familiar metric beyond the 90fps limit which is effectively a locked vsync setting still it's an imperfect metric because it's trying to extrapolate something that you're not going to see now unconstrained FPS does kind of tell you hey this GPU or this CPU can prepare more frames for delivery So in theory going forward it might indicate that that particular product performs better but but it's really too early to say if that's how it's going to play out we'll see so overall first VR test let us know what you think we have some more plans probably for the R5 and maybe the I5 as always you can go to patreon.com gamersus help us out directly with this in-depth testing thank you for watching subscribe for more Gamers nexus.net for the article version of this or if you're a fan store. Gamers nexus.net for the shirts I guess the natural progression of a YouTube channel is that the end slowly widens as you add more and more things to plug so that's it for this one thanks for watching I'll see you all next time\n"