The Importance of Data Science in Game Development: A Conversation with Marie

It's interesting to note that when approaching a Lakino studio, it's often not just one game in development, but multiple games being worked on simultaneously. This means that what we learn from one game can be applied to others, making data science an invaluable tool for studios specializing in specific types of games.

Data science has the potential to revolutionize the gaming industry, especially when it comes to generating assets and audio assets in Unreal Engine. For example, AI-powered tools can automate tasks such as creating textures or models, freeing up developers to focus on more creative aspects of game design. Additionally, data science can be used to create realistic NPC behavior, enemy dialogue, and even entire storylines.

The use of generative AI in gaming is a rapidly evolving field, with exciting possibilities for studios looking to improve their games through data-driven decision-making. One potential application of generative AI is in the creation of high-quality dialogues for NPCs or enemies. This can be especially useful for indie developers who may not have the resources to hire human writers or voice actors.

GPT-3 and other AI models are already being used in game development, with some studios experimenting with their use in creating custom assets or gameplay mechanics. For example, GPT-3 has been used to generate text-based dialogue for games, allowing developers to create more realistic conversations between characters. This technology has the potential to greatly enhance the gaming experience, making it feel more immersive and engaging.

The future of gaming looks bright when it comes to data science and AI-powered tools. With studios like Lakino pushing the boundaries of what is possible with these technologies, we can expect to see even more innovative applications in the coming years. One potential benefit of this technology is that it could reduce development time for indie developers, allowing them to create high-quality games on a smaller budget.

Finally, Marie mentioned that one of the benefits of generative AI is that it can provide developers with access to tools and resources they might not have otherwise. For example, she was able to generate code snippets in just a few seconds using an AI model, saving her time and effort compared to searching online or consulting documentation. As data science continues to evolve, we can expect to see even more exciting applications of these technologies in the gaming industry.

In conclusion, the intersection of data science and game development is an exciting area of research and application. With the continued advancement of AI models like GPT-3, studios are beginning to explore new possibilities for creating engaging and immersive games. From generating high-quality assets to creating realistic NPC behavior, data science has the potential to revolutionize the gaming industry, making it feel more vibrant and engaging than ever before.

As we wrap up our conversation, Marie shared her final thoughts on the future of game development and data science. According to her, we are just beginning to scratch the surface of what is possible with these technologies, and there is still much work to be done in terms of changing developer habits and providing resources for implementing AI-powered tools.

Despite the potential benefits, Marie noted that there is still a need for education and training programs to help developers understand how to effectively use data science and AI models in their games. With the rapid pace of innovation in this field, it's essential that studios and developers stay up-to-date with the latest developments and best practices.

Ultimately, the future of gaming looks bright when it comes to data science and AI-powered tools. As we continue to explore new possibilities for using these technologies, we can expect to see even more innovative applications in the coming years. One thing is certain: the intersection of data science and game development is an exciting area that will have a lasting impact on the gaming industry.

"WEBVTTKind: captionsLanguage: enwe are living very exciting time because we are just beginning to Scotch the surface of everything that could be done with data science in games and this is still a very new you know um kind of um adventure for a lot of studios and I feel like with Chad GPT 4 and other you know improvements in AI there is so much potential so many things that could potentially be done to to make it better for everyone it's great to have you on the show nice to meet you I'm very happy to be there okay so you've been at the intersection of data science and gaming uh for more than 10 years now so maybe to set the stage for our conversation you know let's say I'm playing Guardians of the Galaxy on my PS5 what role did data science or is play or data science is playing in making sure that I'm having a great experience okay so of course uh I can't you know reveal anything that would be uh NTA related but um on a general basis what is going to be happening is the data that is being generated by players when they are playing uh is being collected three black launch and after launch um to help developers actually understand how people are playing the game and um if what they added mind uh is actually what is happening in the game um so there is a lot of different ways um data science can be used to make sure this happen you can use data sense to optimize the system of the game for instance you can use AI to test the game automatically to see if there is any kind of issues and of course in my team also data analyst and they were working with data scientists to provide you know any kind of like uh reporting dashboard to better understand observing in terms of behaviors so that's really great you know in a lot of ways there's a lot of different use cases or areas where data science provides a lot of value in gaming you'll add quite a few different teams data teams and gaming maybe walk through the different types of areas within a game where data science and machine learning play a lot of value and play a big role in delivering value what is very interesting about data science is our facilities its Encompass so many different applications and in my previous job we actually had like an AI team we also had people inside of the development team working on those kind of things um and we had like our machine learning people in my team working on different things related to business so in the game you there is we can just discuss for instance the AI that are actually being developed to help you know a player having fun by engaging with NPCs that are when defined but also being able to defeat enemies that are an actual challenge um so there is more and more data being used during developments to be able to you know achieve this kind of of course and there is also optimizing you know anything like the sound of the game um or the graphics how to use machine learning and data cells to actually help um you know in this kind of challenges um and we've been using uh also on our side data science to determine how people are actually reacting to everything it was related to the game so reviews and people like journalists are publishing different video reviews we were able to extract information from those and say okay so this is what people are actually talking about and this is the sentiment that she is actually uh you know um Associated to them so uh what kind of things could you do based on those kind of things so there is so many possible uh you know application um and different people being uh you know specialized in those specific areas that's very cool and you know I as a gamer myself it's very nice to see you know the applications of data science across the chain of gaming from you know development to the actual AI behind the NPCs and even after release because the community aspect of gaming is such a huge important element that makes for breaks the success of a game so understanding the pulse of the fans what they want Etc is is really important so I want to you know maybe pause out a few use cases that you talked here about let's maybe discuss you know early in development right I'm very keen on discussing what goes into AI development for an NPC but first let's talk about development uh you know there's a lot of data science use cases that go into QA testing into making sure that the game is running smoothly it releases bug free we've seen a lot of disasters in the industry for example where you know games were released prematurely with a lot of bugs we've seen a lot more attention nowadays being being applied towards QA testing for games maybe walk us through an example of how that looks like in action what is a process of setting up a data driven QA process okay so this particular aspect of things we were actually working with the AIT um Studio to develop those basically agents so it was that my team but we were working with them to be able to provide you know the dashboard and everything but the idea is you can like basically um you know set up um Teach an agent to do different kind of actions or to learn from its environment and find stuff by himself and then you can deploy them in a version like of the game or like any kind of place you would like to to improve um so they would be able to just play and they will generate data basically and using this data you will be able to determine all it looks like um this seems to be a problem so for instance if you see your agents are always falling in the same you know uh whole again and again okay maybe there is a problem and we should just get rid of this because it's very confusing and it's it's a problem and the idea is you can generate so many of them and they can do almost anything in the game so it's easier for them to find very specific you know issues that would be difficult to find for a given um and so and you can make them work you know during the nights uh when you're basically sleeping um so there is a lot of um very interesting very interesting uh things that could be done uh to improve the game based on this kind of of Agents just playing it and fighting you know Edge case and um of providing us with over the data they are generating what they play so we can analyze that after that and say okay so this is what we observe so there is more and more uh a company using that are going to use like this this kind of QA not going to replace a QA professional um because actual professionally is absolutely necessary to you know understand interpreter data see um efficing the same kind of issues but it's it's a tool in in their dispatch decision basically tool to help that's very fascinating in a lot of ways you mentioned here I think two main use cases are value drivers behind this is when you uncover potential bugs within a game right by looking at that data but you also make a game better by uncovering potential issues in the level design you mentioned here if I bought continuously Falls and a whole then there's probably something wrong with the level design that needs to be improved maybe I'm backing a bit of the methodology are these reinforcement learning agents that are being used or are they programmed relatively more strictly what is the methodology Behind these tools so from what we were able to to see working with this team in particular they started with something very simple so just to be able to implement the agent and see how it was reacting in the world because the ability is it's a very complex to deploy an agent in a game and so first it was very much a Halls that were you know predatory for the audience okay we would like you to learn how to I don't know like do four you know steps and hit something but the iodine was in the long term for this to be actually reinforcement learning so but be able to learn from their environments and do options by themselves um yeah that's very fascinating and you know related to this agent design in AI is actually the MPC AI that is developed you know let's say you're playing Call of Duty you're playing you know your favorite action game how the enemies attack you where they hide the level of difficulty that is all predetermined as well maybe walk us through that type of AI when it comes to Applications applications in gaming what's the process of that so this one is not um I would say in my particular bag of of expertise this is usually taken care of by a development team itself so there is people specialize at programmers scripture and so on and so forth are dedicated to doing the scope things but they think more and more um there is going to have collaboration between you know the specific people and data scientists that would be able to say okay so instead of scratching a certain number of things you can actually use a reinforcement turning you know model to be able to dictate the uh the basically the behaviors of all of those but I don't think we are there just yet I haven't seen a lot of people doing it uh just yet but I know it's something that is more and more the scarce because obviously there is so many possible applications and you mentioned another set of use cases here being you know actual commercial use cases on you know being able to understand what the Community engagement been like for a game being able to understand uh how the game is being played after the game release so that their further optimization maybe walk us through these use cases in a bit more detail what what does it take to set up these types of use cases free launch what is interesting I was discussing for instance the opportunity to optimize a game system so there is a lot of system in the game that use a lot of fight since economic system uh combat system all of those kind of things and the idea is before the game is being released um it's always interesting to get people to actually play and generate data so we have an idea how to you know connect and uh I'm sorry to analyze those um and so we were living people in crunch coming and doing exactly this and the idea was also first it was what we had in mind at least to use actually those Bots to you know the one I was talking about to generate as much as many of them as we could and let them play and see how they were interacting with the game and where the system was actually fading so it could be for instance we are they are collecting up too many resources um and there is no challenge for them to purchase anything uh it could be uh you wanted a bus to be defeated in no less than 10 minutes and uh they like most of the time the players and the spots are the unable to to defeat it in like three minutes so there is a problem there so it was with the inside of the game back they obtains the shadow of the game um but they were also as you were saying uh business aspects um to what we were doing and this one was for instance having developing a tool uh that was allowing us to stop the reviews from journalists and people you know are reviewing the cage for instance or seeing or any of this kind of stuff and uh being able to after all that to do some topic modeling on it uh through determine what kind of things were actually being discuss in the game and um what sentiment was being associated to that so this is very important for marketing for instance uh to have an ad UK what are people saying at the moment what are this they're discussing what kind of sentiment is being associated with that and using it sexually attack a dashboard and if we see uh diploid and we were able to use it I think it's very fascinating you know from your perspective you sit as a good leader within the industry and it seems like you need to have strong collaboration with almost everyone within a gaming organization whether that's the engineering department the creative Studio who wants you know a boss battle to be 10 minutes or around that vicinity of time you need to be able to provide them like good direction over what makes a challenging boss battle but also to the commercial teams finance and marketing and be able to be supportive of them so maybe let's start on the com development and then creative side of the of the coin what does a good collaboration look like for a data science leader in this space uh to be able to understand what are the true requirements for a game to be successful how can I inform the agenda walk me through the challenges there and how you've seen modes of success yeah so this is very much the art of um of everything so we you need to have everyone on board with you and you need to prove to all of those people that what you are doing is actually being useful to them because developing a game is difficult requires a lot of time and at the fifth a lot of money and basically what we were trying to achieve is we are there for you so we are going like we need to collaborate and anything we do it's to answer the questions you have um and helps you meet your objectives so the team what we had was very much in the middle of as you were saying a lot of Sims so that is we were working with user research we were working with marketing um with TR um and the game development team and so to be able to keep you know an open communication with all of those people um we're basically using the agile methodology so we are working on you know two to three weeks Sprints and during the Sprint we were like collecting any requirements uh making sure they were actually met at the end of the screen and so on and so forth and what we were doing also uh to be able for instance to make sure we were like um align with the development cycle of a game um it was actually asking to be part of you know some of their more important meetings so being able to say hey could we have um could we be there when you are developing features uh could we be there when you are we are having sort of explains meetings and so on and so forth and making sure that the information was actually circulating between everyone and maybe as a as someone who's managing a team how do you split your time between coordination with other teams versus actually working with your team and being a contributor yourself how what's the time split there because that's it strikes me as there's a lot of coordination that needs to happen there there is a lot of coordination and this is why um you know you have all of those different roles in a team you are going to have like the southern system so on and suppose those people are actually doing the of the work and they are like you know transforming um the requirements into actual um deliverables um but everyone was actually participating in this collaboration so basically we are the data scientists data analyst um directly embedded into the development so this way they were able to collect any kind of requirements and work on them as soon as possible and as efficiently as possible um so I would say yeah on my side I was trying to coordinate as much as I could also and it was taking a lot of times so um it's it's busy it was basically one of the core activity I'm making sure okay um what has been discussed uh where are we in terms of you know what we need to deliver um are these and those people aware of what we are working on at the moment and so on and so forth so as a director manager um it's it's it's a lot of it's a lot of time that you have of to actually dedicate to this but I'll meet him at least no communication was not just me coordinating everything like everyone was actually talking to the team directly and reporting to the scene what what kind of new information layout so we can coordinate together I don't know if it makes sense but yeah that's that's the way we're doing it it definitely makes sense and you know you mentioned something is that making games is hard it's very hard right like anyone who's not necessarily aware of like the gaming industry you know for context the gaming industry is bigger than the movie industry in the music industry combined right and making games like your favorite games like Grand Theft Auto or Call of Duty or Rockstar or like for example Red Dead Redemption right they take eight to nine years sometimes and creation of like development time crunch right it's it's quite a lot of effort to create a game and oftentimes there's creative Direction changes that happen mid games right throughout the development cycle how does that affect the data team how are you able to you know be agile and be flexible when there are major shifts that happen in the creative direction of a game okay so this is a very good question and we were affected by this a lot obviously because it's not just it's impacting everything it's impacting the planning of the project um when we are going to have um you know play test uh what kind of features are going to stay or what kind of features are going to be cut um all of this kind of things so this is why first and follows I was saying uh I want people to be part of um the meetings where game designers and others are actually having those discussion um so we have a chance I just as soon as the information is being delivered to like that is being known and we have to basically um have those uh you know like this is where we have offsprinted soil we are agile um having those meetings while we say okay so this is the new situation this is what is going to remain what what is going to go away um what's the new privacy for each of those um and of course as you were saying or if this is a is is not just the dev team it's everyone increasing people working in the background so when we have to do this kind of things it impacts our metrics its impacts our documentation it impacts a record of like Matrix is being implemented uh and will have to change but also which one are going to be the commission and which one are going to need a new implementation which which requires you know Dev time uh for phonics to happen um so we are very used to this um which which is why it's not necessarily impacting us too hard because we expect them we expect those kind of thing to happen and um this is why we have also uh you know like uh this coordination with the dev team to say okay during a Sprint um we need Dev to spend time for us to for instance appear on The Matrix and so on and so forth uh for data science for that data science aspect of things it's also very odd because when the game is uh changing suddenly your model are not uh you know working as they should work and you have to sometimes start again from scratch because new features are suddenly appearing um others are just going to get out completely of the game and so on and so forth so basically you have to you know keep track of all of that and you have to be able to adapt retrain your model um like ask people to play the game and a few of them uh like you know set up some Bots and and see what's happening um it's uh it's very challenging it's very challenging and it's it's costly also it's it's quite expensive because training a model and like you know pushing in in production um is not it's costly in terms of how much time it's taking but also um using the data to train the model on and so on and so forth so what we do is there is phase in a project where this is more likely to happen than others so we know for instance we are not going to develop you know a very um like complex data terminal uh for anything that is at the very beginning of a project we are going to wait for for it to be more mature and more stable uh to start doing any of this kind of stuff before that it's going to be more more you know we we try um to see how we can like even deploy anything put everything in place in terms of pipeline um put everything in place in terms of being able to to work with the data this is extremely fascinating you know especially given you know I think one of my two favorite Industries are data science and gaming so I'm really enjoying this conversation and you know in a lot of ways um the creative aspect and creative changes right puts an additional wrinkle to the data science challenges because there's not necessarily a lot of data on that new feature when you create and deploy a model what are the challenge how do you avoid or how do you solve that challenge for not a lot of data let's say early in the development process and the QA process you want to create a machine learning model that's trained on the environment or something along those lines uh what do you what do you do when there's a lot of data scarcity because there's not a lot of players that have played the game so far absolutely see this is the biggest challenge we had because data science and all of those models they require a lot of data to be able to start the beginning to do something um and when you have only like you know 10 to 30 people playing during play tests it makes it very difficult so this is why we had this idea but he was not deeper yet when I was uh before I left um to actually use those bots so you know those AI agents um that are going to be able to play the version of the game that she's actually available um in a way that she's relatively similar to a player uh so you can generate a lot of them and all of those they are going to send Matrix and those metrics you can use them to determine okay what seems to be um or like the most important features and so on and so forth um but yes data sense in pre-launch it's uh yeah it's not always easy if what you're looking for um is improving any security to player playing the game um it's easier for stuff like you know Graphics of OSU or any of those kind of things because you can generate data easily but uh yeah that's uh I did it was what we were trying to achieve um and I have hope that it could work in the future so that's really great and you know we've been talking quite a lot about what happens like data science use cases and um specificities and challenges pre-launch when the game is being developed but maybe let's talk about it as well post launch right um so one aspect that I imagine is extremely difficult and challenging when setting up these types of data science use cases is getting a lot of data from players when millions of players around the planet are playing at the same time uh and trying to understand and unearth insights from that data right so the way I see it is that there's a lot of challenges on the data collection side like how do you actually set up that streaming platform to get a lot of data in to be able to ingest it and transform it but then is what do you look for when you improve to actually improve a game so let's maybe start in that first puzzle here is what are the challenges related to data collection when you have millions of players streaming data to the cloud then you're able to to collect that data and store it yeah so data collection is actually not so much of an issue because we have pipelines um we have a system to collect the metrics to find them Implement them and this is being tested before the game is released so we are already one suddenly we have millions of people coming and providing all of this data we decide which Matrix we are keeping which one we are not keeping this kind of stuff and then we have access to them directly um so we're using Google bigquery and uh Google cloud services to be able to achieve this so it was not as it was the data engineer and a magician on the devops team those people do stuff I don't understand but I'm really happy because it works and um the idea is unlike a lot of people uh like a lot of team for which when the game is released it's released like it's done congratulations you did your work for us it's actually like like marketing and all of this team um works is actually starting because suddenly we are looking into all of the skip here is to make sure the game is actually performing as we were expecting um we are looking into okay so what we see so during you know the pre-launch when we were trying to improve the system um as as it been effectively collected or not in terms of player behaviors um I always have seen new things that we have not identified uh we are doing also you know that since for instance segmentation of failure so we are going to try to determine well fine who is playing our game what kind of uh you know category of players we have based on what they are actually doing inside of the game so for instance oh there is a category of people that looks like they are very engaged they are doing absolutely everything in the game uh While others are barely touching is the surface and how many of those Chipotle we have um you go and we are going to try to predict uh stuff like activity um which we can expect it to change or to state of send based on you know if we are going to the many kind of of events um there is also of course we are discussing and working with cells uh the sales team to see okay can we could we have like a very simple prediction model just to have like an idea of okay um what uh we can expect uh if like as a baseline um in the coming weeks for instance in the Seattle week we are just uh based on that um so there is a lot of things where looking into in terms of a business but also like player behaviors and understanding how the game was performing and if you wanted to choose what is a more challenging time for the data science team would be pre-launch or post launch in terms of the type of use cases as well as the level of technical expertise needed to it on Earth these insights uh I would say everything should be in place and everything should be working for the release of the game so all of your pipeline all of your model um all of you have said this they should be ready up and running even if after that you need to make notification you need to be trained staff and so on and so forth I would say it's more challenging pre-launch because you have you don't have a lot of data and you are basically putting everything in place uh opinion is going to work whilst the game is released there is a new challenge true you suddenly have a lot of data so you need to be sure you or model was able to you know absorb all of that it's going to cost more money because certainly you are training and your deploying model uh on millions of of people um data so maybe you will have to sample for this kind of things um there is obviously the fact that with more data there is new behaviors that's maybe you didn't catch up or when you were in on pre-launch um suppose so on and so bus but once you have everything in place the kind of issues you you will of um is is usually easier to to tackle then when you are building everything up from scratch and you know you mentioned here a lot of insights that you uncover from player usable it has film player tests after the game is launched how does the communication flow look like then with the creative teams how often do they update the game after that what does that process look like uh so it all depends on the kind of games obviously if it's a live game or if it's a game you put in a box and like just sell on the counter but usually what's going to happen we are we are going to have a live dashboard um with all of the Escape here is that our digits to follow up on the game then he's going to going to have reports also so for instance for Gardens we made a series of six ribbots um Andrews were presented to uh like the high level executive but also the producer of the game um and all those kind of things uh we make it also custom report for people having specific question regarding uh yes like you know one aspect of the game of two um and this way we are making sure people looking at follow-up on the game and follow up on what's Happening inside of the game and and sub questions and sometimes um those kind of question can come very far away after the video the launch after she's very interesting if someone is developing a game that seems to be similar to um the game we develops in the past they are going to come to us and say hey could you do some um you know some specific reports marketing obviously is going to want to know what's happening in terms of player activity for instance um if they decide to do um any kind of events so for instance you're pressing your game name on the game bus okay how many more people did you get and it allows you to say okay maybe how many of those people would have paid if we had not made this specific thing um so there is a lot like we are trying to to keep everyone informed because this is extremely important like people working on the game um like took time to actually you know discuss with us and provide time for us to have metrics in the game and so on and so forth and promise it's super important they they do okay this is the results of everything you did like you you General see how the games is working and what people are doing this is this is really great there's a couple of things that you mentioned here that I would like to further unpack so you mentioned you know that marketing question of you know let's say game the game is on Game Pass what do we do should let's say the game is on Game Pass uh should we have kept the game out of Game Pass yes or no um when it comes to the data coming there how important is the console maker developer ecosystem how do they provide you that data where do you get that data from Microsoft right what does that look like I have to say I'm always extremely uh astonished at our unwilling the uh most of those uh you know console uh developer are to actually provide data in a format that would allow us to go further into the analysis we would like to do on our side we we know we get the data from the game generally we are going to get information about how many people are doing what and so on and so far so bus is completely Anonymous like it's completely HTTP and everything um how the cells it's not us that we're collecting this information it was you know marketing of finance and from my understanding most of the time those information were coming into a different kind of format on Excel extract and it was that even you know this person has made this purchase it's going to be a conglomerate like each like months this is how much you you like you would have sell which which was uh you know kind of a challenge to do a prediction and stuff like this yeah interestingly Netflix follows a similar model not a lot of people know how much their movies have been downloaded for example let's say you are a comedian on Netflix you don't know how much your movie has like your special has been streamed it's which is puzzling me so much because um those are our players basically ourselves um we should be able to collect information at a People level because this is important for us but in the console industry uh it's not working this way um on the PC side I think it's far much easier if I understand well because um you know Steam and so on and so forth they are letting you uh you know get access to all of this kind of information um yeah but yeah it's a challenge this one like for the sales I have to set a controller I it's such an interesting it's an interesting space and then the other question that I wanted to ask you here is well which is you mentioned you know once something that's very specific that I think we can go on a tangent here and discuss quite a lot which is uh it depends on the type of the game right if it's a live game if it's a off-the-counter game and you can even go into more detail here is a sports simulator is it a first person shooter is it a you know like a game like Hades for example that is developed over time right um so that also has a lot of wrinkles when it comes to what type of data science you do so maybe walk us through the relationship between the type of game and what type of data science you do on the game what what does that relationship look like yeah absolutely uh so a live game part by definition um will be updated as it goes but there is going to be events but there is also going to have a lot of bad shots of constant development plus you know the official release date which means there is far much more opportunity for a data analysis and data science team to actually have an impact on the game because they can provide those reports and use those models to actually provide information to the dev and sell their questions and so on so so far as I make the combination to you know improve certain aspect of the game um in terms of whatever you want like it could be the revenue it could be the retention it could be the acquisition it could be a lot of different aspects based on with the PM or did anyone that is concerned needs and wants um when a game is being released um as you know a box basically it's a bit different because the new iteration of that game are foreign so you are going to be able to still learn a lot of things because it's going to inform any kind of like similar project and also what kind of retail and Investments you add um just simply you know based on like the number of people playing you know how much they are staying they are like you know abandoning the game and so on and so forth um but the kind of things you we will do is not necessarily going to be as uh immediate let's say um in terms of the impact on the game in a lot of ways as well that will also determine what goes into a sequel of a game in case the SQL is being developed so I was for example I played cutter four then God of War 2 directly after each other and you could see that there were some quality of life improvements that were done on really small things that would have most likely been informed by data science and by analyzing player data that played it absolutely um so it's it's um it's interesting because it's maybe it might be less immediate let's say but it's still very important because um very often approach a lakino studio is not going to have like just one game in development and that's it and then we start a new one there is multiple game being developed at the same time and so what we learned from one of those game you just released you can use to like determine what could possibly happen for the others especially knowing that most of the studios are specializing to a certain type of game um so even if it's not exactly the same IP or the same game it is just things you can learn how much you just did this is super useful now Marie as we close out our conversation I'd be risk not to talk to you about generative AI you know we've seen if you look back also like two years ago when gpt3 was released one of the first crazy use cases of gp3 was actually a game use case which was a form of Dungeons and Dragons type uh roll base playing game which is through text that is essentially auto-generated each time you run a playthrough uh we've seen a lot of cool use cases in game design from creating assets and Unreal Engine to creating You Know audio assets as well to even creating you know player interactions and having a dialogue of Agents you know so do you see this becoming more and more used in gaming over the next few years um like no one can predict the future but I think any studio um that is a little bit serious about data and how they want to use them to improve their game is going to use this kind of um of Tool uh because it's so powerful one of the things it could be used just to start would be generating dialogues you know for the NPC box for the enemies um you know any of this kind of stuff that is usually painful to do uh by hand and could just be automated so you know people that are developing that our team of the game can can focus on more interesting stuff um there is so many uh ways it could um you know give the feeling of um of a more Vivid world um so the kind of um you know like experience it provides um it's I'm already using it for for a lot of like teach me like I'm a five-year-old stuff and it's impressive like you feel like you are talking to someone um even if obviously it's not the case it's very impressive indeed and I think what's even more exciting for what this means for the future of gaming is what this means for the future of Indie gaming in a lot of ways if a two-team developer team are able to create incredibly high quality dialogues incredible high quality assets with the use of AI then this only means better games for everyone in the future as well so this is a pretty interesting space to follow and it's done the unification because you know you could ask him to generate quotes for you uh it's but since it's pretty impressive I was looking at it and you can just say could you provide me um you know I have this question I want to do these things I would like you to generate some codes in you know blah program and it actually does it so I have to say it so much you know faster and easier than just go to stackoverflow and improve like a bit of code that's going to no tubes so I feel like it's going to maybe reduce uh some you know some development time potentially also by providing held to the programmers and anyone who needs this kind of things I couldn't agree more now maybe as we wrap up our conversation today do you have any final words to share with the audience before we end our chat uh yes I would say that we are living very exciting time um because we are just beginning to scratch the surface of everything that could be done with data science in games and um this is still a very new you know um kind of um adventure for a lot of studios and I feel like with Chad gpt4 and other you know improvements in AI um there is so much potential so many things that could potentially be done to to make it better for everyone but I think there is still the barrier of the habits there is some energy yet because it's very new it's um it's not something that people are used to think about when they are developing a game so I think we will have um uh we will have to actually consider it seriously and provide you know the resource and time that it needed for them to be implemented that is very great money thank you so much for coming on data frame that I really appreciate it thanks to you it was uh it was a very good time speaking it was I had a blast I love talking about gaming any day of the week and let alone talking about data science yeah yep I appreciate it thank you foreignwe are living very exciting time because we are just beginning to Scotch the surface of everything that could be done with data science in games and this is still a very new you know um kind of um adventure for a lot of studios and I feel like with Chad GPT 4 and other you know improvements in AI there is so much potential so many things that could potentially be done to to make it better for everyone it's great to have you on the show nice to meet you I'm very happy to be there okay so you've been at the intersection of data science and gaming uh for more than 10 years now so maybe to set the stage for our conversation you know let's say I'm playing Guardians of the Galaxy on my PS5 what role did data science or is play or data science is playing in making sure that I'm having a great experience okay so of course uh I can't you know reveal anything that would be uh NTA related but um on a general basis what is going to be happening is the data that is being generated by players when they are playing uh is being collected three black launch and after launch um to help developers actually understand how people are playing the game and um if what they added mind uh is actually what is happening in the game um so there is a lot of different ways um data science can be used to make sure this happen you can use data sense to optimize the system of the game for instance you can use AI to test the game automatically to see if there is any kind of issues and of course in my team also data analyst and they were working with data scientists to provide you know any kind of like uh reporting dashboard to better understand observing in terms of behaviors so that's really great you know in a lot of ways there's a lot of different use cases or areas where data science provides a lot of value in gaming you'll add quite a few different teams data teams and gaming maybe walk through the different types of areas within a game where data science and machine learning play a lot of value and play a big role in delivering value what is very interesting about data science is our facilities its Encompass so many different applications and in my previous job we actually had like an AI team we also had people inside of the development team working on those kind of things um and we had like our machine learning people in my team working on different things related to business so in the game you there is we can just discuss for instance the AI that are actually being developed to help you know a player having fun by engaging with NPCs that are when defined but also being able to defeat enemies that are an actual challenge um so there is more and more data being used during developments to be able to you know achieve this kind of of course and there is also optimizing you know anything like the sound of the game um or the graphics how to use machine learning and data cells to actually help um you know in this kind of challenges um and we've been using uh also on our side data science to determine how people are actually reacting to everything it was related to the game so reviews and people like journalists are publishing different video reviews we were able to extract information from those and say okay so this is what people are actually talking about and this is the sentiment that she is actually uh you know um Associated to them so uh what kind of things could you do based on those kind of things so there is so many possible uh you know application um and different people being uh you know specialized in those specific areas that's very cool and you know I as a gamer myself it's very nice to see you know the applications of data science across the chain of gaming from you know development to the actual AI behind the NPCs and even after release because the community aspect of gaming is such a huge important element that makes for breaks the success of a game so understanding the pulse of the fans what they want Etc is is really important so I want to you know maybe pause out a few use cases that you talked here about let's maybe discuss you know early in development right I'm very keen on discussing what goes into AI development for an NPC but first let's talk about development uh you know there's a lot of data science use cases that go into QA testing into making sure that the game is running smoothly it releases bug free we've seen a lot of disasters in the industry for example where you know games were released prematurely with a lot of bugs we've seen a lot more attention nowadays being being applied towards QA testing for games maybe walk us through an example of how that looks like in action what is a process of setting up a data driven QA process okay so this particular aspect of things we were actually working with the AIT um Studio to develop those basically agents so it was that my team but we were working with them to be able to provide you know the dashboard and everything but the idea is you can like basically um you know set up um Teach an agent to do different kind of actions or to learn from its environment and find stuff by himself and then you can deploy them in a version like of the game or like any kind of place you would like to to improve um so they would be able to just play and they will generate data basically and using this data you will be able to determine all it looks like um this seems to be a problem so for instance if you see your agents are always falling in the same you know uh whole again and again okay maybe there is a problem and we should just get rid of this because it's very confusing and it's it's a problem and the idea is you can generate so many of them and they can do almost anything in the game so it's easier for them to find very specific you know issues that would be difficult to find for a given um and so and you can make them work you know during the nights uh when you're basically sleeping um so there is a lot of um very interesting very interesting uh things that could be done uh to improve the game based on this kind of of Agents just playing it and fighting you know Edge case and um of providing us with over the data they are generating what they play so we can analyze that after that and say okay so this is what we observe so there is more and more uh a company using that are going to use like this this kind of QA not going to replace a QA professional um because actual professionally is absolutely necessary to you know understand interpreter data see um efficing the same kind of issues but it's it's a tool in in their dispatch decision basically tool to help that's very fascinating in a lot of ways you mentioned here I think two main use cases are value drivers behind this is when you uncover potential bugs within a game right by looking at that data but you also make a game better by uncovering potential issues in the level design you mentioned here if I bought continuously Falls and a whole then there's probably something wrong with the level design that needs to be improved maybe I'm backing a bit of the methodology are these reinforcement learning agents that are being used or are they programmed relatively more strictly what is the methodology Behind these tools so from what we were able to to see working with this team in particular they started with something very simple so just to be able to implement the agent and see how it was reacting in the world because the ability is it's a very complex to deploy an agent in a game and so first it was very much a Halls that were you know predatory for the audience okay we would like you to learn how to I don't know like do four you know steps and hit something but the iodine was in the long term for this to be actually reinforcement learning so but be able to learn from their environments and do options by themselves um yeah that's very fascinating and you know related to this agent design in AI is actually the MPC AI that is developed you know let's say you're playing Call of Duty you're playing you know your favorite action game how the enemies attack you where they hide the level of difficulty that is all predetermined as well maybe walk us through that type of AI when it comes to Applications applications in gaming what's the process of that so this one is not um I would say in my particular bag of of expertise this is usually taken care of by a development team itself so there is people specialize at programmers scripture and so on and so forth are dedicated to doing the scope things but they think more and more um there is going to have collaboration between you know the specific people and data scientists that would be able to say okay so instead of scratching a certain number of things you can actually use a reinforcement turning you know model to be able to dictate the uh the basically the behaviors of all of those but I don't think we are there just yet I haven't seen a lot of people doing it uh just yet but I know it's something that is more and more the scarce because obviously there is so many possible applications and you mentioned another set of use cases here being you know actual commercial use cases on you know being able to understand what the Community engagement been like for a game being able to understand uh how the game is being played after the game release so that their further optimization maybe walk us through these use cases in a bit more detail what what does it take to set up these types of use cases free launch what is interesting I was discussing for instance the opportunity to optimize a game system so there is a lot of system in the game that use a lot of fight since economic system uh combat system all of those kind of things and the idea is before the game is being released um it's always interesting to get people to actually play and generate data so we have an idea how to you know connect and uh I'm sorry to analyze those um and so we were living people in crunch coming and doing exactly this and the idea was also first it was what we had in mind at least to use actually those Bots to you know the one I was talking about to generate as much as many of them as we could and let them play and see how they were interacting with the game and where the system was actually fading so it could be for instance we are they are collecting up too many resources um and there is no challenge for them to purchase anything uh it could be uh you wanted a bus to be defeated in no less than 10 minutes and uh they like most of the time the players and the spots are the unable to to defeat it in like three minutes so there is a problem there so it was with the inside of the game back they obtains the shadow of the game um but they were also as you were saying uh business aspects um to what we were doing and this one was for instance having developing a tool uh that was allowing us to stop the reviews from journalists and people you know are reviewing the cage for instance or seeing or any of this kind of stuff and uh being able to after all that to do some topic modeling on it uh through determine what kind of things were actually being discuss in the game and um what sentiment was being associated to that so this is very important for marketing for instance uh to have an ad UK what are people saying at the moment what are this they're discussing what kind of sentiment is being associated with that and using it sexually attack a dashboard and if we see uh diploid and we were able to use it I think it's very fascinating you know from your perspective you sit as a good leader within the industry and it seems like you need to have strong collaboration with almost everyone within a gaming organization whether that's the engineering department the creative Studio who wants you know a boss battle to be 10 minutes or around that vicinity of time you need to be able to provide them like good direction over what makes a challenging boss battle but also to the commercial teams finance and marketing and be able to be supportive of them so maybe let's start on the com development and then creative side of the of the coin what does a good collaboration look like for a data science leader in this space uh to be able to understand what are the true requirements for a game to be successful how can I inform the agenda walk me through the challenges there and how you've seen modes of success yeah so this is very much the art of um of everything so we you need to have everyone on board with you and you need to prove to all of those people that what you are doing is actually being useful to them because developing a game is difficult requires a lot of time and at the fifth a lot of money and basically what we were trying to achieve is we are there for you so we are going like we need to collaborate and anything we do it's to answer the questions you have um and helps you meet your objectives so the team what we had was very much in the middle of as you were saying a lot of Sims so that is we were working with user research we were working with marketing um with TR um and the game development team and so to be able to keep you know an open communication with all of those people um we're basically using the agile methodology so we are working on you know two to three weeks Sprints and during the Sprint we were like collecting any requirements uh making sure they were actually met at the end of the screen and so on and so forth and what we were doing also uh to be able for instance to make sure we were like um align with the development cycle of a game um it was actually asking to be part of you know some of their more important meetings so being able to say hey could we have um could we be there when you are developing features uh could we be there when you are we are having sort of explains meetings and so on and so forth and making sure that the information was actually circulating between everyone and maybe as a as someone who's managing a team how do you split your time between coordination with other teams versus actually working with your team and being a contributor yourself how what's the time split there because that's it strikes me as there's a lot of coordination that needs to happen there there is a lot of coordination and this is why um you know you have all of those different roles in a team you are going to have like the southern system so on and suppose those people are actually doing the of the work and they are like you know transforming um the requirements into actual um deliverables um but everyone was actually participating in this collaboration so basically we are the data scientists data analyst um directly embedded into the development so this way they were able to collect any kind of requirements and work on them as soon as possible and as efficiently as possible um so I would say yeah on my side I was trying to coordinate as much as I could also and it was taking a lot of times so um it's it's busy it was basically one of the core activity I'm making sure okay um what has been discussed uh where are we in terms of you know what we need to deliver um are these and those people aware of what we are working on at the moment and so on and so forth so as a director manager um it's it's it's a lot of it's a lot of time that you have of to actually dedicate to this but I'll meet him at least no communication was not just me coordinating everything like everyone was actually talking to the team directly and reporting to the scene what what kind of new information layout so we can coordinate together I don't know if it makes sense but yeah that's that's the way we're doing it it definitely makes sense and you know you mentioned something is that making games is hard it's very hard right like anyone who's not necessarily aware of like the gaming industry you know for context the gaming industry is bigger than the movie industry in the music industry combined right and making games like your favorite games like Grand Theft Auto or Call of Duty or Rockstar or like for example Red Dead Redemption right they take eight to nine years sometimes and creation of like development time crunch right it's it's quite a lot of effort to create a game and oftentimes there's creative Direction changes that happen mid games right throughout the development cycle how does that affect the data team how are you able to you know be agile and be flexible when there are major shifts that happen in the creative direction of a game okay so this is a very good question and we were affected by this a lot obviously because it's not just it's impacting everything it's impacting the planning of the project um when we are going to have um you know play test uh what kind of features are going to stay or what kind of features are going to be cut um all of this kind of things so this is why first and follows I was saying uh I want people to be part of um the meetings where game designers and others are actually having those discussion um so we have a chance I just as soon as the information is being delivered to like that is being known and we have to basically um have those uh you know like this is where we have offsprinted soil we are agile um having those meetings while we say okay so this is the new situation this is what is going to remain what what is going to go away um what's the new privacy for each of those um and of course as you were saying or if this is a is is not just the dev team it's everyone increasing people working in the background so when we have to do this kind of things it impacts our metrics its impacts our documentation it impacts a record of like Matrix is being implemented uh and will have to change but also which one are going to be the commission and which one are going to need a new implementation which which requires you know Dev time uh for phonics to happen um so we are very used to this um which which is why it's not necessarily impacting us too hard because we expect them we expect those kind of thing to happen and um this is why we have also uh you know like uh this coordination with the dev team to say okay during a Sprint um we need Dev to spend time for us to for instance appear on The Matrix and so on and so forth uh for data science for that data science aspect of things it's also very odd because when the game is uh changing suddenly your model are not uh you know working as they should work and you have to sometimes start again from scratch because new features are suddenly appearing um others are just going to get out completely of the game and so on and so forth so basically you have to you know keep track of all of that and you have to be able to adapt retrain your model um like ask people to play the game and a few of them uh like you know set up some Bots and and see what's happening um it's uh it's very challenging it's very challenging and it's it's costly also it's it's quite expensive because training a model and like you know pushing in in production um is not it's costly in terms of how much time it's taking but also um using the data to train the model on and so on and so forth so what we do is there is phase in a project where this is more likely to happen than others so we know for instance we are not going to develop you know a very um like complex data terminal uh for anything that is at the very beginning of a project we are going to wait for for it to be more mature and more stable uh to start doing any of this kind of stuff before that it's going to be more more you know we we try um to see how we can like even deploy anything put everything in place in terms of pipeline um put everything in place in terms of being able to to work with the data this is extremely fascinating you know especially given you know I think one of my two favorite Industries are data science and gaming so I'm really enjoying this conversation and you know in a lot of ways um the creative aspect and creative changes right puts an additional wrinkle to the data science challenges because there's not necessarily a lot of data on that new feature when you create and deploy a model what are the challenge how do you avoid or how do you solve that challenge for not a lot of data let's say early in the development process and the QA process you want to create a machine learning model that's trained on the environment or something along those lines uh what do you what do you do when there's a lot of data scarcity because there's not a lot of players that have played the game so far absolutely see this is the biggest challenge we had because data science and all of those models they require a lot of data to be able to start the beginning to do something um and when you have only like you know 10 to 30 people playing during play tests it makes it very difficult so this is why we had this idea but he was not deeper yet when I was uh before I left um to actually use those bots so you know those AI agents um that are going to be able to play the version of the game that she's actually available um in a way that she's relatively similar to a player uh so you can generate a lot of them and all of those they are going to send Matrix and those metrics you can use them to determine okay what seems to be um or like the most important features and so on and so forth um but yes data sense in pre-launch it's uh yeah it's not always easy if what you're looking for um is improving any security to player playing the game um it's easier for stuff like you know Graphics of OSU or any of those kind of things because you can generate data easily but uh yeah that's uh I did it was what we were trying to achieve um and I have hope that it could work in the future so that's really great and you know we've been talking quite a lot about what happens like data science use cases and um specificities and challenges pre-launch when the game is being developed but maybe let's talk about it as well post launch right um so one aspect that I imagine is extremely difficult and challenging when setting up these types of data science use cases is getting a lot of data from players when millions of players around the planet are playing at the same time uh and trying to understand and unearth insights from that data right so the way I see it is that there's a lot of challenges on the data collection side like how do you actually set up that streaming platform to get a lot of data in to be able to ingest it and transform it but then is what do you look for when you improve to actually improve a game so let's maybe start in that first puzzle here is what are the challenges related to data collection when you have millions of players streaming data to the cloud then you're able to to collect that data and store it yeah so data collection is actually not so much of an issue because we have pipelines um we have a system to collect the metrics to find them Implement them and this is being tested before the game is released so we are already one suddenly we have millions of people coming and providing all of this data we decide which Matrix we are keeping which one we are not keeping this kind of stuff and then we have access to them directly um so we're using Google bigquery and uh Google cloud services to be able to achieve this so it was not as it was the data engineer and a magician on the devops team those people do stuff I don't understand but I'm really happy because it works and um the idea is unlike a lot of people uh like a lot of team for which when the game is released it's released like it's done congratulations you did your work for us it's actually like like marketing and all of this team um works is actually starting because suddenly we are looking into all of the skip here is to make sure the game is actually performing as we were expecting um we are looking into okay so what we see so during you know the pre-launch when we were trying to improve the system um as as it been effectively collected or not in terms of player behaviors um I always have seen new things that we have not identified uh we are doing also you know that since for instance segmentation of failure so we are going to try to determine well fine who is playing our game what kind of uh you know category of players we have based on what they are actually doing inside of the game so for instance oh there is a category of people that looks like they are very engaged they are doing absolutely everything in the game uh While others are barely touching is the surface and how many of those Chipotle we have um you go and we are going to try to predict uh stuff like activity um which we can expect it to change or to state of send based on you know if we are going to the many kind of of events um there is also of course we are discussing and working with cells uh the sales team to see okay can we could we have like a very simple prediction model just to have like an idea of okay um what uh we can expect uh if like as a baseline um in the coming weeks for instance in the Seattle week we are just uh based on that um so there is a lot of things where looking into in terms of a business but also like player behaviors and understanding how the game was performing and if you wanted to choose what is a more challenging time for the data science team would be pre-launch or post launch in terms of the type of use cases as well as the level of technical expertise needed to it on Earth these insights uh I would say everything should be in place and everything should be working for the release of the game so all of your pipeline all of your model um all of you have said this they should be ready up and running even if after that you need to make notification you need to be trained staff and so on and so forth I would say it's more challenging pre-launch because you have you don't have a lot of data and you are basically putting everything in place uh opinion is going to work whilst the game is released there is a new challenge true you suddenly have a lot of data so you need to be sure you or model was able to you know absorb all of that it's going to cost more money because certainly you are training and your deploying model uh on millions of of people um data so maybe you will have to sample for this kind of things um there is obviously the fact that with more data there is new behaviors that's maybe you didn't catch up or when you were in on pre-launch um suppose so on and so bus but once you have everything in place the kind of issues you you will of um is is usually easier to to tackle then when you are building everything up from scratch and you know you mentioned here a lot of insights that you uncover from player usable it has film player tests after the game is launched how does the communication flow look like then with the creative teams how often do they update the game after that what does that process look like uh so it all depends on the kind of games obviously if it's a live game or if it's a game you put in a box and like just sell on the counter but usually what's going to happen we are we are going to have a live dashboard um with all of the Escape here is that our digits to follow up on the game then he's going to going to have reports also so for instance for Gardens we made a series of six ribbots um Andrews were presented to uh like the high level executive but also the producer of the game um and all those kind of things uh we make it also custom report for people having specific question regarding uh yes like you know one aspect of the game of two um and this way we are making sure people looking at follow-up on the game and follow up on what's Happening inside of the game and and sub questions and sometimes um those kind of question can come very far away after the video the launch after she's very interesting if someone is developing a game that seems to be similar to um the game we develops in the past they are going to come to us and say hey could you do some um you know some specific reports marketing obviously is going to want to know what's happening in terms of player activity for instance um if they decide to do um any kind of events so for instance you're pressing your game name on the game bus okay how many more people did you get and it allows you to say okay maybe how many of those people would have paid if we had not made this specific thing um so there is a lot like we are trying to to keep everyone informed because this is extremely important like people working on the game um like took time to actually you know discuss with us and provide time for us to have metrics in the game and so on and so forth and promise it's super important they they do okay this is the results of everything you did like you you General see how the games is working and what people are doing this is this is really great there's a couple of things that you mentioned here that I would like to further unpack so you mentioned you know that marketing question of you know let's say game the game is on Game Pass what do we do should let's say the game is on Game Pass uh should we have kept the game out of Game Pass yes or no um when it comes to the data coming there how important is the console maker developer ecosystem how do they provide you that data where do you get that data from Microsoft right what does that look like I have to say I'm always extremely uh astonished at our unwilling the uh most of those uh you know console uh developer are to actually provide data in a format that would allow us to go further into the analysis we would like to do on our side we we know we get the data from the game generally we are going to get information about how many people are doing what and so on and so far so bus is completely Anonymous like it's completely HTTP and everything um how the cells it's not us that we're collecting this information it was you know marketing of finance and from my understanding most of the time those information were coming into a different kind of format on Excel extract and it was that even you know this person has made this purchase it's going to be a conglomerate like each like months this is how much you you like you would have sell which which was uh you know kind of a challenge to do a prediction and stuff like this yeah interestingly Netflix follows a similar model not a lot of people know how much their movies have been downloaded for example let's say you are a comedian on Netflix you don't know how much your movie has like your special has been streamed it's which is puzzling me so much because um those are our players basically ourselves um we should be able to collect information at a People level because this is important for us but in the console industry uh it's not working this way um on the PC side I think it's far much easier if I understand well because um you know Steam and so on and so forth they are letting you uh you know get access to all of this kind of information um yeah but yeah it's a challenge this one like for the sales I have to set a controller I it's such an interesting it's an interesting space and then the other question that I wanted to ask you here is well which is you mentioned you know once something that's very specific that I think we can go on a tangent here and discuss quite a lot which is uh it depends on the type of the game right if it's a live game if it's a off-the-counter game and you can even go into more detail here is a sports simulator is it a first person shooter is it a you know like a game like Hades for example that is developed over time right um so that also has a lot of wrinkles when it comes to what type of data science you do so maybe walk us through the relationship between the type of game and what type of data science you do on the game what what does that relationship look like yeah absolutely uh so a live game part by definition um will be updated as it goes but there is going to be events but there is also going to have a lot of bad shots of constant development plus you know the official release date which means there is far much more opportunity for a data analysis and data science team to actually have an impact on the game because they can provide those reports and use those models to actually provide information to the dev and sell their questions and so on so so far as I make the combination to you know improve certain aspect of the game um in terms of whatever you want like it could be the revenue it could be the retention it could be the acquisition it could be a lot of different aspects based on with the PM or did anyone that is concerned needs and wants um when a game is being released um as you know a box basically it's a bit different because the new iteration of that game are foreign so you are going to be able to still learn a lot of things because it's going to inform any kind of like similar project and also what kind of retail and Investments you add um just simply you know based on like the number of people playing you know how much they are staying they are like you know abandoning the game and so on and so forth um but the kind of things you we will do is not necessarily going to be as uh immediate let's say um in terms of the impact on the game in a lot of ways as well that will also determine what goes into a sequel of a game in case the SQL is being developed so I was for example I played cutter four then God of War 2 directly after each other and you could see that there were some quality of life improvements that were done on really small things that would have most likely been informed by data science and by analyzing player data that played it absolutely um so it's it's um it's interesting because it's maybe it might be less immediate let's say but it's still very important because um very often approach a lakino studio is not going to have like just one game in development and that's it and then we start a new one there is multiple game being developed at the same time and so what we learned from one of those game you just released you can use to like determine what could possibly happen for the others especially knowing that most of the studios are specializing to a certain type of game um so even if it's not exactly the same IP or the same game it is just things you can learn how much you just did this is super useful now Marie as we close out our conversation I'd be risk not to talk to you about generative AI you know we've seen if you look back also like two years ago when gpt3 was released one of the first crazy use cases of gp3 was actually a game use case which was a form of Dungeons and Dragons type uh roll base playing game which is through text that is essentially auto-generated each time you run a playthrough uh we've seen a lot of cool use cases in game design from creating assets and Unreal Engine to creating You Know audio assets as well to even creating you know player interactions and having a dialogue of Agents you know so do you see this becoming more and more used in gaming over the next few years um like no one can predict the future but I think any studio um that is a little bit serious about data and how they want to use them to improve their game is going to use this kind of um of Tool uh because it's so powerful one of the things it could be used just to start would be generating dialogues you know for the NPC box for the enemies um you know any of this kind of stuff that is usually painful to do uh by hand and could just be automated so you know people that are developing that our team of the game can can focus on more interesting stuff um there is so many uh ways it could um you know give the feeling of um of a more Vivid world um so the kind of um you know like experience it provides um it's I'm already using it for for a lot of like teach me like I'm a five-year-old stuff and it's impressive like you feel like you are talking to someone um even if obviously it's not the case it's very impressive indeed and I think what's even more exciting for what this means for the future of gaming is what this means for the future of Indie gaming in a lot of ways if a two-team developer team are able to create incredibly high quality dialogues incredible high quality assets with the use of AI then this only means better games for everyone in the future as well so this is a pretty interesting space to follow and it's done the unification because you know you could ask him to generate quotes for you uh it's but since it's pretty impressive I was looking at it and you can just say could you provide me um you know I have this question I want to do these things I would like you to generate some codes in you know blah program and it actually does it so I have to say it so much you know faster and easier than just go to stackoverflow and improve like a bit of code that's going to no tubes so I feel like it's going to maybe reduce uh some you know some development time potentially also by providing held to the programmers and anyone who needs this kind of things I couldn't agree more now maybe as we wrap up our conversation today do you have any final words to share with the audience before we end our chat uh yes I would say that we are living very exciting time um because we are just beginning to scratch the surface of everything that could be done with data science in games and um this is still a very new you know um kind of um adventure for a lot of studios and I feel like with Chad gpt4 and other you know improvements in AI um there is so much potential so many things that could potentially be done to to make it better for everyone but I think there is still the barrier of the habits there is some energy yet because it's very new it's um it's not something that people are used to think about when they are developing a game so I think we will have um uh we will have to actually consider it seriously and provide you know the resource and time that it needed for them to be implemented that is very great money thank you so much for coming on data frame that I really appreciate it thanks to you it was uh it was a very good time speaking it was I had a blast I love talking about gaming any day of the week and let alone talking about data science yeah yep I appreciate it thank you foreign\n"