AWS's Latest Announcements: Democratizing Machine Learning and Making it More Accessible to All

At AWS, they have been working tirelessly to make machine learning more accessible to everyone, regardless of their expertise level. In recent times, they have been announcing new features that aim to democratize machine learning and make it easier for developers, students, and experimenters to get started with this powerful technology.

One such announcement is about the serverless environment where customers can benchmark the performance of their inference recommender to get the best price-performance instances. This feature allows customers to make informed decisions when it comes to choosing the right instance type for their applications, ensuring that they get the lowest cost or best performance without breaking the bank. By exposing this functionality, AWS is giving its customers more control over their costs and resources.

Another exciting announcement from AWS is about the Sagemaker Studio Lab. This feature aims to make machine learning more accessible to students and experimenters who want to quickly get started with building models. With Sagemaker Studio Lab, users can create, train, and deploy models without having to write code or worry about infrastructure setup. This makes it an ideal platform for beginners and experts alike to explore the world of machine learning.

In addition to Sagemaker Studio Lab, AWS has also announced Canvas, a new platform that aims to make machine learning more accessible to everyone. Canvas is designed to be user-friendly and intuitive, allowing users to create, train, and deploy models without requiring extensive technical knowledge. With Canvas, users can focus on building their models rather than worrying about the underlying infrastructure.

Furthermore, AWS has also announced Model Compiler, a new feature that aims to make machine learning more efficient for developers. The Model Compiler is designed to take advantage of modern compiler technology to create optimized machine learning models that are faster and more accurate. By leveraging the power of compilers, developers can reduce the time it takes to train and deploy their models, making them more productive and efficient.

Another announcement from AWS is about Ground Truth Plus, a fully turnkey operation for data labeling. With Ground Truth Plus, customers can provide their own data and let AWS handle the rest, including selecting labelers, validating output, managing workforces, and embedding machine learning into the data labeling process. This makes it an ideal solution for companies that need to label large amounts of data quickly and efficiently.

Ground Truth Plus is designed to remove the friction from the data labeling process, making it more efficient and cost-effective. By leveraging machine learning models to automate data labeling, Ground Truth Plus can reduce costs by up to 40%. This makes it an attractive solution for companies that need to label large amounts of data, but don't have the resources or expertise to do so themselves.

In conclusion, AWS's latest announcements demonstrate their commitment to democratizing machine learning and making it more accessible to everyone. With features like Sagemaker Studio Lab, Canvas, Model Compiler, and Ground Truth Plus, AWS is providing its customers with a range of tools and platforms that can help them build, deploy, and manage their machine learning models more efficiently and effectively.

"WEBVTTKind: captionsLanguage: enall right everyone welcome to our annual coverage of the aws re invent conference i'm here today with bratan saha broughton is vice president and general manager of ai and ml at aws broughton welcome to the twiml ai podcast thank you so i'm glad to be here thanks for having me absolutely let's just jump right in i'd love to have you share a little bit about your role and background with our audience so i'm vp for aiml at aws that includes a bunch of our ai services things like personalized forecasting health ai services hml services like panorama and so on ai devops where we use ai to make it easier to run devops or for software development and then that includes the middle layer of the aws services which is sagemaker which is a flagship machine learning service as well as the ml frameworks and infrastructure where we provide both custom hardware gpus as well as machine learning frameworks and then prior to coming to aws i was at nvidia and then prior to that at intel this year as has been the case in years past aws has a lot to announce at reinvent on the ai and ml front uh one of the things that i've noticed is that the announcements are catering to two different communities this year uh there's a set of announcements focused on making ml and ai more accessible on the aws platform but there's also a set of announcements that seem to be aimed at more complex use cases more mature users and i thought maybe a good place to start our discussion is hearing kind of how you're thinking about the the market uh and and where we are and the context in which aws is releasing these new offerings i know one of the ways you think about that is in terms of industrialization of machine learning share your perspective on that please thank you sam yes so it's indeed true that you know we have been releasing and we are going to be releasing features in terms of two customer personas one of them as you suggested is the machine learning practitioner and our goal there is to make it easier and easier to build machine learning models and train and deploy machine learning models and to set some context uh this year in 2021 we have customers who many of whom want to deploy a million models each and they want to be able to train models with billions or tens of billions of parameters now think of it you know if you go back three years let's say when we launch sage maker at that point of time the state of the art model used to have like 20 million parameters and now we are talking of tens of billions of parameters maybe even more than 100 billion parameters so that's the scale at which machine learning is growing and then we have customers who want to do hundreds of billions of predictions per month and if you look at a data labeling services you know we're labeling more than a million objects a day and so what that has meant is that machine learning is no longer really a niche you know it is something that today on aws more than a hundred thousand customers are using it and you know more machine learning happens on aws than anywhere else so we see kind of the need for industrialization potentially ahead of others and so uh what we are trying to do in here and what customers want us to do is to make it easy to deploy machine learning at scale and automate it and make it repeatable so that you know there are no errors in it so that's the industrialization front which is how do we make it easier for people to use and deploy machine learning then there's the other vector which you also pointed out which is that the demand for machine learning practitioners is growing much faster than people companies can hire them you know there's a there's a survey by a third party that said the demand for machine learning and ai practitioners has been growing by 74 annually for the last four years you know 2x more than any other emerging job category and so many of the customers told us that you know can you make it easier for us to have more employees do machine learning you know not just data scientists and ml practitioners but others as well like data analysts like marketing and sales professionals and these are employees who use the data you know who understand the data who would benefit from these machine learning insights and so that's the other one where we are going in and saying how do we fundamentally change the paradigm and make it possible for people who may not be machine learning practitioners who may not have software coding skills yet do machine learning and so that is where we are doing a lot of no code low code tools and that is where sagemaker canvas comes in and so it's really about two things one is democratizing access across many more employees through no code low code machine learning tools the other is making it easier and easier for machine learning practitioners to build and deploy machine learning at scale because you know the the amount of deployment going on has really increased a lot by orders of magnitude awesome let's uh let's maybe jump right into uh some of the announcements you mentioned canvas and that is a no code environment tell us a little bit more about the kind of the design center of canvas and and what you're going for there yeah and you know this is a time when we are taking sage maker to a new audience up until now we were looking more at ml practitioners ml ops engineers uh software developers and this is going to an audience like analysts and sales and marketing professionals who don't want to necessarily be doing coding but who still want to be using machine learning and so canvas is we completely redid the ux we completely redid the ux so that you know it starts from the use case you know you may want to do churn prediction you may want to do sales forecasting and so on so it starts from the use case you don't have to write a single line of code you know you can access data sources multiple kinds of data sources on-prem or in the cloud and then canvas will automatically do the data transformations that you need to get done it will automatically build the right model for your use case it will help you deploy the model all of this without writing a single line of code and then it will also explain to you why it's making certain predictions so for example let's say you want to use it for churn prediction it will tell you you know here are the reasons why i think this customer matures so you are it's not just giving you insights it's giving you actionable insights because it tells you why it's doing something and all of this as i you know emphasize without writing a single piece of single line of code so that we can make it really accessible and i'm really excited about how canvas will make machine learning accessible to a lot more employees now the other innovation that canvas brings is that it's very easy for an analyst or a user to export all of the work that they have done on canvas onto sagemaker studio and so think of it as you know you're building your machine learning models and all that but you can again have an expert data scientist look at everything that has been created it's almost like say in the software industry in your writing code you want someone to code review before you push the code into production and so that is another of the key innovations we bring in here is we increase the collaboration between an analyst user as well as a data scientist i was going to ask about that last point uh traditionally with uh no code environments you know it's often perceived by technical folks as being a bit of a trap in that you um you can do a lot of configuration and get from say zero to one or zero to a half even you know just clicking around on the screen but then when you want to get to you know the next level you end up running up against the wall it sounds like here canvas is a front end that's building the code artifacts on the back end and i can take those and extend them and build on them is that that the case it is exact it's a front end it's using the same back end that sagemaker uses it's the same back end that's that customers are actually using to you know train models with these billions of parameters and deploy millions of models and canvas is going to export the source code for everything it generates so it's going to give you the source code for the models for the data preparation routines and so on so a data scientist can then use that as a starting point and refine it further when you think about about canvas and the process of delivering that as a product what were some of the key challenges in uh in delivering it you know the important thing that we wanted to keep in mind was how do you raise the level of abstraction so that it's easy to use while at the same time not making not raising the level of abstraction so much that you can't do useful stuff with it and that's often the trap as you very rightly pointed out sam that's very often the trap that you fall in with these no code loco tools the second one was how do you design the ux in a way such that you convey the central concepts but you don't expect the user to have a deep understanding of machine learning like you know if you were doing this on sage maker studio you would you know you would have a graph and you say well this is the auc the area under the curve and so on now you can't really use those terms for someone who may not be a deep practitioner of machine learning uh similarly explainability you know you want to be able to explain in a right way that okay this was the input this is the input that's contributing most to the output so it's really kind of you know threading the needle so that you have raised the level of abstraction and made it really easy to use while at the same point you still allow interesting things to be done and you're explaining all of the concepts in such a way that you're not overwhelming the user with machine learning knowledge and is there an example of uh one of those trade-offs that you made in in the product and how it played out yeah so you know one of the things for example is um if you think about a normal you know it's kind of a sage maker studio kind of an environment where you have a notebook and a notebook is an interactive environment for doing machine learning so you know you do get an interactivity there and you know what's going on in the case of canvas we couldn't give that interactive environment like a notebook it has to be much higher level and what that means is that your back end needs to be much more snappy now because you know you can't really take a few hours to train your models you need to be able to give quick feedback you need to have indicators that says how much of the job has progressed you need to be able to um explain it at an at a you know much more intuitive level what your data sets look like what kind of you know what kind of data permutations are allowed to do um what kind of filtering you're allowed to do so we really had to rethink from the perspective of a very different kind of user and i imagine given the description you just gave that the target use cases are kind of those core uh structured data types of use cases as opposed to you know computer vision or nlp or things like that yes it's really more of structured data tabular data you know sales forecasting churn prediction fraud detection that kind of stuff like you know things that a data analyst or you know sales professional marketing professional is using all the time so it's really meant for tabular structured data awesome along those lines targeting the this kind of democratization end of the the user spectrum if we can say it thusly um you are also announcing sagemaker studio lab uh tell us about that offering yeah i'm really excited by what uh sagemaker studio lab will enable students and other experimenters who want to quickly get started with machine learning so sagemaker studio lab offers usage maker notebooks it's integrated with github so it makes collaboration easy it's pre-packaged with all the popular machine learning tools so that you know student or an experiment or for that matter any user can quickly get started it gives you free compute and it gives you free storage and more than that you don't have to do things like you know shutting down your instance or saving your work saving your models after your work is done studio lab does it for you so it's as easy as you know closing your laptop and then coming back to it and resuming your work again and it's no setup no charge what that means is you don't even need an aws account to get started you can just use your email address to log into studiolab and get started with it that's a change for aws it's uh it's a new kind of a product we think you know we think machine learning our customers tell us that they really want us to make machine learning much more easily accessible and so we really looked at what are the friction points and you know there are a lot more we believe there'll be a lot more people wanting to do machine learning tomorrow than there is today and so it's really important for us to help customers leverage machine learning just as we at amazon have been doing it for a long time and so we think it's really important to make it easy for students experimenters other customers to get started so it's it's indeed a pretty groundbreaking game changing product there are other lab offerings out in the market and they are typically limited by the amount of time you can access them or the the type of compute that you can get access to what are the constraints with uh with studiolab so we have looked at we have talked to customers and we have looked at the typical usage scenario and the kind of limits that we have we believe are more than sufficient for the typical usage scenarios so you can use both cpus and gpus and you know there's a time limit for it well was at launch but you know those should capture the majority of the usage uh and you know you get not just cpus but you get gpus as well and the other thing that i'll mention is it comes with you know there is encryption at rest encryption at transit so you know it comes with really good security properties and that is really hard to build because you know you're building this thing where you expect a lot of people to use it because of the nature of the product at the same time you want to make sure that this is something that maybe a scientist at an enterprise is comfortable using it and so we have to take a lot of care to kind of marry these two paradigms in the sense that you know you just don't want to throw out an aws product that doesn't have the highest levels of security and then the other thing that studiolab in my mind does really well is it's not two different islands it's not that sage maker studio lab is a separate island and sage maker studio is a separate island and sagemaker studios you know the enterprise class you know the the most popular machine learning platform today for enterprises when you're using studiolab you can actually easily transfer all of your work to studio okay and so you know you can get started you can do your training with a small data set if you want you know do some experiments and then you say now let me actually do it in a much larger data set or let me use a much larger model you can just export all of your work to studio and then get all of the features that sage maker provides so i think that you know that thing of uh providing still providing good enterprise class security guarantees a while at the same you know so we didn't make it two different islands while at the same time giving it all of the properties of free compute free storage easy to use easy to get started no aws account needed that needed a lot of innovation by the teams is there a free compute component to canvas or can you export from canvas to studio lab to do training so we have a free tier in canvas as well and we have looked at how customers would want to get used to it and see the power of the tool so there is a free tier component of canvas that customers can get started with we haven't yet had customers really ask for an export from you know canvas to studiolab they have asked us for canvas to studio so we are we are going to be releasing that but you know if customers want it we'll do it maybe one last question on the topic of democratization um yeah i've been going to re invent for a long time uh and one of the hallmarks of early reinvent was the cost of compute would drop from year to year to year um do you see the same thing happening on the machine learning side you know obviously there's a relationship between democratization and and the cost of accessing services like these indeed yes that's a great question so um we are doing a lot of hardware and software optimizations to pull down the cost of compute at the hardware level first if you start with inferential inferential provides customers with up to 70 percent lower cost uh than previous generation and comparable gpu instances and then you know we are we also launched trainium in preview and trainium is going to provide the best price performance for training machine learning models in the cloud and then if you look at our for inference we have a g5 instances and the g5 instances are also much more powerful than the previous generation g4 dn instances and the same with on the machine learning training side same with the p4d instances you know which are up to 60 lower cost than the previous generation p p3 p3dn instances so there's that aspect of reducing the hardware infrastructure cost by continuously innovating you know bringing out new gpus bringing out new custom accelerators then there is the software optimizations that add on top of it okay and when you think of that for example we are releasing serverless inference and what serverless inference does is it gives you paper use so there are many use cases where say think of a restaurant recommendation from an online food delivery site you know where you have your workload is very intermittent and so customers don't want that instance to be running all the time at that time and so today they built complex workflows to shut down their instances so we are bringing out sagemaker serverless inference that will automatically provision instances for you scala scale it up and down based on your traffic and then shut down the instance so if there's no traffic you pay nothing then there are these other use cases where customers are going for what i would call you know hyper personalization where they deploy lots of models maybe up to million models or more where they want to be able to really customize it to their end users needs and then they say well if i'm going to deploy so many models i don't want to allocate so many instances and so we have features like multimodal endpoints which let you host up to a million models in a single instance and then we do all of the work of caching things in and out so the combination of both hardware infrastructure optimizations and software optimizations should continue to give customers what they're used to which is the cost of compute the cost that they're paying for compute going down just as it has in the past yeah that's a great segue into digging into some of the more complex use cases that you reference customers with a million models and i think the first question i have on that is can you you know are there patterns that you see to the customer journey you know relative to you know where they start and how they get to having a million models what does that typically look like or what are you know there are some examples that you can share of the trajectory for customers that make it there you know in typically customers will initially start with um not very complex use cases you know something that has concrete business value uh but at the same time isn't kind of at the bleeding edge of the spectrum and you know the initial parts is really to get your data in order so you know have your make sure you have enough data clean data you have you know data lake and all that set up then pick a significant use case that demonstrates clear business value yet is not that difficult and that is where we typically see customers start their proof of concepts and once they have deployed you know few models a dozen models then they just scale it out uh to you know thousands of models tens of thousands of models and even more are there inflection points along the way that are kind of markedly different you know as you as you pass them i think what happens is as you get beyond a certain point it's no longer just the machine learning or the model data aspect of it that is important at that point your processes and your mechanisms and your automation become equally important and it's a lot like say software you know and software if someone is writing small little pieces of code it's one thing but once you get into production massive deployments you need your ci cd you need your automation you need your tools all of which is needed to make sure that the process of creating whether it's software whether it's models what have you that the process of it is repeatable automated reduces errors and so what we have seen is you know the initial few model deployments initial few proof of concepts you can do them in a little bit of an ad hoc manner and you know many customers can do it because they want to go fast and they want to be experimenting with it but sam as you pointed out you get to a point when that is no longer sustainable and so that is when you really need to look at what are my ml operational practices you know or mlops what are my ml operational practices how am i automating the end to end workflow how am i looking at audit and governance what is my security posture all of these come into the picture then and so at that point you really have to make sure you put in all of these and that is what really helps you scale you referenced this as uh industrialization earlier and just thinking about the list that you just rattled off of concerns you know operationalization and ml ops was just one of those things you think of industrialization as a superset that includes mlaps or are they more or less anonymous so we look at industrialization as a superset of ml ops and we really look at industrialization as three different vectors one of them is the infrastructure and as you mentioned you know the cost of the infrastructure the performance of the infrastructure has to keep getting better like if you look at let's say the machine learning models you know three years back when resnet was the state of that model you were looking at about 20 million parameters now you're looking at you know gpt class of models which are talking about billions of parameters and so you need your compute infrastructure and the software infrastructure around it to keep getting faster so that you can do these things so one vector for ml industrialization is performant and cost effective infrastructure the second aspect of it which we look at is tooling you know these are tools that are custom built purpose built for machine learning so that they can reduce the heavy lifting in all of your machine learning tasks and these are you know machine learning development environments like machine learning ides you know machine learning project management systems you know machine learning bias detection tools and then debuggers and profilers that are customized for debugging and profiling machine learning models so all of these tools that help it easy that make it easier for the developer to be more productive and churn out a lot more and then there's the third aspect of it which is the automation aspect of it which is what mlofsc so we really look at it as infrastructure which is faster and cheaper tooling which is easier and automation which is automate and reduce errors and scale it out one of the announcements that i wanted to have you share a bit about was i guess you would say it fits into the first two of those vectors it's the work that you've been doing around compilation uh can you talk about the sagemaker training compiler yeah you know it's one of the first times that really we are you know that machine learning practitioners are going to have going to be able to use a machine learning compiler this comes integrated with the versions of tensorflow and pytorch that are available in sagemaker so you know users of these popular packages will be able to use the compiler by default the compiler the way this came out is customers told us that these days the data scientists spend a lot of time optimizing models and there's a lot of craftsmanship involved in optimizing these models especially when you're looking at recent large natural language processing models with billions of parameters optimizing these can take weeks or months of effort and so we said okay how do we reduce this pain point for customers and you know we do think that compilation is going to become um the standard in how machine learning you know especially these large quality models are being run and so we built this machine learning compiler and it provides up to a 50 improvement in performance automatically you know you don't have to change your workflow because the compilation happens in the background and it does this by generating more efficient code for gpus it's able to use the gpus much more efficiently and you know and we think it is yet another tool that does the things you mentioned which is you know it's a tool that makes it easier for you to optimize models and at the same time you know you ultimately get better performance a lower cost because things just run faster and now this isn't aws's first uh first experiment or first you know product that's based on compiler a compiler neo uh had a compilation uh aspect to it based on tvm uh open source project is this one also based on tvm it is so it shares some lineage from neo uh and you know as you point out neo was meant for compilation and edge devices so the constraints there were more about how do i you know how do i reduce the memory consumption and how do i reduce the footprint so that it can it can be executed in edge devices in this case it's more about you know in the cloud so we don't really have the similar kind of constraints in terms of memory footprint and so on resource constraints it does use the similar intermediate form as tbm and we have also used xla for the front end so that we could compile down from tensorflow and pi torch on to the intermediate representation but there's a lot of you know heritage that they share got it and uh in terms of and the focus for this is specifically training as opposed to inference right um it's really training and i think customers will really get benefit when they use these large models like you know these recent large natural language processing models because especially when you know if you're training something for many hours that is when you know getting a 50 boost helps um so you know if you're doing let's say smaller models xgb boost or these you know regressions and so on which you know these days can run in minutes uh in powerful machines their customers won't see a lot of benefits it's really intended for these large models which are becoming very popular with customers these days and how much are you seeing folks training these large models from scratch relative to kind of transfer learning fine-tuning and using a pre-trained model we are so in in in let's say the gpt to kind of a gpt slash you know star kind of models um or you know whether it's robot or gpt star in these large models we do see customers training them from scratch and we have seen customers who take these models and training them on different corpus of text for example someone wanting to train it on a corpus of financial texts someone wanting to train it on the corpus of you know other other different kinds of texts so we do see that happen i mean transfer learning is also there we have also seen some customers uh start with a pre-trained model and then do transfer learning but but many of our large customers many of the startups are actually taking these large models and training them on different bodies of text to be able to create much more customized offerings got it uh you already mentioned the serverless inference announcement and the the new uh offering there um can you talk a little bit about the relationship between the sagemaker serverless inference and lambda to some degree folks have been able to do inference via aws's existing serverless tooling like lambda uh although with various limitations associated with that environment can you talk about how these fit together and and what's new with the serverless inference approach so this is a fully managed offering so you know you don't have to do any of that management the second is this also gives you a lot of the machine learning capabilities and tooling that you would expect so let's say things like you know this will give you things like model monitoring uh and we are also working on making it easier for customers to bring models in their own container formats so that is easier to use so it's really a combination of a fully managed offering giving you the machine learning capabilities that we think are really important when you're thinking of production deployments and then the third aspect is uh giving a lot more flexibility um and you know we are going to going to bring that up uh over the next year giving a lot more flexibility in how you create your containers that you can use to deploy on the server listen and so the what does the developer experience look like with serverless inference so there it sounds like they're providing a model artifact in some format and potentially as a container yes uh actually along those lines before i get to that one we are also releasing another feature that we call sagemaker inference recommender so think of it today as if you have to do model deployment then first and m and i'll get to your question around how what it works like the serverless inference so today what an engineer has to do is they have to figure out what is the right choice for me if i'm doing online inference there are more than 70 instances that you have to choose from or you do serverless inference and then once you have chosen it you need to configure it then once you have configured it you will need to run load tests on it so that you know that you'll be able to withstand the traffic and then once you have done that you may have to optimize the model for the instance so what sagemaker inference recommender does is you give it the model and you give it you know your use case requirements it will automatically recommend the right mo instance for you that's going to give you the optimal price and performance so as a first step you can actually come to the inference recommender and have it ah figure out which one to use now if serverless is the right option for you then you do it as you mentioned which is you know you basically give it the model artifacts sagemaker is going to provision an instance for you based on the needs of your model and then it's going to scale it up elastically up and down based on the traffic it will shut it down if there's no traffic it'll bring it back up whenever it sees traffic and you're only paying per use so you know there's no traffic means no charge got it so going back to inference recommender can you talk through how that's working is it um is it profiling a live instance of your model and trying to understand its resource requirements or is there some other so when you when you give it your model it will run a benchmark on different instances and then it will find out which one of those instance provides you the best performance or the lowest cost and then it will give you a recommendation that says you know this particular instance maybe inferential for example is giving you the best performance or the lowest cost and then you can go and choose it so it basically runs a number of tests and it also allows you the user to then use it to do load testing on a variety of other instances as well so at the back end there is a framework that is doing uh think of it as a bunch of performance measurements and cost measurements related on the set of instances and then lets you make the choice and i'm imagining that in a serverless environment that same kind of benchmarking that you're allowing customers to do with inference recommender to get kind of the either lowest cost or best price performance instances aws might want to do that to manage its own costs like is this an example of exposing functionality that existed kind of within the the cloud or is it its own thing in some ways it's be it's in response to what customers told us where some of the pain points in model deployment because customers said today we have to do this work ourselves and we have to build an infrastructure to do it for us what ha in our case it's a little hard in general to do it because you know customers own their own models and data and all that we have no knowledge of it so it's hard for us to look at that so this is a feature where customers are coming in and you know providing their inference containers and then we can go and make these changes but it's a little hard to do it in general when we don't really have any visibility into uh into their models and data and so on so much so far but there's uh at least one more uh sagemaker ground truth plus um so ground truth has been around for a few years now this is essentially a kind of a higher level of abstraction on top of mechanical turk for data labeling ground truth plus is a better version ground root plus gives you more yes so i guess we could have it's a plus plus so what happens today is as the need for data labeling has gone up and you know we have customers doing you know terabytes and petabytes of data as the need for data labeling has gone up many customers have teams of say data operations managers program managers who are responsible for producing high quality label data now today when you're using ground truth or even in general these customer teams have to set up the labeling workforce and then they have to validate the output of the label data then they have to manage some of these workforces and that can be daunting for some of these teams and so what ground truth plus does is it gives you a fully turnkey operation so you as the customer you give of ground truth plus the data and then ground truth plus does all the rest it looks at your instructions so let's say you want your data to be labeled by people who are experts in video labeling ground truth plus will select labelers who are good at video labeling it will then do the or do the validation of the label data it will do the management of the labeling workforce and then not just that ground truth plus embeds machine learning into the data labeling process so we are using machine learning to do data labeling for machine learning and what it does is it uses machine learning models to automate the labeled output it also uses machine learning models sometimes to pre-label the data and what this means is human labelers no longer have to label it they just have to verify that the labels were done correctly and that makes the labeling process much more efficient reducing costs by up to 40 percent and so what we are doing in here is basically removing the entire friction from the data labeling process because you just give us the raw data and we hand you back the finished label data awesome awesome well as is always the case at re invent there is a lot to review and absorb and kind of wrap uh one's brain around uh for you rothen what are you most excited about among all of these uh new offerings i think studio lab is going to be really impactful in making machine learning accessible to a lot more students experimenters someone who wants to quickly get something done i think canvas will be a game changer in making machine learning accessible to a lot more people and so going back to the initial thing that you said that this is along the democratization make machine learning accessible to more people that vector i think on the make machine learning easier for machine learning developers i'm excited by what the model compiler will do because you know i think you know what time will we are going to expand the amount of usage so those are really important you know ground truth plus also makes it much easier to do data labeling and you know data is the fuel for all of the machine learning uh and you know we keep innovating on performance and costs so i think those are the things that i think customers will find really helpful i think you just said everything's your favorite well it's hard to pick between so many things but you know i think if i had to pick the top three i would pick studiolab and canvas uh and then on the technology side i think the model compiler because i think those are really going to push the frontier in new ways awesome awesome well bronton thanks so much for uh getting us all caught up on aws's announcements thank you sam it was really nice talking to you youall right everyone welcome to our annual coverage of the aws re invent conference i'm here today with bratan saha broughton is vice president and general manager of ai and ml at aws broughton welcome to the twiml ai podcast thank you so i'm glad to be here thanks for having me absolutely let's just jump right in i'd love to have you share a little bit about your role and background with our audience so i'm vp for aiml at aws that includes a bunch of our ai services things like personalized forecasting health ai services hml services like panorama and so on ai devops where we use ai to make it easier to run devops or for software development and then that includes the middle layer of the aws services which is sagemaker which is a flagship machine learning service as well as the ml frameworks and infrastructure where we provide both custom hardware gpus as well as machine learning frameworks and then prior to coming to aws i was at nvidia and then prior to that at intel this year as has been the case in years past aws has a lot to announce at reinvent on the ai and ml front uh one of the things that i've noticed is that the announcements are catering to two different communities this year uh there's a set of announcements focused on making ml and ai more accessible on the aws platform but there's also a set of announcements that seem to be aimed at more complex use cases more mature users and i thought maybe a good place to start our discussion is hearing kind of how you're thinking about the the market uh and and where we are and the context in which aws is releasing these new offerings i know one of the ways you think about that is in terms of industrialization of machine learning share your perspective on that please thank you sam yes so it's indeed true that you know we have been releasing and we are going to be releasing features in terms of two customer personas one of them as you suggested is the machine learning practitioner and our goal there is to make it easier and easier to build machine learning models and train and deploy machine learning models and to set some context uh this year in 2021 we have customers who many of whom want to deploy a million models each and they want to be able to train models with billions or tens of billions of parameters now think of it you know if you go back three years let's say when we launch sage maker at that point of time the state of the art model used to have like 20 million parameters and now we are talking of tens of billions of parameters maybe even more than 100 billion parameters so that's the scale at which machine learning is growing and then we have customers who want to do hundreds of billions of predictions per month and if you look at a data labeling services you know we're labeling more than a million objects a day and so what that has meant is that machine learning is no longer really a niche you know it is something that today on aws more than a hundred thousand customers are using it and you know more machine learning happens on aws than anywhere else so we see kind of the need for industrialization potentially ahead of others and so uh what we are trying to do in here and what customers want us to do is to make it easy to deploy machine learning at scale and automate it and make it repeatable so that you know there are no errors in it so that's the industrialization front which is how do we make it easier for people to use and deploy machine learning then there's the other vector which you also pointed out which is that the demand for machine learning practitioners is growing much faster than people companies can hire them you know there's a there's a survey by a third party that said the demand for machine learning and ai practitioners has been growing by 74 annually for the last four years you know 2x more than any other emerging job category and so many of the customers told us that you know can you make it easier for us to have more employees do machine learning you know not just data scientists and ml practitioners but others as well like data analysts like marketing and sales professionals and these are employees who use the data you know who understand the data who would benefit from these machine learning insights and so that's the other one where we are going in and saying how do we fundamentally change the paradigm and make it possible for people who may not be machine learning practitioners who may not have software coding skills yet do machine learning and so that is where we are doing a lot of no code low code tools and that is where sagemaker canvas comes in and so it's really about two things one is democratizing access across many more employees through no code low code machine learning tools the other is making it easier and easier for machine learning practitioners to build and deploy machine learning at scale because you know the the amount of deployment going on has really increased a lot by orders of magnitude awesome let's uh let's maybe jump right into uh some of the announcements you mentioned canvas and that is a no code environment tell us a little bit more about the kind of the design center of canvas and and what you're going for there yeah and you know this is a time when we are taking sage maker to a new audience up until now we were looking more at ml practitioners ml ops engineers uh software developers and this is going to an audience like analysts and sales and marketing professionals who don't want to necessarily be doing coding but who still want to be using machine learning and so canvas is we completely redid the ux we completely redid the ux so that you know it starts from the use case you know you may want to do churn prediction you may want to do sales forecasting and so on so it starts from the use case you don't have to write a single line of code you know you can access data sources multiple kinds of data sources on-prem or in the cloud and then canvas will automatically do the data transformations that you need to get done it will automatically build the right model for your use case it will help you deploy the model all of this without writing a single line of code and then it will also explain to you why it's making certain predictions so for example let's say you want to use it for churn prediction it will tell you you know here are the reasons why i think this customer matures so you are it's not just giving you insights it's giving you actionable insights because it tells you why it's doing something and all of this as i you know emphasize without writing a single piece of single line of code so that we can make it really accessible and i'm really excited about how canvas will make machine learning accessible to a lot more employees now the other innovation that canvas brings is that it's very easy for an analyst or a user to export all of the work that they have done on canvas onto sagemaker studio and so think of it as you know you're building your machine learning models and all that but you can again have an expert data scientist look at everything that has been created it's almost like say in the software industry in your writing code you want someone to code review before you push the code into production and so that is another of the key innovations we bring in here is we increase the collaboration between an analyst user as well as a data scientist i was going to ask about that last point uh traditionally with uh no code environments you know it's often perceived by technical folks as being a bit of a trap in that you um you can do a lot of configuration and get from say zero to one or zero to a half even you know just clicking around on the screen but then when you want to get to you know the next level you end up running up against the wall it sounds like here canvas is a front end that's building the code artifacts on the back end and i can take those and extend them and build on them is that that the case it is exact it's a front end it's using the same back end that sagemaker uses it's the same back end that's that customers are actually using to you know train models with these billions of parameters and deploy millions of models and canvas is going to export the source code for everything it generates so it's going to give you the source code for the models for the data preparation routines and so on so a data scientist can then use that as a starting point and refine it further when you think about about canvas and the process of delivering that as a product what were some of the key challenges in uh in delivering it you know the important thing that we wanted to keep in mind was how do you raise the level of abstraction so that it's easy to use while at the same time not making not raising the level of abstraction so much that you can't do useful stuff with it and that's often the trap as you very rightly pointed out sam that's very often the trap that you fall in with these no code loco tools the second one was how do you design the ux in a way such that you convey the central concepts but you don't expect the user to have a deep understanding of machine learning like you know if you were doing this on sage maker studio you would you know you would have a graph and you say well this is the auc the area under the curve and so on now you can't really use those terms for someone who may not be a deep practitioner of machine learning uh similarly explainability you know you want to be able to explain in a right way that okay this was the input this is the input that's contributing most to the output so it's really kind of you know threading the needle so that you have raised the level of abstraction and made it really easy to use while at the same point you still allow interesting things to be done and you're explaining all of the concepts in such a way that you're not overwhelming the user with machine learning knowledge and is there an example of uh one of those trade-offs that you made in in the product and how it played out yeah so you know one of the things for example is um if you think about a normal you know it's kind of a sage maker studio kind of an environment where you have a notebook and a notebook is an interactive environment for doing machine learning so you know you do get an interactivity there and you know what's going on in the case of canvas we couldn't give that interactive environment like a notebook it has to be much higher level and what that means is that your back end needs to be much more snappy now because you know you can't really take a few hours to train your models you need to be able to give quick feedback you need to have indicators that says how much of the job has progressed you need to be able to um explain it at an at a you know much more intuitive level what your data sets look like what kind of you know what kind of data permutations are allowed to do um what kind of filtering you're allowed to do so we really had to rethink from the perspective of a very different kind of user and i imagine given the description you just gave that the target use cases are kind of those core uh structured data types of use cases as opposed to you know computer vision or nlp or things like that yes it's really more of structured data tabular data you know sales forecasting churn prediction fraud detection that kind of stuff like you know things that a data analyst or you know sales professional marketing professional is using all the time so it's really meant for tabular structured data awesome along those lines targeting the this kind of democratization end of the the user spectrum if we can say it thusly um you are also announcing sagemaker studio lab uh tell us about that offering yeah i'm really excited by what uh sagemaker studio lab will enable students and other experimenters who want to quickly get started with machine learning so sagemaker studio lab offers usage maker notebooks it's integrated with github so it makes collaboration easy it's pre-packaged with all the popular machine learning tools so that you know student or an experiment or for that matter any user can quickly get started it gives you free compute and it gives you free storage and more than that you don't have to do things like you know shutting down your instance or saving your work saving your models after your work is done studio lab does it for you so it's as easy as you know closing your laptop and then coming back to it and resuming your work again and it's no setup no charge what that means is you don't even need an aws account to get started you can just use your email address to log into studiolab and get started with it that's a change for aws it's uh it's a new kind of a product we think you know we think machine learning our customers tell us that they really want us to make machine learning much more easily accessible and so we really looked at what are the friction points and you know there are a lot more we believe there'll be a lot more people wanting to do machine learning tomorrow than there is today and so it's really important for us to help customers leverage machine learning just as we at amazon have been doing it for a long time and so we think it's really important to make it easy for students experimenters other customers to get started so it's it's indeed a pretty groundbreaking game changing product there are other lab offerings out in the market and they are typically limited by the amount of time you can access them or the the type of compute that you can get access to what are the constraints with uh with studiolab so we have looked at we have talked to customers and we have looked at the typical usage scenario and the kind of limits that we have we believe are more than sufficient for the typical usage scenarios so you can use both cpus and gpus and you know there's a time limit for it well was at launch but you know those should capture the majority of the usage uh and you know you get not just cpus but you get gpus as well and the other thing that i'll mention is it comes with you know there is encryption at rest encryption at transit so you know it comes with really good security properties and that is really hard to build because you know you're building this thing where you expect a lot of people to use it because of the nature of the product at the same time you want to make sure that this is something that maybe a scientist at an enterprise is comfortable using it and so we have to take a lot of care to kind of marry these two paradigms in the sense that you know you just don't want to throw out an aws product that doesn't have the highest levels of security and then the other thing that studiolab in my mind does really well is it's not two different islands it's not that sage maker studio lab is a separate island and sage maker studio is a separate island and sagemaker studios you know the enterprise class you know the the most popular machine learning platform today for enterprises when you're using studiolab you can actually easily transfer all of your work to studio okay and so you know you can get started you can do your training with a small data set if you want you know do some experiments and then you say now let me actually do it in a much larger data set or let me use a much larger model you can just export all of your work to studio and then get all of the features that sage maker provides so i think that you know that thing of uh providing still providing good enterprise class security guarantees a while at the same you know so we didn't make it two different islands while at the same time giving it all of the properties of free compute free storage easy to use easy to get started no aws account needed that needed a lot of innovation by the teams is there a free compute component to canvas or can you export from canvas to studio lab to do training so we have a free tier in canvas as well and we have looked at how customers would want to get used to it and see the power of the tool so there is a free tier component of canvas that customers can get started with we haven't yet had customers really ask for an export from you know canvas to studiolab they have asked us for canvas to studio so we are we are going to be releasing that but you know if customers want it we'll do it maybe one last question on the topic of democratization um yeah i've been going to re invent for a long time uh and one of the hallmarks of early reinvent was the cost of compute would drop from year to year to year um do you see the same thing happening on the machine learning side you know obviously there's a relationship between democratization and and the cost of accessing services like these indeed yes that's a great question so um we are doing a lot of hardware and software optimizations to pull down the cost of compute at the hardware level first if you start with inferential inferential provides customers with up to 70 percent lower cost uh than previous generation and comparable gpu instances and then you know we are we also launched trainium in preview and trainium is going to provide the best price performance for training machine learning models in the cloud and then if you look at our for inference we have a g5 instances and the g5 instances are also much more powerful than the previous generation g4 dn instances and the same with on the machine learning training side same with the p4d instances you know which are up to 60 lower cost than the previous generation p p3 p3dn instances so there's that aspect of reducing the hardware infrastructure cost by continuously innovating you know bringing out new gpus bringing out new custom accelerators then there is the software optimizations that add on top of it okay and when you think of that for example we are releasing serverless inference and what serverless inference does is it gives you paper use so there are many use cases where say think of a restaurant recommendation from an online food delivery site you know where you have your workload is very intermittent and so customers don't want that instance to be running all the time at that time and so today they built complex workflows to shut down their instances so we are bringing out sagemaker serverless inference that will automatically provision instances for you scala scale it up and down based on your traffic and then shut down the instance so if there's no traffic you pay nothing then there are these other use cases where customers are going for what i would call you know hyper personalization where they deploy lots of models maybe up to million models or more where they want to be able to really customize it to their end users needs and then they say well if i'm going to deploy so many models i don't want to allocate so many instances and so we have features like multimodal endpoints which let you host up to a million models in a single instance and then we do all of the work of caching things in and out so the combination of both hardware infrastructure optimizations and software optimizations should continue to give customers what they're used to which is the cost of compute the cost that they're paying for compute going down just as it has in the past yeah that's a great segue into digging into some of the more complex use cases that you reference customers with a million models and i think the first question i have on that is can you you know are there patterns that you see to the customer journey you know relative to you know where they start and how they get to having a million models what does that typically look like or what are you know there are some examples that you can share of the trajectory for customers that make it there you know in typically customers will initially start with um not very complex use cases you know something that has concrete business value uh but at the same time isn't kind of at the bleeding edge of the spectrum and you know the initial parts is really to get your data in order so you know have your make sure you have enough data clean data you have you know data lake and all that set up then pick a significant use case that demonstrates clear business value yet is not that difficult and that is where we typically see customers start their proof of concepts and once they have deployed you know few models a dozen models then they just scale it out uh to you know thousands of models tens of thousands of models and even more are there inflection points along the way that are kind of markedly different you know as you as you pass them i think what happens is as you get beyond a certain point it's no longer just the machine learning or the model data aspect of it that is important at that point your processes and your mechanisms and your automation become equally important and it's a lot like say software you know and software if someone is writing small little pieces of code it's one thing but once you get into production massive deployments you need your ci cd you need your automation you need your tools all of which is needed to make sure that the process of creating whether it's software whether it's models what have you that the process of it is repeatable automated reduces errors and so what we have seen is you know the initial few model deployments initial few proof of concepts you can do them in a little bit of an ad hoc manner and you know many customers can do it because they want to go fast and they want to be experimenting with it but sam as you pointed out you get to a point when that is no longer sustainable and so that is when you really need to look at what are my ml operational practices you know or mlops what are my ml operational practices how am i automating the end to end workflow how am i looking at audit and governance what is my security posture all of these come into the picture then and so at that point you really have to make sure you put in all of these and that is what really helps you scale you referenced this as uh industrialization earlier and just thinking about the list that you just rattled off of concerns you know operationalization and ml ops was just one of those things you think of industrialization as a superset that includes mlaps or are they more or less anonymous so we look at industrialization as a superset of ml ops and we really look at industrialization as three different vectors one of them is the infrastructure and as you mentioned you know the cost of the infrastructure the performance of the infrastructure has to keep getting better like if you look at let's say the machine learning models you know three years back when resnet was the state of that model you were looking at about 20 million parameters now you're looking at you know gpt class of models which are talking about billions of parameters and so you need your compute infrastructure and the software infrastructure around it to keep getting faster so that you can do these things so one vector for ml industrialization is performant and cost effective infrastructure the second aspect of it which we look at is tooling you know these are tools that are custom built purpose built for machine learning so that they can reduce the heavy lifting in all of your machine learning tasks and these are you know machine learning development environments like machine learning ides you know machine learning project management systems you know machine learning bias detection tools and then debuggers and profilers that are customized for debugging and profiling machine learning models so all of these tools that help it easy that make it easier for the developer to be more productive and churn out a lot more and then there's the third aspect of it which is the automation aspect of it which is what mlofsc so we really look at it as infrastructure which is faster and cheaper tooling which is easier and automation which is automate and reduce errors and scale it out one of the announcements that i wanted to have you share a bit about was i guess you would say it fits into the first two of those vectors it's the work that you've been doing around compilation uh can you talk about the sagemaker training compiler yeah you know it's one of the first times that really we are you know that machine learning practitioners are going to have going to be able to use a machine learning compiler this comes integrated with the versions of tensorflow and pytorch that are available in sagemaker so you know users of these popular packages will be able to use the compiler by default the compiler the way this came out is customers told us that these days the data scientists spend a lot of time optimizing models and there's a lot of craftsmanship involved in optimizing these models especially when you're looking at recent large natural language processing models with billions of parameters optimizing these can take weeks or months of effort and so we said okay how do we reduce this pain point for customers and you know we do think that compilation is going to become um the standard in how machine learning you know especially these large quality models are being run and so we built this machine learning compiler and it provides up to a 50 improvement in performance automatically you know you don't have to change your workflow because the compilation happens in the background and it does this by generating more efficient code for gpus it's able to use the gpus much more efficiently and you know and we think it is yet another tool that does the things you mentioned which is you know it's a tool that makes it easier for you to optimize models and at the same time you know you ultimately get better performance a lower cost because things just run faster and now this isn't aws's first uh first experiment or first you know product that's based on compiler a compiler neo uh had a compilation uh aspect to it based on tvm uh open source project is this one also based on tvm it is so it shares some lineage from neo uh and you know as you point out neo was meant for compilation and edge devices so the constraints there were more about how do i you know how do i reduce the memory consumption and how do i reduce the footprint so that it can it can be executed in edge devices in this case it's more about you know in the cloud so we don't really have the similar kind of constraints in terms of memory footprint and so on resource constraints it does use the similar intermediate form as tbm and we have also used xla for the front end so that we could compile down from tensorflow and pi torch on to the intermediate representation but there's a lot of you know heritage that they share got it and uh in terms of and the focus for this is specifically training as opposed to inference right um it's really training and i think customers will really get benefit when they use these large models like you know these recent large natural language processing models because especially when you know if you're training something for many hours that is when you know getting a 50 boost helps um so you know if you're doing let's say smaller models xgb boost or these you know regressions and so on which you know these days can run in minutes uh in powerful machines their customers won't see a lot of benefits it's really intended for these large models which are becoming very popular with customers these days and how much are you seeing folks training these large models from scratch relative to kind of transfer learning fine-tuning and using a pre-trained model we are so in in in let's say the gpt to kind of a gpt slash you know star kind of models um or you know whether it's robot or gpt star in these large models we do see customers training them from scratch and we have seen customers who take these models and training them on different corpus of text for example someone wanting to train it on a corpus of financial texts someone wanting to train it on the corpus of you know other other different kinds of texts so we do see that happen i mean transfer learning is also there we have also seen some customers uh start with a pre-trained model and then do transfer learning but but many of our large customers many of the startups are actually taking these large models and training them on different bodies of text to be able to create much more customized offerings got it uh you already mentioned the serverless inference announcement and the the new uh offering there um can you talk a little bit about the relationship between the sagemaker serverless inference and lambda to some degree folks have been able to do inference via aws's existing serverless tooling like lambda uh although with various limitations associated with that environment can you talk about how these fit together and and what's new with the serverless inference approach so this is a fully managed offering so you know you don't have to do any of that management the second is this also gives you a lot of the machine learning capabilities and tooling that you would expect so let's say things like you know this will give you things like model monitoring uh and we are also working on making it easier for customers to bring models in their own container formats so that is easier to use so it's really a combination of a fully managed offering giving you the machine learning capabilities that we think are really important when you're thinking of production deployments and then the third aspect is uh giving a lot more flexibility um and you know we are going to going to bring that up uh over the next year giving a lot more flexibility in how you create your containers that you can use to deploy on the server listen and so the what does the developer experience look like with serverless inference so there it sounds like they're providing a model artifact in some format and potentially as a container yes uh actually along those lines before i get to that one we are also releasing another feature that we call sagemaker inference recommender so think of it today as if you have to do model deployment then first and m and i'll get to your question around how what it works like the serverless inference so today what an engineer has to do is they have to figure out what is the right choice for me if i'm doing online inference there are more than 70 instances that you have to choose from or you do serverless inference and then once you have chosen it you need to configure it then once you have configured it you will need to run load tests on it so that you know that you'll be able to withstand the traffic and then once you have done that you may have to optimize the model for the instance so what sagemaker inference recommender does is you give it the model and you give it you know your use case requirements it will automatically recommend the right mo instance for you that's going to give you the optimal price and performance so as a first step you can actually come to the inference recommender and have it ah figure out which one to use now if serverless is the right option for you then you do it as you mentioned which is you know you basically give it the model artifacts sagemaker is going to provision an instance for you based on the needs of your model and then it's going to scale it up elastically up and down based on the traffic it will shut it down if there's no traffic it'll bring it back up whenever it sees traffic and you're only paying per use so you know there's no traffic means no charge got it so going back to inference recommender can you talk through how that's working is it um is it profiling a live instance of your model and trying to understand its resource requirements or is there some other so when you when you give it your model it will run a benchmark on different instances and then it will find out which one of those instance provides you the best performance or the lowest cost and then it will give you a recommendation that says you know this particular instance maybe inferential for example is giving you the best performance or the lowest cost and then you can go and choose it so it basically runs a number of tests and it also allows you the user to then use it to do load testing on a variety of other instances as well so at the back end there is a framework that is doing uh think of it as a bunch of performance measurements and cost measurements related on the set of instances and then lets you make the choice and i'm imagining that in a serverless environment that same kind of benchmarking that you're allowing customers to do with inference recommender to get kind of the either lowest cost or best price performance instances aws might want to do that to manage its own costs like is this an example of exposing functionality that existed kind of within the the cloud or is it its own thing in some ways it's be it's in response to what customers told us where some of the pain points in model deployment because customers said today we have to do this work ourselves and we have to build an infrastructure to do it for us what ha in our case it's a little hard in general to do it because you know customers own their own models and data and all that we have no knowledge of it so it's hard for us to look at that so this is a feature where customers are coming in and you know providing their inference containers and then we can go and make these changes but it's a little hard to do it in general when we don't really have any visibility into uh into their models and data and so on so much so far but there's uh at least one more uh sagemaker ground truth plus um so ground truth has been around for a few years now this is essentially a kind of a higher level of abstraction on top of mechanical turk for data labeling ground truth plus is a better version ground root plus gives you more yes so i guess we could have it's a plus plus so what happens today is as the need for data labeling has gone up and you know we have customers doing you know terabytes and petabytes of data as the need for data labeling has gone up many customers have teams of say data operations managers program managers who are responsible for producing high quality label data now today when you're using ground truth or even in general these customer teams have to set up the labeling workforce and then they have to validate the output of the label data then they have to manage some of these workforces and that can be daunting for some of these teams and so what ground truth plus does is it gives you a fully turnkey operation so you as the customer you give of ground truth plus the data and then ground truth plus does all the rest it looks at your instructions so let's say you want your data to be labeled by people who are experts in video labeling ground truth plus will select labelers who are good at video labeling it will then do the or do the validation of the label data it will do the management of the labeling workforce and then not just that ground truth plus embeds machine learning into the data labeling process so we are using machine learning to do data labeling for machine learning and what it does is it uses machine learning models to automate the labeled output it also uses machine learning models sometimes to pre-label the data and what this means is human labelers no longer have to label it they just have to verify that the labels were done correctly and that makes the labeling process much more efficient reducing costs by up to 40 percent and so what we are doing in here is basically removing the entire friction from the data labeling process because you just give us the raw data and we hand you back the finished label data awesome awesome well as is always the case at re invent there is a lot to review and absorb and kind of wrap uh one's brain around uh for you rothen what are you most excited about among all of these uh new offerings i think studio lab is going to be really impactful in making machine learning accessible to a lot more students experimenters someone who wants to quickly get something done i think canvas will be a game changer in making machine learning accessible to a lot more people and so going back to the initial thing that you said that this is along the democratization make machine learning accessible to more people that vector i think on the make machine learning easier for machine learning developers i'm excited by what the model compiler will do because you know i think you know what time will we are going to expand the amount of usage so those are really important you know ground truth plus also makes it much easier to do data labeling and you know data is the fuel for all of the machine learning uh and you know we keep innovating on performance and costs so i think those are the things that i think customers will find really helpful i think you just said everything's your favorite well it's hard to pick between so many things but you know i think if i had to pick the top three i would pick studiolab and canvas uh and then on the technology side i think the model compiler because i think those are really going to push the frontier in new ways awesome awesome well bronton thanks so much for uh getting us all caught up on aws's announcements thank you sam it was really nice talking to you you\n"