**Transfer Learning and Deep Learning for Yoga App Development**

In this article, we will explore the concept of transfer learning and its application in developing a yoga app. Transfer learning is a technique used in deep learning where a pre-trained model is fine-tuned on a new dataset to adapt to a specific task or domain. This approach can be highly beneficial in developing an app that requires complex computer vision tasks, such as detecting poses in yoga images.

One of the strategies we can employ for transfer learning is to take an existing convolutional network and retrain only the top layers on a new dataset. In this case, we will use pre-trained models available on GitHub, specifically "Smells Like ML" which has already retrained its model on a large collection of yoga images. By taking advantage of this pre-trained model, we can speed up the development process and reduce the computational resources required.

The code repository for this project is hosted on GitHub, where we have created a document detailing the steps to integrate the pre-trained model into our app. The first step involves replacing the existing TF light file with the yoga net TF light file, which is downloaded from the GitHub repository. Once the new model is integrated, we can proceed with integrating text-to-speech functionality into our app.

**Text-to-Speech Integration**

To add a voice assistant to our app, we will use the "TTS" plugin available on pub.dev. This plugin allows us to create a text speech object and have it speak out loud. We will install this plugin in our pubspec.yaml file and then integrate it into our app.

The TTS plugin provides an example of how to use it in code, which we can follow to add the functionality to our app. Once installed, we can access the plugin's features by creating a text speech object and using the `speak()` method to have it speak out loud.

One of the key features of this plugin is its ability to delay the speech by a certain number of seconds, which allows us to control when the voice assistant responds. We can also use event listeners to detect specific poses and trigger the voice assistant's response accordingly.

**Guiding the User through Yoga Poses**

The ultimate goal of our app is to guide users through yoga poses in real-time, using the pre-trained model and text-to-speech functionality. To achieve this, we will create multiple event listeners for different poses, which will trigger when a pose is detected by the camera. We will also add a timer feature that will delay the next pose until a certain threshold is reached.

Once the app detects a pose, it will speak out loud using the text-to-speech plugin, providing feedback to the user. This creates an immersive and interactive experience for the user, as they can move through their yoga practice with guidance from our voice assistant, Macy.

**Improving the App**

There are many ways to improve this app further. One potential feature is to create a generative model that can generate 3D yoga instructors in real-time. This would allow us to add a new level of realism and interactivity to the app. Another idea is to incorporate augmented reality, which would enable users to see their instructor or other virtual elements alongside them.

We could also add more personalization options, such as allowing users to choose specific poses or instructors that suit their needs. Additionally, incorporating sensors for diet tracking and analytics could provide a comprehensive platform for supporting yoga practice.

While this is just the beginning of our app development journey, we hope that this article has provided valuable insights into transfer learning and deep learning techniques used in developing such an app.

"WEBVTTKind: captionsLanguage: enhello world it's Suraj and I've built an AI yoga instructor her name is Amy and she lives in this app that I built in flutter and flutter allows me to build an app for both iOS and Android in the same codebase which is incredible let me show you a quick demo first so I'll log in with my username and password and once I've logged in this is my dashboard she's gonna show me my current level of you know at which stage in the journey of yoga I'm at and once I have logged in and assume that I've paid as well you know you have to pay 100 US dollars a month to use this on-demand automated yoga instruction service and assume that I've paid once I've paid and by the way I've integrated not just stripe but Razer pay as well so India you guys are involved in this as well we're gonna go global with this as soon as you have paid then you can begin this service now I just want to say I'm using my webcam so I can't show my full body here but I just want to show you the pose detection that's happening right now it's gonna be kind of cramped but let's let's start a demo right here so as you can see it is detecting my pose and so this is the warrior pose and once she detects my pose she's gonna tell me to hold that pose for a little bit and once I've held that pose for a little bit after a while she will tell me to go into the next pose and so she's looking at which pose I'm doing and then she's matching that with what she understands about yoga which I've trained over time so I'm gonna show you how to do that so let's jump at it let's X out of this or right back to the dashboard now you might be thinking well why don't you just build a business out of this why are you showing us how to do it why do you have the code for what you built in the video description for everybody to see because it's more impactful for me I believe to give you the building blocks to show you to give you the path on how to gain financial freedom that means you don't work for anybody else you work for yourself either as a freelancer or you start your startup and the reason I'm saying this so strongly at the beginning of the video is because anybody can become rich using this technology anybody can build wealth by taking this internet technology and this laptop computer wherever you are in the world and in the age of infinite leverage if you make the proper decisions in terms of how to architect an app how to design the app how to perform business development to get your first clients you can build a sustainable business no matter where you are in the world and you can gain financial freedom by making your life better and the lives of others so it's a win-win-win situation I want you to gain financial freedom and that's why I am not just showing you all the possibilities of AI in healthcare I am showing you a practical example and I'm gonna show you from start to finish that's my promise in this video I'm gonna show you from start to finish how exactly I built this and make sure that you understand every step of the way so you can either take this code and turn it into a profitable business I will rejoice in your success or you have a different but related idea and you can start working on that so let's be very clear about what this video is about there are three learning objectives in this video the first is how to create a simple flutter app okay flutter is awesome it was just released a few months ago and with one code base I had to learn the dart programming language with one code base you can build an app for both iOS and Android that's super cool that's the first the second is I'm gonna show you how a pose net model is able to detect poses what the demo you just saw was and lastly I will show you how to perform transfer learning this is a hot word in the community these days because it's so popular especially in computer vision and I'll show you exactly how to do that to take an existing convolutional network and train it to detect yoga poses those are the three learning objectives alright in addition to everything else but those are the three main ones that I'm gonna leave you with by the end of this video that's my promise to you so don't go anywhere I'm also gonna wrap in this video I've got some memes we're taking this education stuff to the next level I am just getting started alright so let's just get started with this what are the tools that I use to build this well first of all I use the flutter right for ENSO flutter allows you to make an app for ios and android and kind of web as well which is insane but there's still that's that that part is really buggy so we're not gonna look at that web part we're just gonna focus on the ios and android part okay I also used firebase which is a Google server to store that data in the cloud you know user ID what is my yoga score you know different things like that social features I'm using stripe for payments in the West and I'm using Razer pay for payments in India you know it's really it is really really sad that there is not a global payments service yet it's just insane and that's why cryptocurrency is amazing it's insane how much Bitcoin is rallying right now I'm seeing I'm seeing the pulse of crypto rise these days so I'm gonna keep an eye on that so don't worry alright so what else that I've used I use tensorflow Lite to take this giant pose Nats model and compress it into something that could fit on a mobile device very light on device inference amazing stuff I used the pose net model as you're seeing these guys here super cool stuff this is called multi pose detection and I use an image data set of different yoga poses to perform transfer learning with an existing model I'm gonna show you how all that works and then one more thing that I used was a text-to-speech system to give the assistant a voice okay so there are a lot of opportunities in the healthcare space specifically using AI to improve the healthcare space I mean this graph is from the Harvard Business Review a very reputable magazine ten AI applications that could change healthcare look at all these now this is predicting the potential annual value by 2026 if you don't know how to code if you have never coded in your life and you start today by 2026 you could have a billion dollar business okay or whatever currency you use by 2026 all of these markets these sub markets of healthcare are posed for massive growth from fraud detection the virtual nursing assistant to preliminary diagnosis electronic health records are terrible these are archaic you know Windows 92 esque systems that all doctors use and those could easily be upgraded to internet status find a doctor talk to them about their problems and create a solution using AI technology the second idea I'm giving you ideas so you know make sure that you this this is in the code by the way this is in the code repository all these ideas the second is to create a some kind of new smart healthcare device you know doctors have a bunch of sensors like what is your heartbeat what is your galvanic response what is your sweat you know all these things if we could create a device that learns from this stream of data in real-time and adapts to the user maybe making the bed a little more comfortable or you know increasing the IV drip based on heartbeat etc we can free up nurses time and time is very valuable and so one more thing is that the smartphone can be an incredibly powerful diagnostic tool right you know you don't always need a doctor that was that had four years of college four years of medical school and then residency just to detect whether or not you have you know some kind of skin lesion or something like that here's an incredible statistic more radiologists work in the half dozen hospitals lining the renowned Longwood Avenue in Boston than in all of West Africa this is your opportunity to create a business that has huge impact a I in healthcare has huge impact I am NOT the only person to have said this this is a huge opportunity okay to serve the underserved and to make money doing it right that's a win-win situation that's the kind of person we need in the world more of and that's what what I'm trying to create by creating these free tutorials I will never charge for these to talk I'm just so hype right now I will never charge for these tutorials because it it goes against my my morals I will show you how to do this just continue to watch and I will show you the code I'll show you everything now Accenture says the same thing alright so so you might be thinking okay so where did where do I begin right I get I have an idea but I've never coded before where do I even start there's this great website called github.com and so if you see any and all coders upload their code to this website github.com and the coding culture developer culture is beautiful in that we share everything we share everything a lot of the times and so if you have some kind of idea let's say medical imaging or you know electronic health record databases or something just search it on github and so in this case I searched medical imaging and I got 1,500 results for different applications of AI or any kind of coding technology to medical imaging these are unused tools there is so much uncaptured value on github the reason I say there uncaptured is because most people don't understand code and they don't understand the complexity but if you are able to understand this complexity you can build you can provide society with value by pay walling this and providing a service that people can actually use making it make it easier to use then you know doing pip install and that's the complexity that you're taking on by doing pip install you know by you know installing dependencies and docker izing it and all these things that is a complexity that you can take care of and make money from okay so let's get started with this app the first step for us is to design this app so I'm gonna break out the white board get ready for this okay so here is my white board and by the way yes I'm sweating I don't care I'm here to help you learn so ignore the sweat let's just keep going with this so how are we gonna design this app well we know that we need a login page okay so this is the login page log in and in that login page we'll have a user in the password text field and let's say you know I want to anthropomorphize this app anthropomorphize means I want to make it seem like it's human so to do that I'm gonna put a picture or a gift of some girl or woman here who represents the yoga instructor and let's call her Macy okay so we'll have the title of the app right there and once I have a login to two text fields for user and password then I'm going to have a login button and a sign-up button okay so the signup button we can basically just reuse the code you know that created this login page to add some more text fields for user password repeat your password maybe the email and then sign up right so sign up button and then we just go right back so we're gonna have an arrow in both directions so that's login that's sign up okay but assume that you know I've already signed up and now I've logged in what's the next step well it's prop oh my god clearly I'm a geometry okay so what's the next page oh my god I'm so hyped to teach I am so hyped to do this I love my job I love this so much okay here we go the dashboard the dashboard is where the stats show up so we want to we want to gamify this app so we'll say this is gonna be some scalar value between 0 and 10 that will store in the database that shows where you are in your yogic journey okay we'll also have the image of this you know woman again just to continue this train of anthropomorphize a ssin I'll have a button to pay and they don't have a button to start so begin now for the pay button I'll have some sort of like modal pop-up you know for CVV credit card code your you know your month date whatever and then we'll go back to this once I begin here is the good stuff so first of all I want a nav bar so I can go back I want a nav bar so I'll go back so back button now inside of this that's where I want the model to be running inference the camera is on I will be standing here and it will be performing inference on me so it's gonna detect my pose so what I'll actually do is I'm going to put this phone up against the wall and then I'll just be by myself and I'll be doing yoga and so what's happening here is pose net is running it's detecting my pose because I've retrained it on yoga poses from a yoga image data set it's gonna be able to know what a certain pose looks like in terms of a yoga pose right downward-facing dog the warrior pose lotus position it's gonna know and so if it detects you know any certain pose and let's just say we have an array of poses and so it wants certain poses in a sequential order we'll name it 0 through 10 let's say 10 poses per session hold each pose for some threshold let's say 5 minutes it's gonna detect those poses and once it detects a certain pose we'll have an event listener that's basically just reading in the stream of X Y coordinates and once it detects a certain pose once that classification returns which it's going to automatically in real-time then we'll say ok if pose equals you know whatever value we'll set those 10 values from the array it pose equals x then we'll set some counter variable like int counter equals 1 and then once that counter is 10 we'll know we're done and then it posts equals x move on to the next pose so next and then we'll also have the text-to-speech engine which is her Maisie will say good job or something you know encourage the user now there is a lot we can do here and arguably this is the this is the value of our app right this is the hard part right here like combining pose net with this text-to-speech engine and doing so in a comfortable gentle yoga instructor s Quay so that so we'll get to that but this is the basic structure of the prototype and as you can see it's got all the parts for a for a business like I made this in two days I've never done dart before I made it in two days and you can - okay it's all about the mentality so that's that's what I want to show you here's the dashboard and here's the yoga let's just call this view the yoga so dashboard login sign up those are our four views that we want all right back to let's get let's start coding alright but before we code let me just explain what flutter is the architecture and what what actually it is so flutter contains a framework that's called dart and dart is this programming language that is used to compile down to the native Objective C language for iOS and Java for Android now iOS has been around for a decade and so has Android right so what we learned was you know you can create an app for iOS or Android in that native code and then that native code is gonna be able to speak to that platforms events it's Bluetooth sensors all of the you know low level stuff but some people didn't want to learn both Java and objective-c so then came these cross-platform JavaScript apps like PhoneGap in Cordova and so what happened then was you just wrote your entire app in JavaScript that JavaScript was then compiled down to talk directly to native code and then there was this bridge that allowed you to just write JavaScript and deploy to all these different services the problem is that that was very slow javascript is very slow for that so what happened six about six months ago keep in mind flutter is new what happened six months ago is that the team at Google working on the Chrome browser decided to create this new framework in one language for both platforms and here is an example flutter right here so the difference is that now you are writing it in this new language called dart it is reactive is simple if you've ever coded react je s in JavaScript it's very similar but if you haven't don't worry about it basically everything is a widget okay so it heavily uses this idea of widgets and so widgets can be anything widgets can be centering and padding they can be a text field widgets can be a logo they could be an image they can even be the entire application which it could be even a navigation bar everything is a widget everything is a widget in flutter okay and so you are writing things in dark and it compiles down to native code for both platforms which is incredible and this is faster than the JavaScript based way now these widgets should be beautiful they should be fast they should be customizable you should be able to modify them as you want and so so far I found that to be the case which is amazing so what happens is if everything is a widget what you're doing when you're building your app is you're actually creating a widget tree okay so you have a starting widget as as you can see here I'm now narrating what's happening here you have a starting widget and you just go through the flow of the app widget by widget by widget right each widget contains sub widgets and those sub widgets could contain sub widgets but you might be thinking well that seems not ideal you know I mean I feel like there could be some problems there that seems too simple well it turns out it's not actually it's way better than objective-c I would say or even Java I really like dart dart is amazing because first of all layout oh my god if you've ever done mobile development layout is hell like getting layouts to work properly across different types of Apple devices or Android devices is always a pain the beauty of flutter is that it just kind of takes care of layout for you you just focus on those widgets and the widget tree and it will take care of the rest so how do I really get this idea into your head I could rap about it I think that's gonna work so let me just quickly wrap and then we'll be back to this I built my widget tree making money with the flutter I build my widget tree I build my widget tree I build my widget tree making money with the flutter okay so back to this so we have this tree right here of widgets right so this is an example right so what we do is we create a new widget called a column that widget has two children right this text widget called hello world and this new icon and so this is what you know a widget tree looks like for example okay and you might be thinking well what about transitions and stuff look at this beautiful very complex transition set it's like going in different directions simultaneously all of this is possible with flutter like you can make very complex transitions which is amazing it's using Google's internal design format slash layout called material design and oh my god yes hot reload yes this was a lot of fun building with hot reload was awesome because you know that's one of those things from flask and from node that I love and I always wished that iOS development or Android development had this so hot reload is just like while you're coding it's changing as you're coding like we just you don't have to like stop compile wait and then you know it just it's your coding you make a change it shows up just like that as you're saying here super awesome in the last part I want to talk about before we start coding is installing flutter okay so this flutter has these packages these plugins so for example let's say I want to integrate payments so I'll go to the flutter website type in payments okay and it's got payments from square it's got payments from Paytm it's got payment stripe raiser pay which is awesome stripe and razor pay and a bunch of other payments right so what you do is you go to one of these plugins and you'll say how do I install it and so there are always three steps here depend on it so add it to your pub spec dot YAML then run flutter pub get and then import it into your code and that's it three steps this is for all of these plugins super cool stuff and then you can you know once you import it then you can open the dialog and there you go there's moto papa right super easy now let's get into the code alright so how do we install flutter so we've got to there's you know there's a couple steps here let's go through them let's go through these steps together I mean make this into a new window and we'll go through this together okay so step one to install flutter is to install Android studio so go to Android studio and we'll install it so I actually obviously already have Android studio but I'm just going there with you - let's just do this together now I will click on download Android studio I don't want an extra gig on my hard drive so it won't actually download it so just double click on that download it and then once it's downloaded start it up so let me go to let me actually close out Android studio okay yeah by the way I changed my shirt I was sweating too much now it's black you can't see the sweat great okay so I downloaded interest video now I'm gonna start Android studio okay once I have started Android studio I will open plug-in preferences so go to preferences go to plugins browser repositories select flutter and install it right from Android studio so that's where you install it inside of Android studio and you might be thinking well I want to do this for iOS or I want to do that oh no just do it in Android studio okay so I've already got flutters so I'm going to go to preferences and then I'll go into plugins and then I will what's the next step setting plugins and then flutter there we go this flutter and then so this is the button right here in so I've already in Seoul so you click on install it'll say yes and salton then you restart okay and when you restart let's just you know so assume I've restarted it's gonna be the same screen then you can say okay so either start a new flutter project or download an existing flutter project from the web and then open it by clicking on open and existing Android studio project so let's just start a new flutter project for this case okay it's a flutter application whatever begin creating yes go flutter fly away flutter flutter fly away my bird love you flutter now here's a little demo flutter flutter demo so here's what we've got here let's just look at this for a second I a class it extends a stateless widget okay so what's happening here is I've got my first class in Maine dot dart okay and so all its importing is the material design for the buttons and all the sorts of those little widgets so inside of widget build I'm going to build a single widget which just shows a title and it's got this theme which is the theme of my application so when I click on run hopefully this runs with my simulator so let's let's see what happens there we go okay so now it's running and so if you press the button it is responding see three four five six okay great so how is it doing that how is it responding so it's got this counter I see right here and it's incrementing the counter so counter plus plus whenever increment counter is called which is called where it's called right here on press we've got this floating action button okay which is this blue button right here it makes sense and it's got a scaffold so scaffolds allow you to create multiple different widgets inside of the scaffold I've got this attribute which is text all right so widget title is up there it's got some children right so within children you can have this array of multiple widgets so the first widget is text the second one is text and the third one is floating action button so if we go back here you'll see the tip this text right up here this text underneath which is the scalar value and then the button and you know we can make silly just go back and forth flutter demo homepage where's that flutter demo flutter demo homepage right here so see notice this it's right here so we can actually create several classes and we can point we can switch between different classes very easily I'll show you how to do all that in a second but this was just how to install flutter okay that's that's the point of that all right let's keep going here so what's our next step here we did that part so we install flutter and now we're gonna start building let's build that login page let's add firebase to authenticate the user and then let's build that dashboard page and integrate the payments and then we'll work on the transfer learning part right so where do we even begin with this let's think about it for a second well where we should begin is at the hardest part right so what would be the hardest to integrate into a mobile app it seems like pose net would be the hardest right that's it's a real time streaming camera based model that detects the pose of a user in real time that sounds difficult right because we have to take a tensor flow file convert it into a light file and then have an inference and you know flutters a newer framework tension flow is great but you know you never know how it's gonna interact with all these different systems so that's where we'll start should we do that ourselves I hope you said no because you're on you're getting me right you're getting me if you said no no we're not gonna build this ourselves we're going to use a base app for this so I went on Google and I search for flutter real-time detection and I found this great repository it's got this blog post as well where this this developer was a person named shadow to shack Ian's Shack shukkeien Oh shocked cha-chang cha-chang and it's got it already embedded into the app so we can just build off of this app it's and not only does it have Poe's net integrated it's got a bunch of different different models integrated like object detection a bunch of different convolutional networks things like that which is awesome right Thank You Shaq love the name all right so that's what we'll do so to install it we're gonna download it so let's just do this together we'll download it and then so let's close out that previous app that we had I was just trying to show you what the deal is in terms of how to install and run flutter oh no no no we don't want two of these and so it's downloading and so we're gonna run those instructions okay so flutter open okay and then in terminal open up a new terminal let's make this bigger big big big big big big big big and then CD into it okay I mean it here we are LS by the way Linux commands are amazing okay so what do we do now well let's type in flutter oh my god flutters not found oh you gotta sort your Bosch and stuff here that's right hold on okay make this bigger and there we are okay so flutter okay so there's flutter flutter is there now we do flutter packages yet as it says to install all those packages from the pub spec dot YAML file and let me let's also open that in sublime text so you see what I'm talking about here in the pub spec dot YAML this is where all those dependencies are so under dependencies you list all of those dependencies that you want okay so it's a camera Atia flight whatever and so great install those dependencies now we run it so flutter run now we could run it from terminal we could run it using sublime but I prefer to run it using hand royde studio because I love Android studio so let's do that okay so let's see how do we do this Android studio and then it's called flutter real-time detection master 3 so I'll open an existing Android studio project and it's called that and it's right there and I'll open it just like that okay there we go close so here are the five dart files bounding box main models and camera and home home is where the heart is home is the main one we'll look at so okay so you got a bunch of this stuff what is happening here is this bounding box code is computing a lot of math to create that bounding box around you whenever it detects you or any sort of object we don't have to worry about that let's X out of that the camera is opening the camera it's running whatever model is necessary whenever it's selected so you can select a different model and it will run it make sense and lastly this is the models out our page so it's got four different models that are there as constant strings that we can use throughout our app and pose net is the one that we want to use we're gonna not worry about the rest and inside of this home page state it's got loading models it's got building these buttons as notice remember we create these children inside of this widget builder inside of the scaffold and then so we create a raised button we create a bunch of these buttons okay so that's all of that stuff now that we have that base model we're gonna modify it by adding a login page right so we've already got one screen that is used to detect people with the pose net but now we want to add the login page so how do we do that let's let's let's do that right now okay so this is our model right now right here as you're seeing it right here this is what we've got it's it's just four buttons as you as you see right here it's called SSD mobile net so here's this I can change the tax in real time Siraj so you notice how it changed right there see one of them changed Siraj now I'll change it back to hello and there's hello so that's hot reload for you all right so what we're gonna do is we're gonna create a new class just for that login page okay so you're gonna call it third route we'll call it whatever we want actually I'll just call it third route because I'll have several routes between different screens okay so we'll call it you know third route and then it's going to extend a stateless widget just like the other classes because well it could be a stateful widget but I'm choosing stateless because I'm just starting I'll wait we'll see how it works and it will iterate right move fast and iterate so let's create this widget and inside this widget I'm going to have a context inside of the context I am going to say the background color is going to be a certain color so let's say I want it to be purple so I'll save color and then you've got to say Oh 0x FF and then the hex hexadecimal code which is 7 f8 4b e and that's the background color that I want for this for this app and then what I then want is for there to be this contained inside of a scaffold so inside of this scaffold that's where I'll have the background color that's the background color and then I'll say what's the body of this scaffold well the body is gonna be this model and inside this model in the center of this model I'm gonna have a child the child will have this column widget and this column will have children so just write directly down it's gonna have some children and those children are all widgets they're of the type widget and so it's an array so what's the first one well the first thing I want there to be is so we're thinking about top down right what does it look like top down if I say what's the first image that I want let's assume that I have this gift which remember that girl I'll show that gift Lola gif okay so that's the first widget and then I'll say okay , now what's my next widget my next widget is some text and let's call her Macy which is the name of the app and then what else you know I can add attributes as well like text a line things like that if I wanted to textile text-align:left x2 line is text-align:center a bunch of attributes for each widgets what it looks like and then I'm gonna say what else do we have what else do we have what else do we have we're gonna have a text field or a text form fields right for the login so I'll say this is going to be called the label for this will be user name and then I'll have another text form field call password this will be password and it's a it's a string so password and this is a string as well so user name and we can keep going we'll say okay so let's add a button it's at a button here for login this I mean it's very like CSS like it's very basic I've been compared to objective-c and stuff this is way better like I'm loving this language okay so then I'll have a button and so the button will have some text that will say login text log in and I can set a color for the button let's say deep purple accent this is again material design and then on pressed once that button is pressed remember I'm showing you the gist of the code there's once it's pressed I'm gonna use a navigator which is a widget and I'm going to say when that button is pressed based on this context use the material page route builder based on the context to take us to the second route and the second route will be the next page which is the dashboard and then I can keep adding I could keep adding buttons I can fix these as well that I don't know and once I have that login page then I can continue to move on so if I go to flutter okay that's what that looks like very basic login signup we can keep adding to it right so I want to show you something I want to stress something right here that is that basically you create multiple classes and you can push between the classes using the Navigator so navigator dot pop right so just like this on pressed when a button is pressed you can pop between the context of the different classes and those classes can extend either stateful or stateless widgets and that's just how you move about I've got you know one route two routes three routes forward routes and so what I've got here is a firebase login signup code right so very simple cut and paste right here just comment this out but basically whenever a user clicks the login button and the username and password fields Denari standard stuff like boilerplate this has he been used in thousands of apps like very easy stuff but basically I went to firebase and I was like alright well do you guys have a dart SDK I mean I don't know it turns out I guess of course they do because it's the same company that had made flutter Google so that makes it much much easier right so by the way to install firebase here's what I did so I go to dependencies so in pub spec dot yamo right just like I was saying before and I said here it is so I just typed it in under dev dependencies because I'm in dev mode I could switch to production but I'm in dev mode and then once that's there I'll do flutter pub get from terminal so flutter pub yet and that will install everything in that file and once that's done I could run the app so that it's very fast like super super fast stuff so now I've got that log in page of added firebase in terms of integrating payments what we have to do is we just have to go back to that flutter page so what's what's this payment called that it at a stripe okay and how do we install that well do that and then flutter pup get okay so I'll say there's stripe got integrated now I promised there would be some razor pay as well to keep it international because we love everybody here and there it's that simple so now we've got two of these dependencies okay and then we and we download them with flutter puff yet at the same time at the same time okay great one point six seconds now that is fast okay that is super fast so we've got razor baby we got stripe we can integrate those very easily in the same modal pop-up we can make our own modal pop-up now what else what else what else what else what else we got here okay firebase now the dashboard so I really you know really hacky I just put everything in the home dark file but you know you can just these are class files this is object-oriented programming just like Java just like most programming it's object-oriented so what you can do is you can just say file new you could say dart there we go and we could say you know whatever dashboard and then store the class in that and just say well here's that entire class ahahahahahaha and then just call it from there and then import that class from here so we say import dashboard Dart remember I'm trying to give you the start the the cat the I'm the catalyst on the espresso for you get started this code works by the way just go on good github and um whoa okay so models of Dart and then you can you know import it so for dashboard again just very simple stuff the dashboard is another widget that I created right here so there's three widgets kind of get lost sometimes because it's just like embedded within embedded with an embedded with an embedded widgets I'm sure there's something like promises and JavaScript where you don't get Kol back hell you wouldn't call it callback hell in this case you call it like widget hell which is gonna be a new term now no but we we want to avoid that kind of you know direction of just embedded and infinitely we want to make it make sure it's like the code looks like this right but just create widgets for different pages link them together with a navigator and yeah that's that's basically it okay so that's the dashboard firebase now let's get back to pose net we still have a I stuff to explain how pose networks we got a lot in this video okay now for pose net the way it works is you've got some came Angie okay you've got some image right here and then you you give you feed it into a neural network like like usual of course and then what its gonna do is it's going to create two vectors one is for a heat map and one is for a what's called a PAF it's an affinity field I'll explain how that works in a second you take both of these and you compute what's called the line integral that creates a graph you assign different body parts different vertices you connect it all together into a skeleton merge it and then that's that's that's your result let me go through exactly what I'm talking about there's quite a lot that dependencies here our knowledge of deep learning calculus set theory and graph theory here we go so what we want is a skeleton of a human right that's what we want so we'll take an image of a human okay let's say this is sick Pink Floyd picture love that wish you were here album we'll take a neural network it was trained on whatever you know millions of images of everything cocoa and the cocoa data set and we'll take two of those convolutional blocks from that network okay and we're gonna feed one of them is gonna we're gonna call it the heat that he map block the other one we're gonna call it the part affinity field block okay these are collections of matrices okay and what we do is we take our input image and we multiply them by each okay we're multiplying them by each that's what you're seeing here and it's gonna output two different values the first is a heat map and what the heat map does it's a matrix that shows how confident the network is of each body part that it detects and the the parts that are its most confident in its gonna it's gonna highlight anymore for the part affinity field it's gonna give you the information about the position and orientation of different pairs you've got body parts and then you've got pairs which is connecting those parts together and this tells you which direction each part is moving in in terms of likelihood right so it's kind of gives a radius a bounding box if you will around each body part and so the code looks like this right we take an existing network we feed the image into both of those layers get back those two matrices and now using these matrices we'll hold onto that affinity field for a second will not worry about that we're focused now on the heat map what we're gonna do is we're gonna detect parts of the image by extracting the locations out of the heat map so we'll start with the first pixel surround it with the window find the max value from that and that's those are the values that we'll use then we'll connect those body parts by creating a bipartite graph so for all of those coordinates those XY coordinates link them together and to create a graph out of them that's graph theory so everything is now connected but it's too many connections the head is connected to the foot which is connected to the arm and you know but it should be if everything shouldn't be connected right this is just a very bare bone everything is connected first graph that we're gonna prune so now we're gonna score each of those connections using what's called the line integral okay that's calculus and so this what that what it looks like basically we're performing a matrix multiplication operation to create this weighted bipartite graph which scores all of those parts that connect vertices for your stela ten and it's gonna weight which is the most likely to be the real one based on that heat map score and Khan Academy has a great explainer video on that okay so that's that's what it's doing then it's gonna find its gonna map summize a total score using that bipartite graph by sorting them in in order of the confidence and so that is the assignment algorithm and you can see the code for that there then it's gonna merge those detected connections into one single skeleton and so we can think of this as a you know collection of humans that could be in the in the image it's going to say if this if this body part looks like it's connected to the same human then make it a part of the same human but if it's if this body part if these two body parts have one looks like it's belongs to one human and another looks like it belongs to another human based on distance don't connect them okay so that's kind of the mathematical way of thinking about this and so the last step is that merging step okay so that's a quite a lot to cover in a short amount of time but I want to give you a high-level overview of a pose net right so literally the only learning happening in pose net is to first take a pre trained model take out those trained layers and create a heat map in a affinity field for an image and then you're performing not machine learning but a bunch of static algorithms like assignment or emerging or you know sorting things like that this is all happening in real time as its detecting you okay more information is gonna be in this document that I've created inside of the code repository on github look at it okay it'll it'll compile in your browser now for transfer learning what we could do is we could we have a couple strategies but the strategy that I like the most is to take an existing convolutional network which we have and and retrain you know late convolutional network contains many layers let's say 10 we'll take those top three layers and retrain those on yoga poses but keep the rest of those frozen why because we want to we want to benefit from the skills that it's learned from detecting all sorts of things humans cars dogs shapes because overall it's gonna make our prediction tasks better so we'll freeze those but we'll retrain those top layers for yoga so you might be thinking well you know how do we do this you know we got how we get the yoga images where where are we gonna find all this stuff again github so much uncaptured value on this website okay this person created what's his person named smells like ml great name smells like ml made this app called yo gay eye and it's using a convolutional posed machine model they already retrained it on a bunch of they've even got this hackster pose they've already retrained it on a bunch of yoga images which you can find here and we can find the models pre-trained right here as a TF light file now if we go back to our dark code here are the TF light files okay so we just have to replace it with our yoga net TF light file okay and then we'll place it into our assets right here all the yoga net so we'll download it and place in our assets and there we go we've got yoga net running in our model we took two different repositories and combined them together now after we've integrated that model into our app we want to integrate text-to-speech so how are we gonna do that well we'll go back to this and we'll say text-to-speech do we have any plugins for that yes luckily we got two so we'll go right here and how does this work so once we install it let's go to the github repository and it doesn't look like it has that much documentation so next how about this one TTS example ooh okay so we create a text speech object and then we just say speak that's easy hello world just like that great and so here's how it works once I've added it to my pub spec dot YAML letter text-to-speech there it is then I will go to home then I will create say right here you know TTS speak hello world and then you know you can delay it by a certain number of seconds okay so just like that so what happens is whenever it detects a pose whenever it detects any pose it will speak you know some it will say something so it'll say good job hold that pose okay so if the pose is detected it's getting this real-time stream say good job hold that pose and so we'll have a timer and we'll say after say five minutes or some threshold you know go to the next pose and then it will start listening so we'll create another event listener for the next pose and so we'll create different event listeners for all those different poses for as many as we want let's say ten for a session and it's gonna be you know guided by this voice which is our assistant Macy and so it's a text-to-speech engine responding to the stream coming from this pose net model happening in real time and because we can put these together enough in a graceful way we can anthropomorphize our app and make it you know a user feel you know happier more peaceful their lives are more meaningful their yoga practice doesn't cost as much because it's cheaper than a human and they can do it anywhere can you you know at any time put the phone against the wall and start you know moving in stuff so that's what we that that's that's the next step and that's that's it for this demo video now how to improve this app there's a lot we could do to improve this app okay this is this is just the beginning first of all we can create a generative model to create a 3d yoga instructor in real time we can add augmented reality so this person could be right next to us we can add more personalization more poses more interactive dialogue that is you know responding to the user with more intelligence we can add other features you know other sensors for your diet and things like that more analytics there's a lot we can do but this say first bare-bones app I hope you found this video useful and please subscribe for more programming videos for now I'm going to learn more about flutter thanks for watchinghello world it's Suraj and I've built an AI yoga instructor her name is Amy and she lives in this app that I built in flutter and flutter allows me to build an app for both iOS and Android in the same codebase which is incredible let me show you a quick demo first so I'll log in with my username and password and once I've logged in this is my dashboard she's gonna show me my current level of you know at which stage in the journey of yoga I'm at and once I have logged in and assume that I've paid as well you know you have to pay 100 US dollars a month to use this on-demand automated yoga instruction service and assume that I've paid once I've paid and by the way I've integrated not just stripe but Razer pay as well so India you guys are involved in this as well we're gonna go global with this as soon as you have paid then you can begin this service now I just want to say I'm using my webcam so I can't show my full body here but I just want to show you the pose detection that's happening right now it's gonna be kind of cramped but let's let's start a demo right here so as you can see it is detecting my pose and so this is the warrior pose and once she detects my pose she's gonna tell me to hold that pose for a little bit and once I've held that pose for a little bit after a while she will tell me to go into the next pose and so she's looking at which pose I'm doing and then she's matching that with what she understands about yoga which I've trained over time so I'm gonna show you how to do that so let's jump at it let's X out of this or right back to the dashboard now you might be thinking well why don't you just build a business out of this why are you showing us how to do it why do you have the code for what you built in the video description for everybody to see because it's more impactful for me I believe to give you the building blocks to show you to give you the path on how to gain financial freedom that means you don't work for anybody else you work for yourself either as a freelancer or you start your startup and the reason I'm saying this so strongly at the beginning of the video is because anybody can become rich using this technology anybody can build wealth by taking this internet technology and this laptop computer wherever you are in the world and in the age of infinite leverage if you make the proper decisions in terms of how to architect an app how to design the app how to perform business development to get your first clients you can build a sustainable business no matter where you are in the world and you can gain financial freedom by making your life better and the lives of others so it's a win-win-win situation I want you to gain financial freedom and that's why I am not just showing you all the possibilities of AI in healthcare I am showing you a practical example and I'm gonna show you from start to finish that's my promise in this video I'm gonna show you from start to finish how exactly I built this and make sure that you understand every step of the way so you can either take this code and turn it into a profitable business I will rejoice in your success or you have a different but related idea and you can start working on that so let's be very clear about what this video is about there are three learning objectives in this video the first is how to create a simple flutter app okay flutter is awesome it was just released a few months ago and with one code base I had to learn the dart programming language with one code base you can build an app for both iOS and Android that's super cool that's the first the second is I'm gonna show you how a pose net model is able to detect poses what the demo you just saw was and lastly I will show you how to perform transfer learning this is a hot word in the community these days because it's so popular especially in computer vision and I'll show you exactly how to do that to take an existing convolutional network and train it to detect yoga poses those are the three learning objectives alright in addition to everything else but those are the three main ones that I'm gonna leave you with by the end of this video that's my promise to you so don't go anywhere I'm also gonna wrap in this video I've got some memes we're taking this education stuff to the next level I am just getting started alright so let's just get started with this what are the tools that I use to build this well first of all I use the flutter right for ENSO flutter allows you to make an app for ios and android and kind of web as well which is insane but there's still that's that that part is really buggy so we're not gonna look at that web part we're just gonna focus on the ios and android part okay I also used firebase which is a Google server to store that data in the cloud you know user ID what is my yoga score you know different things like that social features I'm using stripe for payments in the West and I'm using Razer pay for payments in India you know it's really it is really really sad that there is not a global payments service yet it's just insane and that's why cryptocurrency is amazing it's insane how much Bitcoin is rallying right now I'm seeing I'm seeing the pulse of crypto rise these days so I'm gonna keep an eye on that so don't worry alright so what else that I've used I use tensorflow Lite to take this giant pose Nats model and compress it into something that could fit on a mobile device very light on device inference amazing stuff I used the pose net model as you're seeing these guys here super cool stuff this is called multi pose detection and I use an image data set of different yoga poses to perform transfer learning with an existing model I'm gonna show you how all that works and then one more thing that I used was a text-to-speech system to give the assistant a voice okay so there are a lot of opportunities in the healthcare space specifically using AI to improve the healthcare space I mean this graph is from the Harvard Business Review a very reputable magazine ten AI applications that could change healthcare look at all these now this is predicting the potential annual value by 2026 if you don't know how to code if you have never coded in your life and you start today by 2026 you could have a billion dollar business okay or whatever currency you use by 2026 all of these markets these sub markets of healthcare are posed for massive growth from fraud detection the virtual nursing assistant to preliminary diagnosis electronic health records are terrible these are archaic you know Windows 92 esque systems that all doctors use and those could easily be upgraded to internet status find a doctor talk to them about their problems and create a solution using AI technology the second idea I'm giving you ideas so you know make sure that you this this is in the code by the way this is in the code repository all these ideas the second is to create a some kind of new smart healthcare device you know doctors have a bunch of sensors like what is your heartbeat what is your galvanic response what is your sweat you know all these things if we could create a device that learns from this stream of data in real-time and adapts to the user maybe making the bed a little more comfortable or you know increasing the IV drip based on heartbeat etc we can free up nurses time and time is very valuable and so one more thing is that the smartphone can be an incredibly powerful diagnostic tool right you know you don't always need a doctor that was that had four years of college four years of medical school and then residency just to detect whether or not you have you know some kind of skin lesion or something like that here's an incredible statistic more radiologists work in the half dozen hospitals lining the renowned Longwood Avenue in Boston than in all of West Africa this is your opportunity to create a business that has huge impact a I in healthcare has huge impact I am NOT the only person to have said this this is a huge opportunity okay to serve the underserved and to make money doing it right that's a win-win situation that's the kind of person we need in the world more of and that's what what I'm trying to create by creating these free tutorials I will never charge for these to talk I'm just so hype right now I will never charge for these tutorials because it it goes against my my morals I will show you how to do this just continue to watch and I will show you the code I'll show you everything now Accenture says the same thing alright so so you might be thinking okay so where did where do I begin right I get I have an idea but I've never coded before where do I even start there's this great website called github.com and so if you see any and all coders upload their code to this website github.com and the coding culture developer culture is beautiful in that we share everything we share everything a lot of the times and so if you have some kind of idea let's say medical imaging or you know electronic health record databases or something just search it on github and so in this case I searched medical imaging and I got 1,500 results for different applications of AI or any kind of coding technology to medical imaging these are unused tools there is so much uncaptured value on github the reason I say there uncaptured is because most people don't understand code and they don't understand the complexity but if you are able to understand this complexity you can build you can provide society with value by pay walling this and providing a service that people can actually use making it make it easier to use then you know doing pip install and that's the complexity that you're taking on by doing pip install you know by you know installing dependencies and docker izing it and all these things that is a complexity that you can take care of and make money from okay so let's get started with this app the first step for us is to design this app so I'm gonna break out the white board get ready for this okay so here is my white board and by the way yes I'm sweating I don't care I'm here to help you learn so ignore the sweat let's just keep going with this so how are we gonna design this app well we know that we need a login page okay so this is the login page log in and in that login page we'll have a user in the password text field and let's say you know I want to anthropomorphize this app anthropomorphize means I want to make it seem like it's human so to do that I'm gonna put a picture or a gift of some girl or woman here who represents the yoga instructor and let's call her Macy okay so we'll have the title of the app right there and once I have a login to two text fields for user and password then I'm going to have a login button and a sign-up button okay so the signup button we can basically just reuse the code you know that created this login page to add some more text fields for user password repeat your password maybe the email and then sign up right so sign up button and then we just go right back so we're gonna have an arrow in both directions so that's login that's sign up okay but assume that you know I've already signed up and now I've logged in what's the next step well it's prop oh my god clearly I'm a geometry okay so what's the next page oh my god I'm so hyped to teach I am so hyped to do this I love my job I love this so much okay here we go the dashboard the dashboard is where the stats show up so we want to we want to gamify this app so we'll say this is gonna be some scalar value between 0 and 10 that will store in the database that shows where you are in your yogic journey okay we'll also have the image of this you know woman again just to continue this train of anthropomorphize a ssin I'll have a button to pay and they don't have a button to start so begin now for the pay button I'll have some sort of like modal pop-up you know for CVV credit card code your you know your month date whatever and then we'll go back to this once I begin here is the good stuff so first of all I want a nav bar so I can go back I want a nav bar so I'll go back so back button now inside of this that's where I want the model to be running inference the camera is on I will be standing here and it will be performing inference on me so it's gonna detect my pose so what I'll actually do is I'm going to put this phone up against the wall and then I'll just be by myself and I'll be doing yoga and so what's happening here is pose net is running it's detecting my pose because I've retrained it on yoga poses from a yoga image data set it's gonna be able to know what a certain pose looks like in terms of a yoga pose right downward-facing dog the warrior pose lotus position it's gonna know and so if it detects you know any certain pose and let's just say we have an array of poses and so it wants certain poses in a sequential order we'll name it 0 through 10 let's say 10 poses per session hold each pose for some threshold let's say 5 minutes it's gonna detect those poses and once it detects a certain pose we'll have an event listener that's basically just reading in the stream of X Y coordinates and once it detects a certain pose once that classification returns which it's going to automatically in real-time then we'll say ok if pose equals you know whatever value we'll set those 10 values from the array it pose equals x then we'll set some counter variable like int counter equals 1 and then once that counter is 10 we'll know we're done and then it posts equals x move on to the next pose so next and then we'll also have the text-to-speech engine which is her Maisie will say good job or something you know encourage the user now there is a lot we can do here and arguably this is the this is the value of our app right this is the hard part right here like combining pose net with this text-to-speech engine and doing so in a comfortable gentle yoga instructor s Quay so that so we'll get to that but this is the basic structure of the prototype and as you can see it's got all the parts for a for a business like I made this in two days I've never done dart before I made it in two days and you can - okay it's all about the mentality so that's that's what I want to show you here's the dashboard and here's the yoga let's just call this view the yoga so dashboard login sign up those are our four views that we want all right back to let's get let's start coding alright but before we code let me just explain what flutter is the architecture and what what actually it is so flutter contains a framework that's called dart and dart is this programming language that is used to compile down to the native Objective C language for iOS and Java for Android now iOS has been around for a decade and so has Android right so what we learned was you know you can create an app for iOS or Android in that native code and then that native code is gonna be able to speak to that platforms events it's Bluetooth sensors all of the you know low level stuff but some people didn't want to learn both Java and objective-c so then came these cross-platform JavaScript apps like PhoneGap in Cordova and so what happened then was you just wrote your entire app in JavaScript that JavaScript was then compiled down to talk directly to native code and then there was this bridge that allowed you to just write JavaScript and deploy to all these different services the problem is that that was very slow javascript is very slow for that so what happened six about six months ago keep in mind flutter is new what happened six months ago is that the team at Google working on the Chrome browser decided to create this new framework in one language for both platforms and here is an example flutter right here so the difference is that now you are writing it in this new language called dart it is reactive is simple if you've ever coded react je s in JavaScript it's very similar but if you haven't don't worry about it basically everything is a widget okay so it heavily uses this idea of widgets and so widgets can be anything widgets can be centering and padding they can be a text field widgets can be a logo they could be an image they can even be the entire application which it could be even a navigation bar everything is a widget everything is a widget in flutter okay and so you are writing things in dark and it compiles down to native code for both platforms which is incredible and this is faster than the JavaScript based way now these widgets should be beautiful they should be fast they should be customizable you should be able to modify them as you want and so so far I found that to be the case which is amazing so what happens is if everything is a widget what you're doing when you're building your app is you're actually creating a widget tree okay so you have a starting widget as as you can see here I'm now narrating what's happening here you have a starting widget and you just go through the flow of the app widget by widget by widget right each widget contains sub widgets and those sub widgets could contain sub widgets but you might be thinking well that seems not ideal you know I mean I feel like there could be some problems there that seems too simple well it turns out it's not actually it's way better than objective-c I would say or even Java I really like dart dart is amazing because first of all layout oh my god if you've ever done mobile development layout is hell like getting layouts to work properly across different types of Apple devices or Android devices is always a pain the beauty of flutter is that it just kind of takes care of layout for you you just focus on those widgets and the widget tree and it will take care of the rest so how do I really get this idea into your head I could rap about it I think that's gonna work so let me just quickly wrap and then we'll be back to this I built my widget tree making money with the flutter I build my widget tree I build my widget tree I build my widget tree making money with the flutter okay so back to this so we have this tree right here of widgets right so this is an example right so what we do is we create a new widget called a column that widget has two children right this text widget called hello world and this new icon and so this is what you know a widget tree looks like for example okay and you might be thinking well what about transitions and stuff look at this beautiful very complex transition set it's like going in different directions simultaneously all of this is possible with flutter like you can make very complex transitions which is amazing it's using Google's internal design format slash layout called material design and oh my god yes hot reload yes this was a lot of fun building with hot reload was awesome because you know that's one of those things from flask and from node that I love and I always wished that iOS development or Android development had this so hot reload is just like while you're coding it's changing as you're coding like we just you don't have to like stop compile wait and then you know it just it's your coding you make a change it shows up just like that as you're saying here super awesome in the last part I want to talk about before we start coding is installing flutter okay so this flutter has these packages these plugins so for example let's say I want to integrate payments so I'll go to the flutter website type in payments okay and it's got payments from square it's got payments from Paytm it's got payment stripe raiser pay which is awesome stripe and razor pay and a bunch of other payments right so what you do is you go to one of these plugins and you'll say how do I install it and so there are always three steps here depend on it so add it to your pub spec dot YAML then run flutter pub get and then import it into your code and that's it three steps this is for all of these plugins super cool stuff and then you can you know once you import it then you can open the dialog and there you go there's moto papa right super easy now let's get into the code alright so how do we install flutter so we've got to there's you know there's a couple steps here let's go through them let's go through these steps together I mean make this into a new window and we'll go through this together okay so step one to install flutter is to install Android studio so go to Android studio and we'll install it so I actually obviously already have Android studio but I'm just going there with you - let's just do this together now I will click on download Android studio I don't want an extra gig on my hard drive so it won't actually download it so just double click on that download it and then once it's downloaded start it up so let me go to let me actually close out Android studio okay yeah by the way I changed my shirt I was sweating too much now it's black you can't see the sweat great okay so I downloaded interest video now I'm gonna start Android studio okay once I have started Android studio I will open plug-in preferences so go to preferences go to plugins browser repositories select flutter and install it right from Android studio so that's where you install it inside of Android studio and you might be thinking well I want to do this for iOS or I want to do that oh no just do it in Android studio okay so I've already got flutters so I'm going to go to preferences and then I'll go into plugins and then I will what's the next step setting plugins and then flutter there we go this flutter and then so this is the button right here in so I've already in Seoul so you click on install it'll say yes and salton then you restart okay and when you restart let's just you know so assume I've restarted it's gonna be the same screen then you can say okay so either start a new flutter project or download an existing flutter project from the web and then open it by clicking on open and existing Android studio project so let's just start a new flutter project for this case okay it's a flutter application whatever begin creating yes go flutter fly away flutter flutter fly away my bird love you flutter now here's a little demo flutter flutter demo so here's what we've got here let's just look at this for a second I a class it extends a stateless widget okay so what's happening here is I've got my first class in Maine dot dart okay and so all its importing is the material design for the buttons and all the sorts of those little widgets so inside of widget build I'm going to build a single widget which just shows a title and it's got this theme which is the theme of my application so when I click on run hopefully this runs with my simulator so let's let's see what happens there we go okay so now it's running and so if you press the button it is responding see three four five six okay great so how is it doing that how is it responding so it's got this counter I see right here and it's incrementing the counter so counter plus plus whenever increment counter is called which is called where it's called right here on press we've got this floating action button okay which is this blue button right here it makes sense and it's got a scaffold so scaffolds allow you to create multiple different widgets inside of the scaffold I've got this attribute which is text all right so widget title is up there it's got some children right so within children you can have this array of multiple widgets so the first widget is text the second one is text and the third one is floating action button so if we go back here you'll see the tip this text right up here this text underneath which is the scalar value and then the button and you know we can make silly just go back and forth flutter demo homepage where's that flutter demo flutter demo homepage right here so see notice this it's right here so we can actually create several classes and we can point we can switch between different classes very easily I'll show you how to do all that in a second but this was just how to install flutter okay that's that's the point of that all right let's keep going here so what's our next step here we did that part so we install flutter and now we're gonna start building let's build that login page let's add firebase to authenticate the user and then let's build that dashboard page and integrate the payments and then we'll work on the transfer learning part right so where do we even begin with this let's think about it for a second well where we should begin is at the hardest part right so what would be the hardest to integrate into a mobile app it seems like pose net would be the hardest right that's it's a real time streaming camera based model that detects the pose of a user in real time that sounds difficult right because we have to take a tensor flow file convert it into a light file and then have an inference and you know flutters a newer framework tension flow is great but you know you never know how it's gonna interact with all these different systems so that's where we'll start should we do that ourselves I hope you said no because you're on you're getting me right you're getting me if you said no no we're not gonna build this ourselves we're going to use a base app for this so I went on Google and I search for flutter real-time detection and I found this great repository it's got this blog post as well where this this developer was a person named shadow to shack Ian's Shack shukkeien Oh shocked cha-chang cha-chang and it's got it already embedded into the app so we can just build off of this app it's and not only does it have Poe's net integrated it's got a bunch of different different models integrated like object detection a bunch of different convolutional networks things like that which is awesome right Thank You Shaq love the name all right so that's what we'll do so to install it we're gonna download it so let's just do this together we'll download it and then so let's close out that previous app that we had I was just trying to show you what the deal is in terms of how to install and run flutter oh no no no we don't want two of these and so it's downloading and so we're gonna run those instructions okay so flutter open okay and then in terminal open up a new terminal let's make this bigger big big big big big big big big and then CD into it okay I mean it here we are LS by the way Linux commands are amazing okay so what do we do now well let's type in flutter oh my god flutters not found oh you gotta sort your Bosch and stuff here that's right hold on okay make this bigger and there we are okay so flutter okay so there's flutter flutter is there now we do flutter packages yet as it says to install all those packages from the pub spec dot YAML file and let me let's also open that in sublime text so you see what I'm talking about here in the pub spec dot YAML this is where all those dependencies are so under dependencies you list all of those dependencies that you want okay so it's a camera Atia flight whatever and so great install those dependencies now we run it so flutter run now we could run it from terminal we could run it using sublime but I prefer to run it using hand royde studio because I love Android studio so let's do that okay so let's see how do we do this Android studio and then it's called flutter real-time detection master 3 so I'll open an existing Android studio project and it's called that and it's right there and I'll open it just like that okay there we go close so here are the five dart files bounding box main models and camera and home home is where the heart is home is the main one we'll look at so okay so you got a bunch of this stuff what is happening here is this bounding box code is computing a lot of math to create that bounding box around you whenever it detects you or any sort of object we don't have to worry about that let's X out of that the camera is opening the camera it's running whatever model is necessary whenever it's selected so you can select a different model and it will run it make sense and lastly this is the models out our page so it's got four different models that are there as constant strings that we can use throughout our app and pose net is the one that we want to use we're gonna not worry about the rest and inside of this home page state it's got loading models it's got building these buttons as notice remember we create these children inside of this widget builder inside of the scaffold and then so we create a raised button we create a bunch of these buttons okay so that's all of that stuff now that we have that base model we're gonna modify it by adding a login page right so we've already got one screen that is used to detect people with the pose net but now we want to add the login page so how do we do that let's let's let's do that right now okay so this is our model right now right here as you're seeing it right here this is what we've got it's it's just four buttons as you as you see right here it's called SSD mobile net so here's this I can change the tax in real time Siraj so you notice how it changed right there see one of them changed Siraj now I'll change it back to hello and there's hello so that's hot reload for you all right so what we're gonna do is we're gonna create a new class just for that login page okay so you're gonna call it third route we'll call it whatever we want actually I'll just call it third route because I'll have several routes between different screens okay so we'll call it you know third route and then it's going to extend a stateless widget just like the other classes because well it could be a stateful widget but I'm choosing stateless because I'm just starting I'll wait we'll see how it works and it will iterate right move fast and iterate so let's create this widget and inside this widget I'm going to have a context inside of the context I am going to say the background color is going to be a certain color so let's say I want it to be purple so I'll save color and then you've got to say Oh 0x FF and then the hex hexadecimal code which is 7 f8 4b e and that's the background color that I want for this for this app and then what I then want is for there to be this contained inside of a scaffold so inside of this scaffold that's where I'll have the background color that's the background color and then I'll say what's the body of this scaffold well the body is gonna be this model and inside this model in the center of this model I'm gonna have a child the child will have this column widget and this column will have children so just write directly down it's gonna have some children and those children are all widgets they're of the type widget and so it's an array so what's the first one well the first thing I want there to be is so we're thinking about top down right what does it look like top down if I say what's the first image that I want let's assume that I have this gift which remember that girl I'll show that gift Lola gif okay so that's the first widget and then I'll say okay , now what's my next widget my next widget is some text and let's call her Macy which is the name of the app and then what else you know I can add attributes as well like text a line things like that if I wanted to textile text-align:left x2 line is text-align:center a bunch of attributes for each widgets what it looks like and then I'm gonna say what else do we have what else do we have what else do we have we're gonna have a text field or a text form fields right for the login so I'll say this is going to be called the label for this will be user name and then I'll have another text form field call password this will be password and it's a it's a string so password and this is a string as well so user name and we can keep going we'll say okay so let's add a button it's at a button here for login this I mean it's very like CSS like it's very basic I've been compared to objective-c and stuff this is way better like I'm loving this language okay so then I'll have a button and so the button will have some text that will say login text log in and I can set a color for the button let's say deep purple accent this is again material design and then on pressed once that button is pressed remember I'm showing you the gist of the code there's once it's pressed I'm gonna use a navigator which is a widget and I'm going to say when that button is pressed based on this context use the material page route builder based on the context to take us to the second route and the second route will be the next page which is the dashboard and then I can keep adding I could keep adding buttons I can fix these as well that I don't know and once I have that login page then I can continue to move on so if I go to flutter okay that's what that looks like very basic login signup we can keep adding to it right so I want to show you something I want to stress something right here that is that basically you create multiple classes and you can push between the classes using the Navigator so navigator dot pop right so just like this on pressed when a button is pressed you can pop between the context of the different classes and those classes can extend either stateful or stateless widgets and that's just how you move about I've got you know one route two routes three routes forward routes and so what I've got here is a firebase login signup code right so very simple cut and paste right here just comment this out but basically whenever a user clicks the login button and the username and password fields Denari standard stuff like boilerplate this has he been used in thousands of apps like very easy stuff but basically I went to firebase and I was like alright well do you guys have a dart SDK I mean I don't know it turns out I guess of course they do because it's the same company that had made flutter Google so that makes it much much easier right so by the way to install firebase here's what I did so I go to dependencies so in pub spec dot yamo right just like I was saying before and I said here it is so I just typed it in under dev dependencies because I'm in dev mode I could switch to production but I'm in dev mode and then once that's there I'll do flutter pub get from terminal so flutter pub yet and that will install everything in that file and once that's done I could run the app so that it's very fast like super super fast stuff so now I've got that log in page of added firebase in terms of integrating payments what we have to do is we just have to go back to that flutter page so what's what's this payment called that it at a stripe okay and how do we install that well do that and then flutter pup get okay so I'll say there's stripe got integrated now I promised there would be some razor pay as well to keep it international because we love everybody here and there it's that simple so now we've got two of these dependencies okay and then we and we download them with flutter puff yet at the same time at the same time okay great one point six seconds now that is fast okay that is super fast so we've got razor baby we got stripe we can integrate those very easily in the same modal pop-up we can make our own modal pop-up now what else what else what else what else what else we got here okay firebase now the dashboard so I really you know really hacky I just put everything in the home dark file but you know you can just these are class files this is object-oriented programming just like Java just like most programming it's object-oriented so what you can do is you can just say file new you could say dart there we go and we could say you know whatever dashboard and then store the class in that and just say well here's that entire class ahahahahahaha and then just call it from there and then import that class from here so we say import dashboard Dart remember I'm trying to give you the start the the cat the I'm the catalyst on the espresso for you get started this code works by the way just go on good github and um whoa okay so models of Dart and then you can you know import it so for dashboard again just very simple stuff the dashboard is another widget that I created right here so there's three widgets kind of get lost sometimes because it's just like embedded within embedded with an embedded with an embedded widgets I'm sure there's something like promises and JavaScript where you don't get Kol back hell you wouldn't call it callback hell in this case you call it like widget hell which is gonna be a new term now no but we we want to avoid that kind of you know direction of just embedded and infinitely we want to make it make sure it's like the code looks like this right but just create widgets for different pages link them together with a navigator and yeah that's that's basically it okay so that's the dashboard firebase now let's get back to pose net we still have a I stuff to explain how pose networks we got a lot in this video okay now for pose net the way it works is you've got some came Angie okay you've got some image right here and then you you give you feed it into a neural network like like usual of course and then what its gonna do is it's going to create two vectors one is for a heat map and one is for a what's called a PAF it's an affinity field I'll explain how that works in a second you take both of these and you compute what's called the line integral that creates a graph you assign different body parts different vertices you connect it all together into a skeleton merge it and then that's that's that's your result let me go through exactly what I'm talking about there's quite a lot that dependencies here our knowledge of deep learning calculus set theory and graph theory here we go so what we want is a skeleton of a human right that's what we want so we'll take an image of a human okay let's say this is sick Pink Floyd picture love that wish you were here album we'll take a neural network it was trained on whatever you know millions of images of everything cocoa and the cocoa data set and we'll take two of those convolutional blocks from that network okay and we're gonna feed one of them is gonna we're gonna call it the heat that he map block the other one we're gonna call it the part affinity field block okay these are collections of matrices okay and what we do is we take our input image and we multiply them by each okay we're multiplying them by each that's what you're seeing here and it's gonna output two different values the first is a heat map and what the heat map does it's a matrix that shows how confident the network is of each body part that it detects and the the parts that are its most confident in its gonna it's gonna highlight anymore for the part affinity field it's gonna give you the information about the position and orientation of different pairs you've got body parts and then you've got pairs which is connecting those parts together and this tells you which direction each part is moving in in terms of likelihood right so it's kind of gives a radius a bounding box if you will around each body part and so the code looks like this right we take an existing network we feed the image into both of those layers get back those two matrices and now using these matrices we'll hold onto that affinity field for a second will not worry about that we're focused now on the heat map what we're gonna do is we're gonna detect parts of the image by extracting the locations out of the heat map so we'll start with the first pixel surround it with the window find the max value from that and that's those are the values that we'll use then we'll connect those body parts by creating a bipartite graph so for all of those coordinates those XY coordinates link them together and to create a graph out of them that's graph theory so everything is now connected but it's too many connections the head is connected to the foot which is connected to the arm and you know but it should be if everything shouldn't be connected right this is just a very bare bone everything is connected first graph that we're gonna prune so now we're gonna score each of those connections using what's called the line integral okay that's calculus and so this what that what it looks like basically we're performing a matrix multiplication operation to create this weighted bipartite graph which scores all of those parts that connect vertices for your stela ten and it's gonna weight which is the most likely to be the real one based on that heat map score and Khan Academy has a great explainer video on that okay so that's that's what it's doing then it's gonna find its gonna map summize a total score using that bipartite graph by sorting them in in order of the confidence and so that is the assignment algorithm and you can see the code for that there then it's gonna merge those detected connections into one single skeleton and so we can think of this as a you know collection of humans that could be in the in the image it's going to say if this if this body part looks like it's connected to the same human then make it a part of the same human but if it's if this body part if these two body parts have one looks like it's belongs to one human and another looks like it belongs to another human based on distance don't connect them okay so that's kind of the mathematical way of thinking about this and so the last step is that merging step okay so that's a quite a lot to cover in a short amount of time but I want to give you a high-level overview of a pose net right so literally the only learning happening in pose net is to first take a pre trained model take out those trained layers and create a heat map in a affinity field for an image and then you're performing not machine learning but a bunch of static algorithms like assignment or emerging or you know sorting things like that this is all happening in real time as its detecting you okay more information is gonna be in this document that I've created inside of the code repository on github look at it okay it'll it'll compile in your browser now for transfer learning what we could do is we could we have a couple strategies but the strategy that I like the most is to take an existing convolutional network which we have and and retrain you know late convolutional network contains many layers let's say 10 we'll take those top three layers and retrain those on yoga poses but keep the rest of those frozen why because we want to we want to benefit from the skills that it's learned from detecting all sorts of things humans cars dogs shapes because overall it's gonna make our prediction tasks better so we'll freeze those but we'll retrain those top layers for yoga so you might be thinking well you know how do we do this you know we got how we get the yoga images where where are we gonna find all this stuff again github so much uncaptured value on this website okay this person created what's his person named smells like ml great name smells like ml made this app called yo gay eye and it's using a convolutional posed machine model they already retrained it on a bunch of they've even got this hackster pose they've already retrained it on a bunch of yoga images which you can find here and we can find the models pre-trained right here as a TF light file now if we go back to our dark code here are the TF light files okay so we just have to replace it with our yoga net TF light file okay and then we'll place it into our assets right here all the yoga net so we'll download it and place in our assets and there we go we've got yoga net running in our model we took two different repositories and combined them together now after we've integrated that model into our app we want to integrate text-to-speech so how are we gonna do that well we'll go back to this and we'll say text-to-speech do we have any plugins for that yes luckily we got two so we'll go right here and how does this work so once we install it let's go to the github repository and it doesn't look like it has that much documentation so next how about this one TTS example ooh okay so we create a text speech object and then we just say speak that's easy hello world just like that great and so here's how it works once I've added it to my pub spec dot YAML letter text-to-speech there it is then I will go to home then I will create say right here you know TTS speak hello world and then you know you can delay it by a certain number of seconds okay so just like that so what happens is whenever it detects a pose whenever it detects any pose it will speak you know some it will say something so it'll say good job hold that pose okay so if the pose is detected it's getting this real-time stream say good job hold that pose and so we'll have a timer and we'll say after say five minutes or some threshold you know go to the next pose and then it will start listening so we'll create another event listener for the next pose and so we'll create different event listeners for all those different poses for as many as we want let's say ten for a session and it's gonna be you know guided by this voice which is our assistant Macy and so it's a text-to-speech engine responding to the stream coming from this pose net model happening in real time and because we can put these together enough in a graceful way we can anthropomorphize our app and make it you know a user feel you know happier more peaceful their lives are more meaningful their yoga practice doesn't cost as much because it's cheaper than a human and they can do it anywhere can you you know at any time put the phone against the wall and start you know moving in stuff so that's what we that that's that's the next step and that's that's it for this demo video now how to improve this app there's a lot we could do to improve this app okay this is this is just the beginning first of all we can create a generative model to create a 3d yoga instructor in real time we can add augmented reality so this person could be right next to us we can add more personalization more poses more interactive dialogue that is you know responding to the user with more intelligence we can add other features you know other sensors for your diet and things like that more analytics there's a lot we can do but this say first bare-bones app I hope you found this video useful and please subscribe for more programming videos for now I'm going to learn more about flutter thanks for watching\n"