The History and Development of Machine Learning and Artificial Intelligence

====================================================================

Machine learning and artificial intelligence have become increasingly popular in recent years, with applications in various fields such as image recognition, natural language processing, and predictive analytics. However, the journey to develop these technologies was not an overnight success. The history of machine learning dates back to the 1950s, when the first neural networks were developed.

The Early Days of Machine Learning

----------------------------------

One of the pioneers in machine learning is Andrew Ng, a Canadian computer scientist who co-founded Google Brain and Coursera's Machine Learning course. Ng has been instrumental in popularizing machine learning and artificial intelligence through his work on the Google Brain project, which aimed to develop an AI system that could learn from large amounts of data.

The Development of TensorFlow

------------------------------

In 2015, researchers at Google developed TensorFlow, a deep learning framework that was designed to be highly flexible and scalable. TensorFlow has since become one of the most widely used machine learning frameworks in the world. The development of TensorFlow was made possible by the contributions of many researchers and engineers, including Ian Goodfellow, who is often credited with inventing Generative Adversarial Networks (GANs).

The Rise of Kaggle

------------------

Kaggle, a popular platform for machine learning competitions, was co-founded by two Australians, Anthony Goldbloom and Ben Hamner. Kaggle provides a range of tools and resources for data scientists, including datasets, notebooks, and forums where users can share their knowledge and experiences.

The Power of Mobile Devices

---------------------------

Mobile devices, such as smartphones and tablets, have become increasingly powerful over the years, making it possible to run complex machine learning models on these devices. However, the power of mobile devices is still limited compared to desktop computers, which are better suited for running computationally intensive machine learning algorithms.

The Calculations Behind Machine Learning

------------------------------------------

Machine learning involves a range of calculations, including matrix operations and activation functions. The most famous cost measure in machine learning is the mean squared error (MSE), which measures the difference between predicted and actual outputs. Backpropagation, an algorithm used to optimize the weights of a neural network, is also widely used.

The Importance of Regularization

----------------------------------

Regularization techniques, such as dropout and L1/L2 regularization, are essential for preventing overfitting in machine learning models. Overfitting occurs when a model becomes too complex and starts to fit the noise in the training data rather than the underlying patterns. Regularization helps to prevent this by adding a penalty term to the cost function.

The Role of Hidden Layers

-------------------------

Hidden layers, which are layers between input and output layers, play a crucial role in improving the non-linearity of machine learning models. The more hidden layers used, the more complex the model becomes, but also the more precise the predictions can be. However, this comes with a risk of overfitting.

The Limitations of Machine Learning

-------------------------------------

Machine learning has several limitations, including the need for large amounts of data and computational resources. Additionally, machine learning models can suffer from overfitting, which means they may not generalize well to new data. Furthermore, machine learning models require careful tuning of hyperparameters, such as learning rates and regularization strengths.

The Future of Machine Learning

------------------------------

As machine learning continues to evolve, we can expect to see significant advances in areas such as computer vision, natural language processing, and predictive analytics. The development of more efficient algorithms, increased computing power, and the availability of large datasets will all contribute to the advancement of machine learning technology.

The Code Behind Machine Learning

---------------------------------

Machine learning code is often written in popular languages such as Python, Java, or C++. Deep learning frameworks like TensorFlow, PyTorch, and Keras provide an interface for building and training neural networks. The code typically involves a range of operations, including matrix multiplications, activations, and backpropagation.

Conclusion

----------

Machine learning has come a long way since its inception in the 1950s. From the early days of neural networks to the development of deep learning frameworks like TensorFlow, machine learning has evolved significantly over the years. As we move forward, it's essential to continue pushing the boundaries of what is possible with machine learning and artificial intelligence.

"WEBVTTKind: captionsLanguage: enso this talk being just half an hour so i am repeating myself for those who have been here a bit before is obviously uh it's a high level presentation about what you can do with javascript in and machine learning and what libraries are out there and what you know current libraries are trying to overcome so there are a lot of things that you can do in javascript right now with pure javascript first you can do a cat recognition for those who like cats so i'm just showing you up hopefully it will work if it does sorry i need ah here we go and oh it found a cat in the middle of the picture i don't know if you there we go so you can do a digital recognition which is a classic uh machine learning example so here i'm gonna draw a seven very roughly recognize oh it's a nice up i recognize here it's rubbish it's very rough obviously my handwriting is however but yeah other interesting applications um so self-driving car simulations obviously you know tesla being in the media and stuff but there is a the mit did a a course on self-driving gas and and so there was several in tesla and to simulate how you know you need to recognize or adjust the parameters they they've done this simulation which is on top of a famous uh javascript framework called confnet gs and basically you can model your red car and see how it goes and how fast you can go so in my presentation i have all the links that you need if you want to experience these libraries image manipulation so to file uh here lighthouse and you can see how far how fast it is to actually manipulate the data so grayscale the picture uh draw the edges flip the picture or resize it zoom it actually zoomed and flipped actually a bit reverse but what as you can do you know for developers that's interesting the automatic logo generator so i'm gonna try to do it live so camgs and if you go down you'll see they it generates different types of logos like you know just to improve your creativity or add you know arrows to your creativity uh the other thing you machine learning you not only work with numbers you can work with words as well so classif example are spam filters um and uh yeah email filtering um or you know when you work when you have a salary on your iphone or or search boxes it's all about work processing so it's called natural language processing and there is also this uh project that i found on and on github recently it's a ai for robots so for those um i think jessica you talked about sensor and anna was talking about arduinos so this combines you know artificial intelligence and arduino and robots so you can have fun with everything and all in node.js and node very high level uh i'll try to expose the key concepts of machine learning so we oppose supervised to unsupervised machine learning so supervisors when you you have a specific set of inputs and you know uh your output you know uh what to expect at the end of the day so you're trying to find a function if you want that will map all the x variables to the targeted output y so typical uh there are two types like classification when it's a discrete set of outputs so if you are you know let's say you have a set of colors so you categorize red or blue or your emails if it's spam or not spam it's classification problems then you have regular regression problems so regression is when the output variable is continuous so it can be in dollars or weights for instance if you think of let's say you wanted to do a study on property prices and you would think of well what's what's the location what's the square footage whole number of rooms uh you know does it have a garage a swimming pool all those features and you try to have at the end of the day how much your property would be worth so that's kind of regression uh problem so examples of algorithms that support that it's a linear regression uh support vector machines and random force um random force is like a tree and basically you do multiple trees randomly you set up the parameters randomly so that's why we call it random forest unsupervised learning is when you don't actually know the output or you want actually to discover the different groupings and the different classifications out of the blue so you have clustering problems and association problems so clustering um given a certain set of data it's used for instance in in the medical world like if you have a a certain set of features uh your sex your age your your health habits uh uh how uh how and uh or in biology in general like if you have a certain set of features uh and you group them what type of disease or what type of plants you come to associations is to discover the rules among this data so really at the high level it's supervised versus unsupervised so that's a classic um description of neural network which is the one of the main algorithm in machine learning so you have a set of inputs so as i said before for instance in the real estate example the square footage the number of rooms sets your input layer so you have a number the number of inputs are the number of features and the output so here you have two different outputs it could be a spam not spam or it could be a just one output but a price uh you have an output layer and in between the output and out sorry in between the input and the output layers you can have hidden layers and there can be uh uh like it can be from one to n number of uh of layers depending on the conclusion complexity and here it is where it's more an art than a science to decide which number of hidden layers are in a network but this might not appeal to everybody in terms of uh understanding of a neural network i find sometimes it's easier to have this kind of representation like a typical tabula or presentation so the features are um you know from you can see in terms of widths of the data set and the number of examples are like the height of x in that example and so it's uh obviously it's it looks like a matrix and you try to arrive at the for each example you know uh which output you're gonna get so it's you try to approximate the y the column y that you have to next to the matrix so you're trying to find out the uh the function or the the weights that you apply to the your data set the the x and this approximation gives you a it's called a y-hat which is um you know what outputs given a current model so uh in in mathematical terms it would be just it's f of x gives you a y and you try to minimize y hat compared to hawaii i don't know if it makes sense to everybody yes this one just looks easier for you yeah i don't know it's uh i say maybe the second representation makes you more i understand that you know you you have all this number of examples the output that you expect the why and we are trying the game is really to try to minimize uh why and why hearts okay and we are and to basically to do that you adjust you you you adjust the w which are the weights that you apply to the number of features y hat it's basically the result of your model of w times the x if you want the different weights that you apply to your data set so basically this thing called perceptron is just one part of let's say what i showed in the picture of the person from just this part so you have a different number of inputs your plan number of whites in the middle and you activate them and you have one neuron in your network so the in terms of calculations if you apply to your inputs uh different weights you sum them like in a multi linear regression and you activate them making them like you know a binary is it zero or one do you accept it in your model or not and so this perceptron is or all the hidden layers and the output layers are calculated in that manner you apply a different set of weights between the input and the hidden layers and a different set of weights between hidden layer and output layer and you at the end of the day you try to compare the output layer to the calculations from your weights and the different layers to the output layer and you try to minimize uh this error between what you've calculated and what is actual does it make sense it makes sense so another key concept and it was discovered relatively recently recently in that case means in the last 15 years is the concept of back propagation so you can do this calculations about weights in one way but really uh for it to improve and so it's that the network learns you have this concept of back propagation so the you you back up the errors that you found out at the way that you calculate at the end of the day between the um the calculated output and the real output you try to integrate it backwards into the weights um into the weights that are between the hidden and output layer and between the input and hidden layer so how um so you re-integrate the error and it's a iterative process until you your error is close to zero or as close to zero as possible and that can be done basically uh thanks to um because we all know the properties of the activation functions so this is possible to to go backwards or to back propagate your error thanks to the derivatives so activation these are examples of activation functions so i really you have to think of activation functions as trying to squeeze a number here in this case for the logistic functions between 0 and 1 you know that everything between infinity minus infinity plus infinity will be squeezed between 0 and 1 i don't know how you say it in english sorry just ten times ten h okay it's you squeeze every number from minus infinity to plus infinity between minus one and plus one and we can back propagate uh and do this inaudible process because actually mathematically we know what the derivative of this um of these different functions are and i don't know if you have recollection from you are doing linear algebra and maybe in high school or in college when you want so most cost functions have a convex feature okay and so um you know what convex is like it looks like like a u basically and when you want to minimize so to go to the bottom to find the local minimum on a on a convex function you use the derivative and try to go to the zero basically so you use a slope of the curve in in the convex function to try to reach the minimum as a local minimum so that's that's how we achieve back propagation and that's how we can achieve the whole iterative process of improvement so you calculate a certain output you calculate the cost error and back propagate it into the weights and you and then you there is a learning rate which makes it improve every time so these are the classic uh cost functions the most i guess the most known one in terms of statistics is a mean square error but you have a number of different types of squares okay enough math for today though so obviously all these calculations uh you know at the end of the day you need performance from your computer to um to realize this and the browser obviously is not the uh the lowest level you can do your calculations and most libraries would be in uh c plus plus i don't think they're in any assembly to my knowledge but they would be in in a very in much more low level language however when you handle calculations it's often done in python and python is just a wrapper around these c plus plus libraries that being said if we are just doing uh python versus javascript performance comparisons uh most of the time javascript with the improvements of the v8 engine and you know the different browser engines it's actually uh faster than python uh it's just that when you do uh deep uh learning networks that uh when you you do like image recognition and not just of cats because here it was a bit cheating the cat was recognized because we could uh find two ears like you know the the sign of so that's it was a shortcut but if you want to do it at a production level like you know at google size you need really like a lot of gpus to do that and why gpu is because coming from the gaming world that's where most calculations for refreshing screens come from and so they are very much used to all the vectors and matrix calculations that are actually used in machine learning so the leader in this in that field is nvidia or when you have nvidia a gpu cat they have actually also created libraries that enable you advanced mathematical operations on it so in the javascript world we are used to the simplest way to do that is array of arrays but it's actually not the most performing a way to do that uh for accessing a particular item you do what you know given a matrix a an array of arrays you you access your elements by you know i mean a i j which you know i'm sure everybody would be able to do but it's not if you want to iterate over that or access or transpose or in this metrics it it won't be very fast one uh faster way uh to do that is uh to use typed arrays which is my second example and it's it's faster but the thing is here the you you can see this so this is just the same matrix as this one but it's just flat it has been flattened and here the index is um so as you can see it's a three rows and three columns matrix and here you can see that the index has been fixed so one so it's very fast using typed arrays but to access the the different elements when you want to do uh higher order operations is is a bit cumbersome so there is a compromise where this index the three in my example here is actually um it's sorry it's a general linear function so here this formula basically becomes to that so so here you can really switch rows and columns very quickly you can transpose numbers in your matrix much quicker and there's been a library that has used this idea of strided arrays and which is ndrs it's used in a number of low-level and high-level libraries numgs which is a kind of numpy for those who are familiar with python that tries to use the same kind of functions that numpy or matlab have has keras takes the name of of a famous uh library which is a higher order library a higher level library on top of tensorflow or tiano which are more lower level or matrix calculations so keras it would be easier to do artificial neural network a recurrent neural network it's it's good for prototyping machine learning models the only thing the keras javascript model you can only in a way do a forward propagation you can only do what's called inference and not back propagation so the training of your models won't be done in javascript it would be done in in python and then once you have your model you use javascript to visualize these things so tensorfire is the latest library it uses ndis but it also uses webgl under the hood so other ways to do calculations in on the cpu is using uh web assembly there are fewer examples of that and they are less mature but it's certainly avenues to explore to improve the speed of calculations on the cpu and also to increase the the parallelism of calculations in browsers you you can use service workers that being said there is a cost of transferring your data from your browser thread to the service workers and back so there will be a slight delay compared to pure cpu so as i mentioned you can do calculations leveraging your cpu or you can leverage uh webgl so a number of uh of libraries have emerged and the latest one being tensorfire which was i saw that on twitter like uh two weeks ago so it's really recent but um they haven't for the moment open source their code so as i said i found the npm library but i couldn't find a github basically it leverages jrsl so the the language that is used with webgl you can use also uh leverage uh your gpu differently using opencl or web gpu but then i guess it's not uh it will be a different setup it will be using a node rather than purely the browser and usually it's you can it's restricted to a certain number of browsers it's not cross browser compatible right um so that was my high level uh in terms of concepts from machine learning uh i wanted to export a number of blogs and libraries that you can use in javascript or courses if you are interested in the area so this bright camber did an old blog in machine learning addressing not neural network but a lot of unsupervised learning algorithm and the second one is that i mentioned here andrej capaci is is quite interesting because this guy is relatively famous he developed one of the first deep learning library in javascript called confnet gs to do convolutional neural networks on pictures uh and he's uh he used to work for so he's from stanford and he used to do uh work for open li and recently he has been he has been hired by tesla to improve machine learning for cars so there are a number of javascript specific resources so content us unfortunately is no longer maintained tensorflow you have a nice tensorflow playground to play with which is here you can add a number of hidden layers different features and you can so it's nice to play around in terms of what you can achieve and how long it takes to to improve your model um so it's tensorflow playground so it's it's a really it's written in typescript and uh it's only a very small set of the tensorflow library do you know actually do do people understand what a tensor is by any chance no okay so you should think of tensor as a container like a a number a scalar would be a tensor of of size zero a vector so i'm gonna go to so a vector would be a tensor of one dimension a matrix with rows and columns would be a tensor of two dimensions and a cube so three dimensions would be a tensorflow a tensor of three dimensions so it's you have to think of it as a container so libraries uh and if if you are more interested so you can get involved at different levels maybe some like to go into the lower level so the matrix and vector calculations uh or you can go higher level and talk about different types of architecture and convert yes and more recently this one knit optic which tries to do neural evolution so neural evolution for those who don't know it's it's kind of uh it tries to it's a gene kind of genetic algorithm which means that they let's say you can't to randomly generate a number of uh of examples and basically you you try to the you try to eliminate those who don't work as much so it's it's a very darwinian process in a way you generate a number of of examples you you keep the ones who survive the longest then you regenerate a number of set of examples and and and so forth back in so um so there has been a number of um universities uh involved in uh to do machine learning in javascript so one would i would say is more on the edges from the university of tokyo maldives they do libraries which are relatively very japanese friendly like sushi too tempura sukiyaki but they have like sushi too is for instance for matrix calculation sukiyaki is for on top of sushi 2 to do different machine learning models they do distributed computing so they have created a framework where you have different browsers and basically you leverage the capabilities from you know maybe using web sockets so you can connect different uh computers different browsers and you leverage the cpus of these different browsers you have two one swiss university which the other two universities are maybe less known or less marketed in a way i had really i learned just by chance that they existed and they have actually quite a few uh libraries uh on npm on github um the last one actually doesn't have a npm but it does have a github i asked them if you wanted to leverage npm but the guy didn't see the point of using uh javascript who is not yes i don't know that's his point but they have a nice online uh laboratory where you can actually download some files and you can you know use the most common machine learning uh libraries on it to you know try to um create a model um coursera i personally so i'm not an expert in the field i uh i'm still doing it actually um i recommend the corsair course from uh from underneath who is a uh who used to work at google um and and baidu he was a chief um machine learning or chief artificial intelligence guy from baidu the chinese company jeffrey hinton saw his course on our networks in corsa jeffrey hinton is one of the leaders in machine learning and it's it's quite unique that you can find his you know course from him completely free on conserva udacity proposes a number of of nanodegrees in the field the example that i showed you on deep traffic the mit course a lot of people from the self-driving engineer uh were involved in that so i don't oh i don't have network so unfortunately on behavior but basically they participated in in this in this setup as well there are lots of resources on the internet uh blogs uh which are not most of the time not javascript specific the javascript ones are the these ones that i mentioned yes so you can get involved also to encourage getting involved there is a ai grant where for whatever projects and tensorfire the latest library that i mentioned they participated in the ai grant competition and they won 5 000 from that to to when they created tensorflow or kaggle kaggle actually was co-founded by an australian guy and it's a it's a website where people can do machine learning competitions and and there are open data sets and there is a forum where you share your tips and tricks to improve the accuracy of your models sorry there was very fast and uh you know very involved but if you have any questions happy sorry for all the links yes sure uh yeah i will put this yes graphical forum machine learning activity effect screen rendering is it going to uh if you've got an app that yeah so turn safari i think you can try the uh the demos that they have on your mobile uh i personally don't know uh you know the performance uh comparisons for that i the one of the thing is i guess the calculations would be i mean i don't i don't think your gpu on the mobile is in terms of power it's it's great that great so i don't think you know um i mean it will improve a bit in a way because it's paralyzed so it's uh you know in parallel to your cpu but in terms of pure raw performance i don't think it's right that being said i know that google with tensorflow has tried to you know i mean democratize the use of its library and so they encourage it and they do android apps on that but i personally haven't tested it so so the whites you have to think it's the same set of weights that you apply to the examples okay so you will try to follow them together very quickly um you can vary yeah so obviously it's different set of inputs but basically it's m and you can see it's a number of features any number of features you multiply each feature but it's white okay so first it's randomly generated and then so so first you randomly generate these weights so then you do the calculations so the like the matrix calculations okay you activate them using one of the functions that i showed you before and um you do that for every neuron every layer and gives you y hat the what comes from your weights okay so then you try to calculate costs you have a cost measure so the most famous one is a mean square arrow of comparing it's basically y y minus y hat square divided by the number of examples okay so once you have that measure of error okay you you back propagate this error into thanks to the derivatives of the active er evaluation functions back to the initial input okay so it changed and and then it's an iterative process thanks to what's called the learning rate where you add that at the end of the day you will me you will diminish the error that you calculated a numerator example or coding example if you want on a like the number of uh i suppose dimensions that you can work on the problem is it just for like a two dimension problem and then anything after that the error rate you know just can't really model it very well uh so actually the fact that you have hidden layers and and you have so it improves uh if you want the non-linearity of your form so let me show you what i mean so you're thinking of giving us a set like that you can see it's it's not clear if you were just doing linear regression it would be a straight line okay so if you were doing just input outputs that's what you would expect it's a it's a linear regression but the fact that you implement a hidden layer in the middle give improves the non-linearity of your uh of your model if you want so uh i i've done that example in in javascript so just logistic regression and just input output without hidden layers would give you a straight line okay but if you had a hidden layer and and i said i mean here the color is in titan but i've done the thing in javascript you the fact that you a hidden layer they will find you know here so you can see here this invariable for hidden layer size three you can see you you add a non-linearity to your problem so and the more you add basically the more precise in your non-linearity you can go okay so then it's a complex then there is i didn't want to go over that but you can overfit your data so then in terms of uh if you want to predict you know a cert with your model you have to be uh very careful that it doesn't offer that your model doesn't overfit your test data but if you do that what's the cost in terms of processing oh obviously that's numerical complexities i mean i i i don't have the exact answer but you know it's it's adding a role it's basically adding uh a few mattresses to your uh to your calculations so in terms of i know computer scientists calculate in terms of or something it would be at least i mean square added square capacity any other question youso this talk being just half an hour so i am repeating myself for those who have been here a bit before is obviously uh it's a high level presentation about what you can do with javascript in and machine learning and what libraries are out there and what you know current libraries are trying to overcome so there are a lot of things that you can do in javascript right now with pure javascript first you can do a cat recognition for those who like cats so i'm just showing you up hopefully it will work if it does sorry i need ah here we go and oh it found a cat in the middle of the picture i don't know if you there we go so you can do a digital recognition which is a classic uh machine learning example so here i'm gonna draw a seven very roughly recognize oh it's a nice up i recognize here it's rubbish it's very rough obviously my handwriting is however but yeah other interesting applications um so self-driving car simulations obviously you know tesla being in the media and stuff but there is a the mit did a a course on self-driving gas and and so there was several in tesla and to simulate how you know you need to recognize or adjust the parameters they they've done this simulation which is on top of a famous uh javascript framework called confnet gs and basically you can model your red car and see how it goes and how fast you can go so in my presentation i have all the links that you need if you want to experience these libraries image manipulation so to file uh here lighthouse and you can see how far how fast it is to actually manipulate the data so grayscale the picture uh draw the edges flip the picture or resize it zoom it actually zoomed and flipped actually a bit reverse but what as you can do you know for developers that's interesting the automatic logo generator so i'm gonna try to do it live so camgs and if you go down you'll see they it generates different types of logos like you know just to improve your creativity or add you know arrows to your creativity uh the other thing you machine learning you not only work with numbers you can work with words as well so classif example are spam filters um and uh yeah email filtering um or you know when you work when you have a salary on your iphone or or search boxes it's all about work processing so it's called natural language processing and there is also this uh project that i found on and on github recently it's a ai for robots so for those um i think jessica you talked about sensor and anna was talking about arduinos so this combines you know artificial intelligence and arduino and robots so you can have fun with everything and all in node.js and node very high level uh i'll try to expose the key concepts of machine learning so we oppose supervised to unsupervised machine learning so supervisors when you you have a specific set of inputs and you know uh your output you know uh what to expect at the end of the day so you're trying to find a function if you want that will map all the x variables to the targeted output y so typical uh there are two types like classification when it's a discrete set of outputs so if you are you know let's say you have a set of colors so you categorize red or blue or your emails if it's spam or not spam it's classification problems then you have regular regression problems so regression is when the output variable is continuous so it can be in dollars or weights for instance if you think of let's say you wanted to do a study on property prices and you would think of well what's what's the location what's the square footage whole number of rooms uh you know does it have a garage a swimming pool all those features and you try to have at the end of the day how much your property would be worth so that's kind of regression uh problem so examples of algorithms that support that it's a linear regression uh support vector machines and random force um random force is like a tree and basically you do multiple trees randomly you set up the parameters randomly so that's why we call it random forest unsupervised learning is when you don't actually know the output or you want actually to discover the different groupings and the different classifications out of the blue so you have clustering problems and association problems so clustering um given a certain set of data it's used for instance in in the medical world like if you have a a certain set of features uh your sex your age your your health habits uh uh how uh how and uh or in biology in general like if you have a certain set of features uh and you group them what type of disease or what type of plants you come to associations is to discover the rules among this data so really at the high level it's supervised versus unsupervised so that's a classic um description of neural network which is the one of the main algorithm in machine learning so you have a set of inputs so as i said before for instance in the real estate example the square footage the number of rooms sets your input layer so you have a number the number of inputs are the number of features and the output so here you have two different outputs it could be a spam not spam or it could be a just one output but a price uh you have an output layer and in between the output and out sorry in between the input and the output layers you can have hidden layers and there can be uh uh like it can be from one to n number of uh of layers depending on the conclusion complexity and here it is where it's more an art than a science to decide which number of hidden layers are in a network but this might not appeal to everybody in terms of uh understanding of a neural network i find sometimes it's easier to have this kind of representation like a typical tabula or presentation so the features are um you know from you can see in terms of widths of the data set and the number of examples are like the height of x in that example and so it's uh obviously it's it looks like a matrix and you try to arrive at the for each example you know uh which output you're gonna get so it's you try to approximate the y the column y that you have to next to the matrix so you're trying to find out the uh the function or the the weights that you apply to the your data set the the x and this approximation gives you a it's called a y-hat which is um you know what outputs given a current model so uh in in mathematical terms it would be just it's f of x gives you a y and you try to minimize y hat compared to hawaii i don't know if it makes sense to everybody yes this one just looks easier for you yeah i don't know it's uh i say maybe the second representation makes you more i understand that you know you you have all this number of examples the output that you expect the why and we are trying the game is really to try to minimize uh why and why hearts okay and we are and to basically to do that you adjust you you you adjust the w which are the weights that you apply to the number of features y hat it's basically the result of your model of w times the x if you want the different weights that you apply to your data set so basically this thing called perceptron is just one part of let's say what i showed in the picture of the person from just this part so you have a different number of inputs your plan number of whites in the middle and you activate them and you have one neuron in your network so the in terms of calculations if you apply to your inputs uh different weights you sum them like in a multi linear regression and you activate them making them like you know a binary is it zero or one do you accept it in your model or not and so this perceptron is or all the hidden layers and the output layers are calculated in that manner you apply a different set of weights between the input and the hidden layers and a different set of weights between hidden layer and output layer and you at the end of the day you try to compare the output layer to the calculations from your weights and the different layers to the output layer and you try to minimize uh this error between what you've calculated and what is actual does it make sense it makes sense so another key concept and it was discovered relatively recently recently in that case means in the last 15 years is the concept of back propagation so you can do this calculations about weights in one way but really uh for it to improve and so it's that the network learns you have this concept of back propagation so the you you back up the errors that you found out at the way that you calculate at the end of the day between the um the calculated output and the real output you try to integrate it backwards into the weights um into the weights that are between the hidden and output layer and between the input and hidden layer so how um so you re-integrate the error and it's a iterative process until you your error is close to zero or as close to zero as possible and that can be done basically uh thanks to um because we all know the properties of the activation functions so this is possible to to go backwards or to back propagate your error thanks to the derivatives so activation these are examples of activation functions so i really you have to think of activation functions as trying to squeeze a number here in this case for the logistic functions between 0 and 1 you know that everything between infinity minus infinity plus infinity will be squeezed between 0 and 1 i don't know how you say it in english sorry just ten times ten h okay it's you squeeze every number from minus infinity to plus infinity between minus one and plus one and we can back propagate uh and do this inaudible process because actually mathematically we know what the derivative of this um of these different functions are and i don't know if you have recollection from you are doing linear algebra and maybe in high school or in college when you want so most cost functions have a convex feature okay and so um you know what convex is like it looks like like a u basically and when you want to minimize so to go to the bottom to find the local minimum on a on a convex function you use the derivative and try to go to the zero basically so you use a slope of the curve in in the convex function to try to reach the minimum as a local minimum so that's that's how we achieve back propagation and that's how we can achieve the whole iterative process of improvement so you calculate a certain output you calculate the cost error and back propagate it into the weights and you and then you there is a learning rate which makes it improve every time so these are the classic uh cost functions the most i guess the most known one in terms of statistics is a mean square error but you have a number of different types of squares okay enough math for today though so obviously all these calculations uh you know at the end of the day you need performance from your computer to um to realize this and the browser obviously is not the uh the lowest level you can do your calculations and most libraries would be in uh c plus plus i don't think they're in any assembly to my knowledge but they would be in in a very in much more low level language however when you handle calculations it's often done in python and python is just a wrapper around these c plus plus libraries that being said if we are just doing uh python versus javascript performance comparisons uh most of the time javascript with the improvements of the v8 engine and you know the different browser engines it's actually uh faster than python uh it's just that when you do uh deep uh learning networks that uh when you you do like image recognition and not just of cats because here it was a bit cheating the cat was recognized because we could uh find two ears like you know the the sign of so that's it was a shortcut but if you want to do it at a production level like you know at google size you need really like a lot of gpus to do that and why gpu is because coming from the gaming world that's where most calculations for refreshing screens come from and so they are very much used to all the vectors and matrix calculations that are actually used in machine learning so the leader in this in that field is nvidia or when you have nvidia a gpu cat they have actually also created libraries that enable you advanced mathematical operations on it so in the javascript world we are used to the simplest way to do that is array of arrays but it's actually not the most performing a way to do that uh for accessing a particular item you do what you know given a matrix a an array of arrays you you access your elements by you know i mean a i j which you know i'm sure everybody would be able to do but it's not if you want to iterate over that or access or transpose or in this metrics it it won't be very fast one uh faster way uh to do that is uh to use typed arrays which is my second example and it's it's faster but the thing is here the you you can see this so this is just the same matrix as this one but it's just flat it has been flattened and here the index is um so as you can see it's a three rows and three columns matrix and here you can see that the index has been fixed so one so it's very fast using typed arrays but to access the the different elements when you want to do uh higher order operations is is a bit cumbersome so there is a compromise where this index the three in my example here is actually um it's sorry it's a general linear function so here this formula basically becomes to that so so here you can really switch rows and columns very quickly you can transpose numbers in your matrix much quicker and there's been a library that has used this idea of strided arrays and which is ndrs it's used in a number of low-level and high-level libraries numgs which is a kind of numpy for those who are familiar with python that tries to use the same kind of functions that numpy or matlab have has keras takes the name of of a famous uh library which is a higher order library a higher level library on top of tensorflow or tiano which are more lower level or matrix calculations so keras it would be easier to do artificial neural network a recurrent neural network it's it's good for prototyping machine learning models the only thing the keras javascript model you can only in a way do a forward propagation you can only do what's called inference and not back propagation so the training of your models won't be done in javascript it would be done in in python and then once you have your model you use javascript to visualize these things so tensorfire is the latest library it uses ndis but it also uses webgl under the hood so other ways to do calculations in on the cpu is using uh web assembly there are fewer examples of that and they are less mature but it's certainly avenues to explore to improve the speed of calculations on the cpu and also to increase the the parallelism of calculations in browsers you you can use service workers that being said there is a cost of transferring your data from your browser thread to the service workers and back so there will be a slight delay compared to pure cpu so as i mentioned you can do calculations leveraging your cpu or you can leverage uh webgl so a number of uh of libraries have emerged and the latest one being tensorfire which was i saw that on twitter like uh two weeks ago so it's really recent but um they haven't for the moment open source their code so as i said i found the npm library but i couldn't find a github basically it leverages jrsl so the the language that is used with webgl you can use also uh leverage uh your gpu differently using opencl or web gpu but then i guess it's not uh it will be a different setup it will be using a node rather than purely the browser and usually it's you can it's restricted to a certain number of browsers it's not cross browser compatible right um so that was my high level uh in terms of concepts from machine learning uh i wanted to export a number of blogs and libraries that you can use in javascript or courses if you are interested in the area so this bright camber did an old blog in machine learning addressing not neural network but a lot of unsupervised learning algorithm and the second one is that i mentioned here andrej capaci is is quite interesting because this guy is relatively famous he developed one of the first deep learning library in javascript called confnet gs to do convolutional neural networks on pictures uh and he's uh he used to work for so he's from stanford and he used to do uh work for open li and recently he has been he has been hired by tesla to improve machine learning for cars so there are a number of javascript specific resources so content us unfortunately is no longer maintained tensorflow you have a nice tensorflow playground to play with which is here you can add a number of hidden layers different features and you can so it's nice to play around in terms of what you can achieve and how long it takes to to improve your model um so it's tensorflow playground so it's it's a really it's written in typescript and uh it's only a very small set of the tensorflow library do you know actually do do people understand what a tensor is by any chance no okay so you should think of tensor as a container like a a number a scalar would be a tensor of of size zero a vector so i'm gonna go to so a vector would be a tensor of one dimension a matrix with rows and columns would be a tensor of two dimensions and a cube so three dimensions would be a tensorflow a tensor of three dimensions so it's you have to think of it as a container so libraries uh and if if you are more interested so you can get involved at different levels maybe some like to go into the lower level so the matrix and vector calculations uh or you can go higher level and talk about different types of architecture and convert yes and more recently this one knit optic which tries to do neural evolution so neural evolution for those who don't know it's it's kind of uh it tries to it's a gene kind of genetic algorithm which means that they let's say you can't to randomly generate a number of uh of examples and basically you you try to the you try to eliminate those who don't work as much so it's it's a very darwinian process in a way you generate a number of of examples you you keep the ones who survive the longest then you regenerate a number of set of examples and and and so forth back in so um so there has been a number of um universities uh involved in uh to do machine learning in javascript so one would i would say is more on the edges from the university of tokyo maldives they do libraries which are relatively very japanese friendly like sushi too tempura sukiyaki but they have like sushi too is for instance for matrix calculation sukiyaki is for on top of sushi 2 to do different machine learning models they do distributed computing so they have created a framework where you have different browsers and basically you leverage the capabilities from you know maybe using web sockets so you can connect different uh computers different browsers and you leverage the cpus of these different browsers you have two one swiss university which the other two universities are maybe less known or less marketed in a way i had really i learned just by chance that they existed and they have actually quite a few uh libraries uh on npm on github um the last one actually doesn't have a npm but it does have a github i asked them if you wanted to leverage npm but the guy didn't see the point of using uh javascript who is not yes i don't know that's his point but they have a nice online uh laboratory where you can actually download some files and you can you know use the most common machine learning uh libraries on it to you know try to um create a model um coursera i personally so i'm not an expert in the field i uh i'm still doing it actually um i recommend the corsair course from uh from underneath who is a uh who used to work at google um and and baidu he was a chief um machine learning or chief artificial intelligence guy from baidu the chinese company jeffrey hinton saw his course on our networks in corsa jeffrey hinton is one of the leaders in machine learning and it's it's quite unique that you can find his you know course from him completely free on conserva udacity proposes a number of of nanodegrees in the field the example that i showed you on deep traffic the mit course a lot of people from the self-driving engineer uh were involved in that so i don't oh i don't have network so unfortunately on behavior but basically they participated in in this in this setup as well there are lots of resources on the internet uh blogs uh which are not most of the time not javascript specific the javascript ones are the these ones that i mentioned yes so you can get involved also to encourage getting involved there is a ai grant where for whatever projects and tensorfire the latest library that i mentioned they participated in the ai grant competition and they won 5 000 from that to to when they created tensorflow or kaggle kaggle actually was co-founded by an australian guy and it's a it's a website where people can do machine learning competitions and and there are open data sets and there is a forum where you share your tips and tricks to improve the accuracy of your models sorry there was very fast and uh you know very involved but if you have any questions happy sorry for all the links yes sure uh yeah i will put this yes graphical forum machine learning activity effect screen rendering is it going to uh if you've got an app that yeah so turn safari i think you can try the uh the demos that they have on your mobile uh i personally don't know uh you know the performance uh comparisons for that i the one of the thing is i guess the calculations would be i mean i don't i don't think your gpu on the mobile is in terms of power it's it's great that great so i don't think you know um i mean it will improve a bit in a way because it's paralyzed so it's uh you know in parallel to your cpu but in terms of pure raw performance i don't think it's right that being said i know that google with tensorflow has tried to you know i mean democratize the use of its library and so they encourage it and they do android apps on that but i personally haven't tested it so so the whites you have to think it's the same set of weights that you apply to the examples okay so you will try to follow them together very quickly um you can vary yeah so obviously it's different set of inputs but basically it's m and you can see it's a number of features any number of features you multiply each feature but it's white okay so first it's randomly generated and then so so first you randomly generate these weights so then you do the calculations so the like the matrix calculations okay you activate them using one of the functions that i showed you before and um you do that for every neuron every layer and gives you y hat the what comes from your weights okay so then you try to calculate costs you have a cost measure so the most famous one is a mean square arrow of comparing it's basically y y minus y hat square divided by the number of examples okay so once you have that measure of error okay you you back propagate this error into thanks to the derivatives of the active er evaluation functions back to the initial input okay so it changed and and then it's an iterative process thanks to what's called the learning rate where you add that at the end of the day you will me you will diminish the error that you calculated a numerator example or coding example if you want on a like the number of uh i suppose dimensions that you can work on the problem is it just for like a two dimension problem and then anything after that the error rate you know just can't really model it very well uh so actually the fact that you have hidden layers and and you have so it improves uh if you want the non-linearity of your form so let me show you what i mean so you're thinking of giving us a set like that you can see it's it's not clear if you were just doing linear regression it would be a straight line okay so if you were doing just input outputs that's what you would expect it's a it's a linear regression but the fact that you implement a hidden layer in the middle give improves the non-linearity of your uh of your model if you want so uh i i've done that example in in javascript so just logistic regression and just input output without hidden layers would give you a straight line okay but if you had a hidden layer and and i said i mean here the color is in titan but i've done the thing in javascript you the fact that you a hidden layer they will find you know here so you can see here this invariable for hidden layer size three you can see you you add a non-linearity to your problem so and the more you add basically the more precise in your non-linearity you can go okay so then it's a complex then there is i didn't want to go over that but you can overfit your data so then in terms of uh if you want to predict you know a cert with your model you have to be uh very careful that it doesn't offer that your model doesn't overfit your test data but if you do that what's the cost in terms of processing oh obviously that's numerical complexities i mean i i i don't have the exact answer but you know it's it's adding a role it's basically adding uh a few mattresses to your uh to your calculations so in terms of i know computer scientists calculate in terms of or something it would be at least i mean square added square capacity any other question youso this talk being just half an hour so i am repeating myself for those who have been here a bit before is obviously uh it's a high level presentation about what you can do with javascript in and machine learning and what libraries are out there and what you know current libraries are trying to overcome so there are a lot of things that you can do in javascript right now with pure javascript first you can do a cat recognition for those who like cats so i'm just showing you up hopefully it will work if it does sorry i need ah here we go and oh it found a cat in the middle of the picture i don't know if you there we go so you can do a digital recognition which is a classic uh machine learning example so here i'm gonna draw a seven very roughly recognize oh it's a nice up i recognize here it's rubbish it's very rough obviously my handwriting is however but yeah other interesting applications um so self-driving car simulations obviously you know tesla being in the media and stuff but there is a the mit did a a course on self-driving gas and and so there was several in tesla and to simulate how you know you need to recognize or adjust the parameters they they've done this simulation which is on top of a famous uh javascript framework called confnet gs and basically you can model your red car and see how it goes and how fast you can go so in my presentation i have all the links that you need if you want to experience these libraries image manipulation so to file uh here lighthouse and you can see how far how fast it is to actually manipulate the data so grayscale the picture uh draw the edges flip the picture or resize it zoom it actually zoomed and flipped actually a bit reverse but what as you can do you know for developers that's interesting the automatic logo generator so i'm gonna try to do it live so camgs and if you go down you'll see they it generates different types of logos like you know just to improve your creativity or add you know arrows to your creativity uh the other thing you machine learning you not only work with numbers you can work with words as well so classif example are spam filters um and uh yeah email filtering um or you know when you work when you have a salary on your iphone or or search boxes it's all about work processing so it's called natural language processing and there is also this uh project that i found on and on github recently it's a ai for robots so for those um i think jessica you talked about sensor and anna was talking about arduinos so this combines you know artificial intelligence and arduino and robots so you can have fun with everything and all in node.js and node very high level uh i'll try to expose the key concepts of machine learning so we oppose supervised to unsupervised machine learning so supervisors when you you have a specific set of inputs and you know uh your output you know uh what to expect at the end of the day so you're trying to find a function if you want that will map all the x variables to the targeted output y so typical uh there are two types like classification when it's a discrete set of outputs so if you are you know let's say you have a set of colors so you categorize red or blue or your emails if it's spam or not spam it's classification problems then you have regular regression problems so regression is when the output variable is continuous so it can be in dollars or weights for instance if you think of let's say you wanted to do a study on property prices and you would think of well what's what's the location what's the square footage whole number of rooms uh you know does it have a garage a swimming pool all those features and you try to have at the end of the day how much your property would be worth so that's kind of regression uh problem so examples of algorithms that support that it's a linear regression uh support vector machines and random force um random force is like a tree and basically you do multiple trees randomly you set up the parameters randomly so that's why we call it random forest unsupervised learning is when you don't actually know the output or you want actually to discover the different groupings and the different classifications out of the blue so you have clustering problems and association problems so clustering um given a certain set of data it's used for instance in in the medical world like if you have a a certain set of features uh your sex your age your your health habits uh uh how uh how and uh or in biology in general like if you have a certain set of features uh and you group them what type of disease or what type of plants you come to associations is to discover the rules among this data so really at the high level it's supervised versus unsupervised so that's a classic um description of neural network which is the one of the main algorithm in machine learning so you have a set of inputs so as i said before for instance in the real estate example the square footage the number of rooms sets your input layer so you have a number the number of inputs are the number of features and the output so here you have two different outputs it could be a spam not spam or it could be a just one output but a price uh you have an output layer and in between the output and out sorry in between the input and the output layers you can have hidden layers and there can be uh uh like it can be from one to n number of uh of layers depending on the conclusion complexity and here it is where it's more an art than a science to decide which number of hidden layers are in a network but this might not appeal to everybody in terms of uh understanding of a neural network i find sometimes it's easier to have this kind of representation like a typical tabula or presentation so the features are um you know from you can see in terms of widths of the data set and the number of examples are like the height of x in that example and so it's uh obviously it's it looks like a matrix and you try to arrive at the for each example you know uh which output you're gonna get so it's you try to approximate the y the column y that you have to next to the matrix so you're trying to find out the uh the function or the the weights that you apply to the your data set the the x and this approximation gives you a it's called a y-hat which is um you know what outputs given a current model so uh in in mathematical terms it would be just it's f of x gives you a y and you try to minimize y hat compared to hawaii i don't know if it makes sense to everybody yes this one just looks easier for you yeah i don't know it's uh i say maybe the second representation makes you more i understand that you know you you have all this number of examples the output that you expect the why and we are trying the game is really to try to minimize uh why and why hearts okay and we are and to basically to do that you adjust you you you adjust the w which are the weights that you apply to the number of features y hat it's basically the result of your model of w times the x if you want the different weights that you apply to your data set so basically this thing called perceptron is just one part of let's say what i showed in the picture of the person from just this part so you have a different number of inputs your plan number of whites in the middle and you activate them and you have one neuron in your network so the in terms of calculations if you apply to your inputs uh different weights you sum them like in a multi linear regression and you activate them making them like you know a binary is it zero or one do you accept it in your model or not and so this perceptron is or all the hidden layers and the output layers are calculated in that manner you apply a different set of weights between the input and the hidden layers and a different set of weights between hidden layer and output layer and you at the end of the day you try to compare the output layer to the calculations from your weights and the different layers to the output layer and you try to minimize uh this error between what you've calculated and what is actual does it make sense it makes sense so another key concept and it was discovered relatively recently recently in that case means in the last 15 years is the concept of back propagation so you can do this calculations about weights in one way but really uh for it to improve and so it's that the network learns you have this concept of back propagation so the you you back up the errors that you found out at the way that you calculate at the end of the day between the um the calculated output and the real output you try to integrate it backwards into the weights um into the weights that are between the hidden and output layer and between the input and hidden layer so how um so you re-integrate the error and it's a iterative process until you your error is close to zero or as close to zero as possible and that can be done basically uh thanks to um because we all know the properties of the activation functions so this is possible to to go backwards or to back propagate your error thanks to the derivatives so activation these are examples of activation functions so i really you have to think of activation functions as trying to squeeze a number here in this case for the logistic functions between 0 and 1 you know that everything between infinity minus infinity plus infinity will be squeezed between 0 and 1 i don't know how you say it in english sorry just ten times ten h okay it's you squeeze every number from minus infinity to plus infinity between minus one and plus one and we can back propagate uh and do this inaudible process because actually mathematically we know what the derivative of this um of these different functions are and i don't know if you have recollection from you are doing linear algebra and maybe in high school or in college when you want so most cost functions have a convex feature okay and so um you know what convex is like it looks like like a u basically and when you want to minimize so to go to the bottom to find the local minimum on a on a convex function you use the derivative and try to go to the zero basically so you use a slope of the curve in in the convex function to try to reach the minimum as a local minimum so that's that's how we achieve back propagation and that's how we can achieve the whole iterative process of improvement so you calculate a certain output you calculate the cost error and back propagate it into the weights and you and then you there is a learning rate which makes it improve every time so these are the classic uh cost functions the most i guess the most known one in terms of statistics is a mean square error but you have a number of different types of squares okay enough math for today though so obviously all these calculations uh you know at the end of the day you need performance from your computer to um to realize this and the browser obviously is not the uh the lowest level you can do your calculations and most libraries would be in uh c plus plus i don't think they're in any assembly to my knowledge but they would be in in a very in much more low level language however when you handle calculations it's often done in python and python is just a wrapper around these c plus plus libraries that being said if we are just doing uh python versus javascript performance comparisons uh most of the time javascript with the improvements of the v8 engine and you know the different browser engines it's actually uh faster than python uh it's just that when you do uh deep uh learning networks that uh when you you do like image recognition and not just of cats because here it was a bit cheating the cat was recognized because we could uh find two ears like you know the the sign of so that's it was a shortcut but if you want to do it at a production level like you know at google size you need really like a lot of gpus to do that and why gpu is because coming from the gaming world that's where most calculations for refreshing screens come from and so they are very much used to all the vectors and matrix calculations that are actually used in machine learning so the leader in this in that field is nvidia or when you have nvidia a gpu cat they have actually also created libraries that enable you advanced mathematical operations on it so in the javascript world we are used to the simplest way to do that is array of arrays but it's actually not the most performing a way to do that uh for accessing a particular item you do what you know given a matrix a an array of arrays you you access your elements by you know i mean a i j which you know i'm sure everybody would be able to do but it's not if you want to iterate over that or access or transpose or in this metrics it it won't be very fast one uh faster way uh to do that is uh to use typed arrays which is my second example and it's it's faster but the thing is here the you you can see this so this is just the same matrix as this one but it's just flat it has been flattened and here the index is um so as you can see it's a three rows and three columns matrix and here you can see that the index has been fixed so one so it's very fast using typed arrays but to access the the different elements when you want to do uh higher order operations is is a bit cumbersome so there is a compromise where this index the three in my example here is actually um it's sorry it's a general linear function so here this formula basically becomes to that so so here you can really switch rows and columns very quickly you can transpose numbers in your matrix much quicker and there's been a library that has used this idea of strided arrays and which is ndrs it's used in a number of low-level and high-level libraries numgs which is a kind of numpy for those who are familiar with python that tries to use the same kind of functions that numpy or matlab have has keras takes the name of of a famous uh library which is a higher order library a higher level library on top of tensorflow or tiano which are more lower level or matrix calculations so keras it would be easier to do artificial neural network a recurrent neural network it's it's good for prototyping machine learning models the only thing the keras javascript model you can only in a way do a forward propagation you can only do what's called inference and not back propagation so the training of your models won't be done in javascript it would be done in in python and then once you have your model you use javascript to visualize these things so tensorfire is the latest library it uses ndis but it also uses webgl under the hood so other ways to do calculations in on the cpu is using uh web assembly there are fewer examples of that and they are less mature but it's certainly avenues to explore to improve the speed of calculations on the cpu and also to increase the the parallelism of calculations in browsers you you can use service workers that being said there is a cost of transferring your data from your browser thread to the service workers and back so there will be a slight delay compared to pure cpu so as i mentioned you can do calculations leveraging your cpu or you can leverage uh webgl so a number of uh of libraries have emerged and the latest one being tensorfire which was i saw that on twitter like uh two weeks ago so it's really recent but um they haven't for the moment open source their code so as i said i found the npm library but i couldn't find a github basically it leverages jrsl so the the language that is used with webgl you can use also uh leverage uh your gpu differently using opencl or web gpu but then i guess it's not uh it will be a different setup it will be using a node rather than purely the browser and usually it's you can it's restricted to a certain number of browsers it's not cross browser compatible right um so that was my high level uh in terms of concepts from machine learning uh i wanted to export a number of blogs and libraries that you can use in javascript or courses if you are interested in the area so this bright camber did an old blog in machine learning addressing not neural network but a lot of unsupervised learning algorithm and the second one is that i mentioned here andrej capaci is is quite interesting because this guy is relatively famous he developed one of the first deep learning library in javascript called confnet gs to do convolutional neural networks on pictures uh and he's uh he used to work for so he's from stanford and he used to do uh work for open li and recently he has been he has been hired by tesla to improve machine learning for cars so there are a number of javascript specific resources so content us unfortunately is no longer maintained tensorflow you have a nice tensorflow playground to play with which is here you can add a number of hidden layers different features and you can so it's nice to play around in terms of what you can achieve and how long it takes to to improve your model um so it's tensorflow playground so it's it's a really it's written in typescript and uh it's only a very small set of the tensorflow library do you know actually do do people understand what a tensor is by any chance no okay so you should think of tensor as a container like a a number a scalar would be a tensor of of size zero a vector so i'm gonna go to so a vector would be a tensor of one dimension a matrix with rows and columns would be a tensor of two dimensions and a cube so three dimensions would be a tensorflow a tensor of three dimensions so it's you have to think of it as a container so libraries uh and if if you are more interested so you can get involved at different levels maybe some like to go into the lower level so the matrix and vector calculations uh or you can go higher level and talk about different types of architecture and convert yes and more recently this one knit optic which tries to do neural evolution so neural evolution for those who don't know it's it's kind of uh it tries to it's a gene kind of genetic algorithm which means that they let's say you can't to randomly generate a number of uh of examples and basically you you try to the you try to eliminate those who don't work as much so it's it's a very darwinian process in a way you generate a number of of examples you you keep the ones who survive the longest then you regenerate a number of set of examples and and and so forth back in so um so there has been a number of um universities uh involved in uh to do machine learning in javascript so one would i would say is more on the edges from the university of tokyo maldives they do libraries which are relatively very japanese friendly like sushi too tempura sukiyaki but they have like sushi too is for instance for matrix calculation sukiyaki is for on top of sushi 2 to do different machine learning models they do distributed computing so they have created a framework where you have different browsers and basically you leverage the capabilities from you know maybe using web sockets so you can connect different uh computers different browsers and you leverage the cpus of these different browsers you have two one swiss university which the other two universities are maybe less known or less marketed in a way i had really i learned just by chance that they existed and they have actually quite a few uh libraries uh on npm on github um the last one actually doesn't have a npm but it does have a github i asked them if you wanted to leverage npm but the guy didn't see the point of using uh javascript who is not yes i don't know that's his point but they have a nice online uh laboratory where you can actually download some files and you can you know use the most common machine learning uh libraries on it to you know try to um create a model um coursera i personally so i'm not an expert in the field i uh i'm still doing it actually um i recommend the corsair course from uh from underneath who is a uh who used to work at google um and and baidu he was a chief um machine learning or chief artificial intelligence guy from baidu the chinese company jeffrey hinton saw his course on our networks in corsa jeffrey hinton is one of the leaders in machine learning and it's it's quite unique that you can find his you know course from him completely free on conserva udacity proposes a number of of nanodegrees in the field the example that i showed you on deep traffic the mit course a lot of people from the self-driving engineer uh were involved in that so i don't oh i don't have network so unfortunately on behavior but basically they participated in in this in this setup as well there are lots of resources on the internet uh blogs uh which are not most of the time not javascript specific the javascript ones are the these ones that i mentioned yes so you can get involved also to encourage getting involved there is a ai grant where for whatever projects and tensorfire the latest library that i mentioned they participated in the ai grant competition and they won 5 000 from that to to when they created tensorflow or kaggle kaggle actually was co-founded by an australian guy and it's a it's a website where people can do machine learning competitions and and there are open data sets and there is a forum where you share your tips and tricks to improve the accuracy of your models sorry there was very fast and uh you know very involved but if you have any questions happy sorry for all the links yes sure uh yeah i will put this yes graphical forum machine learning activity effect screen rendering is it going to uh if you've got an app that yeah so turn safari i think you can try the uh the demos that they have on your mobile uh i personally don't know uh you know the performance uh comparisons for that i the one of the thing is i guess the calculations would be i mean i don't i don't think your gpu on the mobile is in terms of power it's it's great that great so i don't think you know um i mean it will improve a bit in a way because it's paralyzed so it's uh you know in parallel to your cpu but in terms of pure raw performance i don't think it's right that being said i know that google with tensorflow has tried to you know i mean democratize the use of its library and so they encourage it and they do android apps on that but i personally haven't tested it so so the whites you have to think it's the same set of weights that you apply to the examples okay so you will try to follow them together very quickly um you can vary yeah so obviously it's different set of inputs but basically it's m and you can see it's a number of features any number of features you multiply each feature but it's white okay so first it's randomly generated and then so so first you randomly generate these weights so then you do the calculations so the like the matrix calculations okay you activate them using one of the functions that i showed you before and um you do that for every neuron every layer and gives you y hat the what comes from your weights okay so then you try to calculate costs you have a cost measure so the most famous one is a mean square arrow of comparing it's basically y y minus y hat square divided by the number of examples okay so once you have that measure of error okay you you back propagate this error into thanks to the derivatives of the active er evaluation functions back to the initial input okay so it changed and and then it's an iterative process thanks to what's called the learning rate where you add that at the end of the day you will me you will diminish the error that you calculated a numerator example or coding example if you want on a like the number of uh i suppose dimensions that you can work on the problem is it just for like a two dimension problem and then anything after that the error rate you know just can't really model it very well uh so actually the fact that you have hidden layers and and you have so it improves uh if you want the non-linearity of your form so let me show you what i mean so you're thinking of giving us a set like that you can see it's it's not clear if you were just doing linear regression it would be a straight line okay so if you were doing just input outputs that's what you would expect it's a it's a linear regression but the fact that you implement a hidden layer in the middle give improves the non-linearity of your uh of your model if you want so uh i i've done that example in in javascript so just logistic regression and just input output without hidden layers would give you a straight line okay but if you had a hidden layer and and i said i mean here the color is in titan but i've done the thing in javascript you the fact that you a hidden layer they will find you know here so you can see here this invariable for hidden layer size three you can see you you add a non-linearity to your problem so and the more you add basically the more precise in your non-linearity you can go okay so then it's a complex then there is i didn't want to go over that but you can overfit your data so then in terms of uh if you want to predict you know a cert with your model you have to be uh very careful that it doesn't offer that your model doesn't overfit your test data but if you do that what's the cost in terms of processing oh obviously that's numerical complexities i mean i i i don't have the exact answer but you know it's it's adding a role it's basically adding uh a few mattresses to your uh to your calculations so in terms of i know computer scientists calculate in terms of or something it would be at least i mean square added square capacity any other question you\n"