How to Get Started with Deep Learning: A Guide to Building Your First Deep Neural Network

If you're interested in getting started with deep learning, you're probably wondering how to begin. With so many different learning paths and tools available, it can be difficult to know where to start. In this article, we'll explore the benefits of using a deep learning library called Karos to build your first deep neural networks and compare it to other options.

Deep Learning: A Brief History

Deep learning started gaining popularity around a couple of years ago when Hinton's team submitted a model that blew away the competition for the large-scale visual recognition challenge. Their deep neural network was significantly better than all benchmarks, thanks in part to lots of GPU computation and data. This caught the attention of others, who began to implement their own deep neural networks for different tasks, resulting in a deep learning renaissance.

The Rise of Deep Learning

Deep learning played a huge part in the biggest AI success story of 2017 - AlphaGo. Google's algorithm that mastered the game of Go was previously thought to be impossible. Similar improvements were made in fields like vision, text, and speech recognition. For example, Wavenet was a model that massively sped up improvements to speech-to-text and text-to-speech.

New Competitors Emerge

In 2017, new competitors emerged for deep learning libraries. But Karos is still the easiest way to get started. Pi Torch is getting really popular and is a great alternative to Karos.

Building Your First Deep Neural Network with Karos

To build your first deep neural network with Karos, you'll need to define a sequence of layers using the sequential class. Once you create an instance of the sequential class, you can add new layers where each new line is a new layer. The number of inputs to expect will depend on the type of network.

The Sequential Model

Think of a sequential model as a pipeline with your raw data fed in at the bottom and predictions that come out at the top. This is helpful because it allows you to clearly show the role of each layer in transforming the input data into predictions. Activation functions can also be extracted and added as separate layers, which is important for the output layer.

Compiling Your Network

Once you've defined your model, you'll need to compile it. Compiling transforms a simple sequence of layers into a highly efficient series of matrix transforms that are intended to be executed on a GPU or CPU, depending on your configuration. Compilation requires a number of parameters to be specified, including the optimization algorithm and loss function used to train the network.

Fitting Your Network

Once your network is compiled, you'll need to fit it using the back propagation algorithm and optimized according to the optimization algorithm and loss function specified when compiling the model. This involves adapting the weights on a training dataset. Fitting your network requires a training dataset to be specified both a matrix of input patterns X and an array of matching output patterns.

Making Predictions

Once you're satisfied with the performance of your fit model, you can use it to make predictions on new data. This is as easy as calling the predict function on the model with an array of new input patterns. For example, we'll see that our text generation sample generates text in the style of our favorite author.

Conclusion

Building a deep neural network with Karos is a great way to get started with deep learning. Pi Torch is also a popular alternative. But regardless of which library you choose, building a deep network involves defining a sequence of layers, compiling your model, fitting it using the back propagation algorithm, and making predictions on new data. With practice and patience, you can build a powerful deep neural network that can solve complex problems in image recognition, natural language processing, and more.

Top-Notch Work: Coding Challenges

This week's coding challenge is to use chaos to build your own deep neural network. The winner of the coding challenge will be announced next week. Please subscribe for more programming videos and thank you for watching.

"WEBVTTKind: captionsLanguage: enHello worldIt's Siraj and the question I get asked the most by far isHow do I get started with deep learning?and it makes sense to ask that there are so many different learning paths andTools, you could use. It's hard to just pick one and roll with it in this videoI'm gonna explain why you should use a deep learning library called Karos to build your first deep neural networks andCompare it to other optionsThen we'll use Karos to build an app that generates text in the style of any given author deep learning only started getting really popularA couple years ago when Hinton's team submitted a model that blew away the competition for the large scale visual recognitionChallenge their deep neural network was significantly better than all benchmarksIlluminati confirmedBecause it used lots of GPU computation and dataOthers began to take notice and implemented their own deep neural networks for different tasks resulting in a deep learningRenaissance deep learning played a huge part in the biggest a I success story of 2017 alphagoGoogle's algorithm that mastered the game of Go previously thought nearImpossible similar improvements were made in fields like vision text and speech recognitionWavenet for example was a model that massively sped up improvements to speech to text and text-to-speechResulting in lifelike generated audio Giotto was really the first widely adopted deep learningLibrary it was maintained by the University of MontrealBut in September of last year they announced that they would stop developing for Theano in 2018 yesdifferent open-source Python deep learning frameworks have been introduced the past couple of years andSome got lots of traction as of nowtensorflow seems to be the most used deep learning library based on the number of github stars andForks as well as Stack Overflow activityBut there are other libraries that are growing passionate user bases as well pi torch is a great exampleIt was introduced in January2017 by Facebook they basically ported the popular torch framework which was written in Lua to Python the main driver behindPipe torches popularity was the fact that it used dynamicComputation graphs that means they are defined by run instead of the traditionalDefine and run when inputs can vary like if we're using unstructured data with textThis is super useful and efficient when it comes to static graphsWe first draw the graph then inject the data to run it. That's defined aedra for dynamic graphsThe graph is defined on the fly via the forward computation of the dataThat's defined by runBut in addition to tensor flows main frameworkseveral companions libraries were released including the tensorflow fold for dynamic computation graphs and tensorflowtransform for data input pipelines the temperature flow team also announced a neweager execution mode which works similar to pi torches dynamic computation graphsBut wait other tech giants have also been getting in on the gameMicrosoft launched its cognitive toolkit last year Facebook launched cafe toAmazon launched MX net deepmind released sonnet there's also deep learning forJ-d libe h2o AI and spark oh and Facebook and Microsoft announced the Onix openformat to share deep learning models acrossFrameworks for example you can train your model in one framework, but then serve it in production in another oneI know I know I know deep AF overload but look the best way to learn how some AIConcept works is to start building it and figure it out as you goAnd the best way to do that is by first using a high-level library called chaosChaos is effectively an interface that wraps multiple frameworksYou can use it as an interface tensorflow Theano or CNT k it works the same no matter what back-end you useFrancois chalette a deep learning researcher at google created it and maintains it last year Google announcedIt was chosen as the official high level API of tensorflowwhen it comes to writing and debugging custom modules and layersPi torch is the faster option?While Karros is definitely the fastest track when you need to quickly train and test a model built from standard layersUsing chaos the pipeline for building a deep network looks like this you define it compile it fit itEvaluate it and then use it to make predictions consider a simple three layer neural network with an input layer hidden layer and output layereach of these layers is just a matrix operation input times await a Tobias andActivate the results repeat that twice and get a predictionDeep networks have multiple layers. They can have three four five whateverThat's why they're called deep and these layers don't have to use just one type of operationThere are all sorts of layers out there for different types of networksConvolutional layers drop out layers or current layers the list goes onBut the basic idea of a deep neural network is applying a series of mathoperations in order to some input data each layerRepresents a different operation that then passes the result on to the next layer so in a wayWe can think of these layers as building blocksIf we can list out all the different types of layers we can wrap them into their own classesAnd then reuse them as modular building blocksThat's exactly what Kerris does it also abstract away a lot of the magic numbersYou'd have to input into a deep Network written in say pure tensorflow when we define a networkThey're defined as a sequence of layers using the sequential classOnce we create an instance of the sequential class we can add new layers where each new line is a new layerWe could do this in just two steps or we could do this in one step by creating an array of layersBeforehand and pasting it to the constructor of the sequential model the first layer in the network must define the number ofPuts to expect the way that this is specified can defer depending on the network typethink of a sequential model as a pipeline with your raw data fed in at the bottom andPredictions that come out at the topThis is helpful in Cara's as concept that were traditionally associated with the layerCan also be split out and added as separate layers clearly showing the role in the transform of data from input to prediction for exampleactivation functions thatTransform a some signal from each neuron in a layer can be extracted and added to the sequential class as a layer likeObject called activation the choice of activation function is most important for the output layer as it will define the format thatPredictions will take once we defined our networkWe'll compile it that means it transforms a simple sequence of layers into a highly efficient series of matrix transformsIntended to be executed on a GPU or CPU depending on our configuration setting it's a pre compute step for the networkIt's required after defining a modelcompilation requires a number of parameters to be specified specifically tailored to training our network theOptimization algorithm we use to Train the network and the loss function used to evaluate it are things that we can decideThis is the art of deep learning once the network is compiled it can be fitWhich means adapting the weights on a training data set fitting the network requires a training data to be specified both a matrix ofinput patterns X and an array of matching output patterns why the network is trained using the back propagation algorithm andOptimized according to the optimization algorithm and loss function specified when compiling the model finally once we are satisfiedWith the performance of our fit model we can use it to make predictions on new dataThis is as easy as calling the predict function on the model with an array of new input patternsFor our text generation sample well see that it generates text in the style of our favorite authorJust as we fed it in three points to remember there are lots of new competitors that showed up in2017 for deep learning librariesBut Karos is still the easiest way to get started pi torch is getting really popular and isBest way to build models next to Karros and deep networks are a series of math operations in the form of layers justMix and match them to get different results every time the coding challenge winner from the war robots video is Alberto GarcesHe used a proximal policy optimization algorithm to train an AI to balance a pendulum using the open AI gym environmentTop-notch work, Alberto and the runner up is Sven near der Berger who landed a simulated space X rocketUsing PP o such a cool use case this week's coding challenge is to use chaos to build your own deep neural networkGithub links go in the description and coding challenge winners will be announced next weekPlease subscribe for more programming videos and for now I've gotta not use anything made by Microsoft, so thanks for watching\n"