What is a Neural Network (C1W1L02)

# Understanding Neural Networks and Deep Learning: A Step-by-Step Guide

## Introduction to Deep Learning

Deep learning is a subset of machine learning that involves training neural networks, sometimes very large ones. The concept can be complex, but we'll break it down using a practical example.

## Housing Price Prediction with Linear Regression

Let's start with a simple housing price prediction example. Suppose you have a dataset of six houses, each with the size (in square feet or meters) and the corresponding price. Your goal is to fit a function that predicts house prices based on their size.

If you're familiar with linear regression, you might begin by fitting a straight line to the data. This approach works well for simple relationships but has limitations. For instance, house prices cannot be negative, so if your straight-line model predicts negative values for small-sized houses, it's flawed.

To address this, we can modify our approach. Instead of a straight line that might dip below zero, we can create a curve that bends to ensure the price remains non-negative. This results in a function that starts at zero and then follows a straight line, much like the blue curve shown earlier.

## Neural Networks: The Simplest Form

This modified function can be seen as the simplest form of a neural network—a single neuron. Here's how it works:

- **Input (X):** The size of the house.

- **Neuron:** This node takes the input, computes a linear function, applies the rectified linear activation (ReLU), and outputs the predicted price.

The ReLU function is crucial here. It ensures that the output is always non-negative by taking the maximum of zero and the linear function's result. This activation function is widely used in neural networks because it helps introduce non-linearity into the model, allowing it to capture more complex patterns in the data.

## Building a Larger Neural Network

A larger neural network is constructed by stacking many such neurons together. Imagine each neuron as a Lego brick; you build a bigger structure by combining multiple bricks.

For example, instead of predicting house prices based solely on size, we can include additional features like:

- **Number of bedrooms:** Affects the price significantly.

- **Zip code (postal code):** Indicates walkability and school quality in certain areas.

Each feature becomes an input to the network. The magic of neural networks lies in their ability to learn complex relationships between inputs and outputs by processing these features through multiple layers.

## Hidden Units and Layer Connectivity

In a neural network, each circle (or node) in the middle layer is called a hidden unit. These units take all four input features (size, number of bedrooms, zip code, etc.) as their input. The remarkable thing about neural networks is that you don't need to specify how each hidden unit should process the inputs; the network learns this during training.

The layers in a neural network are densely connected, meaning every input feature is connected to every hidden unit in the next layer. This dense connectivity allows the network to learn intricate patterns and relationships within the data.

## The Power of Neural Networks

Given enough data (pairs of X inputs and Y outputs), neural networks can approximate complex functions with remarkable accuracy. They are particularly powerful in supervised learning, where the goal is to map an input (X) to an output (Y). The housing price prediction example is a classic case of supervised learning.

## Conclusion: Applications Beyond Housing Prices

As you build your own neural networks, you'll find them incredibly useful for various applications beyond housing price prediction. In the next video, we'll explore more examples of supervised learning and see how neural networks can be applied to solve real-world problems effectively.

By understanding the basics—starting from simple linear regression and moving to complex neural networks—you're taking a crucial step toward mastering deep learning and its many applications.

"WEBVTTKind: captionsLanguage: enthe term deep learning refers to training neural network sometimes very large neural networks so what exactly is in your network in this video let's try to give you some of the basic intuitions let's start with a housing price prediction example let's say you have a data set with six houses so you know the size of holders in square feet or square meters and you know the price of the house and you want to fit a function to predict the price of house as a function of the size so if you're familiar with linear regression you might say well let's fit a straight line to the data so you get a straight line like that but to be the fancier you might say well we know that prices can never be negative right so instead of a straight line fit will eventually become negative let's bend the curve here so just ends up zero here so this thick blue line ends up being you know your function for predicting the price of a house as a function of the size with zero here and then is a straight line fit so there right so you can think of this function that you've just fit to housing prices as a very simple neural network there's almost the simplest possible neural network let me draw here we have as the input to the new network the size of the house which one we call X it goes into this node this little circle and then it outputs the price when we call y so this little circle which is a single neuron in on your network implements this function that we drew on the left and all that the neuron does is it inputs the size computers linear function takes a max of 0 and then outputs the estimated price and by the way in the neural network literature you see this function a lot this function which goes as 0 at some time and it takes off as a straight line this function is called a rare loop function which stands for rectified Linnea so RDL you then rectify just means taking a max of 0 which is why you get a function shape like this you don't need to worry about regular units for now but it's just something you see a game later in this course so if this is a single neuron neural network really a tiny little neural network a larger neural network is then formed by taking many of these single neurons and stacking them together so if you think of this neuron as being you know like a single Lego break you then get a bigger neural network by stacking together many of these Lego bricks let's see an example let's say that instead of predicting the price of the house just from the size you now have other features you know other things about the house such as the number of bedrooms which only write as count bedrooms and you might think that you know one of the things that really affects the price of a house is family size so you know can just house such a family of three or family four family of five and it's really based on the size and square feet of square meters and the number of bedrooms that determines whether or not a house can fit your family's family size and then maybe you know the zip code in different countries is called the postal code right at the house um and the zip code maybe is a feature that tells you you know walkability so is this neighborhood highly walkable you know being just watch the grocery store walk to school or do you need to drive so some people prefer highly walkable neighborhoods and then the zip code as well as you know the wealth may be um tells you right certainly in the United States but some other countries as well tells you how good is the school quality so each of these little circles I'm drawing can be one of those rare you rectify learning unions or some other slightly nonlinear function so that based on the size and number of bedrooms you can family size the zip code as bloaty based on zip code networks can estimate the school quality and finally you might think that well the way people decide how much they're willing to pay for a house is they look at the things that really mattered to them in this case family size walkability and school quality and that helps you predict the price so in this example X is all of these four inputs and Y is the price to try to predict and so by stacking together a few of the single neurons or the simple predictors we had from the previous slide we now have a slightly larger in your network how the magic of a neural network is that when you implement it you need to give it just the input X and the output Y for a number of examples in your training set and all these things in the middle they will figure out by itself so what you actually implement is this where here you have a neural network with four inputs so the upper features might be the size number of bedrooms the zip code or postal code and well for the neighborhood and so given these um input features the job of the neural network will be to predict the price Y and notice also that each of these circles these we call hidden units in a neural network that each of them takes as input all four input features so for example rather than saying this first node represents family size and family size depends only on the features x1 and x2 right instead we're going to say well neural network you decide whatever you want just know to be and we'll give you all for it for features to compute whatever you want so we say that layers that is this input layer and this layer in the middle of the neural nets are densely connected because every input feature is connected to every one of these circles in the middle and the remarkable thing about neural networks is that given enough data about X&Y given enough training examples with both X&Y neuro-networks a remarkably good at freaking out functions that accurately map from x to y so that's a basic neural network it turns out that as you build out your own neural networks you probably find them to be most useful most powerful in supervised learning settings meaning that you're trying to take an input X and map it to some output Y like we just saw in the housing price prediction example in the next video let's go over some more examples of supervised learning and some examples of where you might find your networks to be incredibly helpful for your applications as wellthe term deep learning refers to training neural network sometimes very large neural networks so what exactly is in your network in this video let's try to give you some of the basic intuitions let's start with a housing price prediction example let's say you have a data set with six houses so you know the size of holders in square feet or square meters and you know the price of the house and you want to fit a function to predict the price of house as a function of the size so if you're familiar with linear regression you might say well let's fit a straight line to the data so you get a straight line like that but to be the fancier you might say well we know that prices can never be negative right so instead of a straight line fit will eventually become negative let's bend the curve here so just ends up zero here so this thick blue line ends up being you know your function for predicting the price of a house as a function of the size with zero here and then is a straight line fit so there right so you can think of this function that you've just fit to housing prices as a very simple neural network there's almost the simplest possible neural network let me draw here we have as the input to the new network the size of the house which one we call X it goes into this node this little circle and then it outputs the price when we call y so this little circle which is a single neuron in on your network implements this function that we drew on the left and all that the neuron does is it inputs the size computers linear function takes a max of 0 and then outputs the estimated price and by the way in the neural network literature you see this function a lot this function which goes as 0 at some time and it takes off as a straight line this function is called a rare loop function which stands for rectified Linnea so RDL you then rectify just means taking a max of 0 which is why you get a function shape like this you don't need to worry about regular units for now but it's just something you see a game later in this course so if this is a single neuron neural network really a tiny little neural network a larger neural network is then formed by taking many of these single neurons and stacking them together so if you think of this neuron as being you know like a single Lego break you then get a bigger neural network by stacking together many of these Lego bricks let's see an example let's say that instead of predicting the price of the house just from the size you now have other features you know other things about the house such as the number of bedrooms which only write as count bedrooms and you might think that you know one of the things that really affects the price of a house is family size so you know can just house such a family of three or family four family of five and it's really based on the size and square feet of square meters and the number of bedrooms that determines whether or not a house can fit your family's family size and then maybe you know the zip code in different countries is called the postal code right at the house um and the zip code maybe is a feature that tells you you know walkability so is this neighborhood highly walkable you know being just watch the grocery store walk to school or do you need to drive so some people prefer highly walkable neighborhoods and then the zip code as well as you know the wealth may be um tells you right certainly in the United States but some other countries as well tells you how good is the school quality so each of these little circles I'm drawing can be one of those rare you rectify learning unions or some other slightly nonlinear function so that based on the size and number of bedrooms you can family size the zip code as bloaty based on zip code networks can estimate the school quality and finally you might think that well the way people decide how much they're willing to pay for a house is they look at the things that really mattered to them in this case family size walkability and school quality and that helps you predict the price so in this example X is all of these four inputs and Y is the price to try to predict and so by stacking together a few of the single neurons or the simple predictors we had from the previous slide we now have a slightly larger in your network how the magic of a neural network is that when you implement it you need to give it just the input X and the output Y for a number of examples in your training set and all these things in the middle they will figure out by itself so what you actually implement is this where here you have a neural network with four inputs so the upper features might be the size number of bedrooms the zip code or postal code and well for the neighborhood and so given these um input features the job of the neural network will be to predict the price Y and notice also that each of these circles these we call hidden units in a neural network that each of them takes as input all four input features so for example rather than saying this first node represents family size and family size depends only on the features x1 and x2 right instead we're going to say well neural network you decide whatever you want just know to be and we'll give you all for it for features to compute whatever you want so we say that layers that is this input layer and this layer in the middle of the neural nets are densely connected because every input feature is connected to every one of these circles in the middle and the remarkable thing about neural networks is that given enough data about X&Y given enough training examples with both X&Y neuro-networks a remarkably good at freaking out functions that accurately map from x to y so that's a basic neural network it turns out that as you build out your own neural networks you probably find them to be most useful most powerful in supervised learning settings meaning that you're trying to take an input X and map it to some output Y like we just saw in the housing price prediction example in the next video let's go over some more examples of supervised learning and some examples of where you might find your networks to be incredibly helpful for your applications as well\n"