### [[Artificial Neural Networks (ANN) ]] ### [[Neural Network Layers]] ### Feed Forward Deep feedforward neural networks = feedforward neural networks or multi-layer perceptrons (MLPs) > Goal of a feedforward neural network is to approximate some function f* - for example, a classifier, y = f*(x), maps an input x to a category y. A feed forward network defines a mapping of y = f(x,theta) and learns the value of theta that result in the best function representation - Feedforward because the information through the function being evaluated from x, through the intermediary computations used to define f, and finally to the output y - There are no feedback connections where the output of the model are fed back into itself ![[Pasted image 20210914105822.png]] > Recurrent Neural Networks are when feedforward neural networks are extended to include feedback connections. > Applying neural network we can overcome the problem of non-linearity Neural networks emerged from a very popular ML algorithm called perceptron. We want the network to learn the weights and biases so that the output correctly classifies the digit. ### Weights Desirably, a small change in weights should correspond to a small change in output. - Weights are applied to inputs and passed into an activation function along with bias - Weights reflect how important an input is ### Bias - Bias is simply a constant value (or a constant vector) that is added to the product of inputs and weights. Bias is utilised to offset the result. - The bias is used to **shift the result of activation function **towards the positive or negative side. - The addition of bias **reduces the variance** and hence **introduces flexibility and better generalisation to the neural network**. - Bias is essentially the negative of the threshold, therefore the value of bias controls **when to activate the activation function.** ### [[Activation Function]] Activation function is a mathematical function that can normalise the inputs and convert it into an output ![[Pasted image 20210914105653.png]] ![[Pasted image 20210914105155.png]] ![[Pasted image 20210914105716.png]] #neuralnetworks