tags: machine learning, deep learning
Stochastic Gradient Descent
Instead of using every training example to improve the models parameters, we use only one feature and try to fit it
repeat { for i to m {
} }
In short we select a training example and use it for all and then select another and so on.
Applying to neural networks
When applied to neural networks, we divide the dataset to a mini-batch of training sets. We can conclude that the sum of all averages of the error function of the mini-batches is nearly equal to the average of the error function of the complete dataset
TODO! equations
Exhausting these mini-batches during training is called and epoch of training.
Normal Equation
Used only in linear regression to get into the global minima in one step Derivation of the normal equation https://eli.thegreenplace.net/2015/the-normal-equation-and-matrix-calculus/
Some Derivaitions
Trace of A, Trace properties are in the lecture