Tag: Vanishing gradient descent
What is Vanishing and exploding gradient descent?
Vanishing and exploding gradient descent is a type of optimization algorithm used in deep learning. Vanishing Gradient Vanishing Gradient occurs when the gradient is smaller than expected. It causes the earlier layers to start degrading before the later ones do, causing a decrease in the overall learning rate of that subset of layers. The weights…
Read More