Tag: vanishing gradient problem

• The Vanishing Gradient Problem

  The vanishing gradient problem can plague deep neural networks which consists of many hidden layers. It generally occurs when the derivatives of the activation functions are less than one. This leads to the multiplication of small numbers during back-propagation, which in turn shrink the gradients exponentially. As a result of this issue, the network will…

  Anu Bhatia

  September 13, 2025

  ai notes

  ai compendium, ai notes, gradient descent, identifying the vanishing gradient problem, machine learning, neural networks, vanishing gradient problem