Tag: xavier initialization

• Xavier Initialization

  Training deep networks requires careful initialization. If weights are too small, activations and gradients can shrink toward zero; if too large, they can grow out of control. Also known as the Glorot Initialization, the Xavier Initialization was proposed by Xavier Glorot and Yoshua Bengio in 2010. It is designed to combat both, vanishing and exploding…

  Anu Bhatia

  October 6, 2025

  ai notes

  ai compendium, ai notes, glorot initialization, machine learning, neural networks, weight initialization, xavier initialization