About this question

Vanishing Gradient in Deep Networks

Medium · Linear Algebra & Machine Learning · Quant Trader interview question · vanishing-gradient, deep-learning, neural-networks, sigmoid, backpropagation

You are training a very deep neural network (100+ layers) for a complex financial time-series forecasting task. You observe that the early layers of your network learn very slowly compared to the later layers. The activation function used in each layer is a sigmoid function. Explain why this phenomenon, known as the vanishing gradient problem, occurs in the early layers.