Convergence of Learning Algorithms with Constant Learning Rates
Resource
IEEE Transactions on Neural Networks, n.2 p.484-489
Journal
IEEE Transactions on Neural Networks
Journal Issue
2
Pages
484-489
Date Issued
1991
Author(s)
Hornik, K.
Abstract
The behavior of neural network learning algorithms with a small, constant learning rate, epsilon, in stationary, random input environments is investigated. It is rigorously established that the sequence of weight estimates can be approximated by a certain ordinary differential equation, in the sense of weak convergence of random processes as epsilon tends to zero. As applications, backpropagation in feedforward architectures and some feature extraction algorithms are studied in more detail.
Type
journal article