On the Determination of the Step Size in Stochastic Quasigradient Methods
Abstract
For algorithms of the Robbins-Monro type, the best choice (from the asymptotic point of view) for the step-size constants a_n is known to be a/n. From the practical point of view, however, adaptive step-size rules seem more likely to produce quick convergence. In this paper a new adaptive rule for controlling the stepsize is presented and its behavior is studied.