Assaf Zeevi

General Bounds and Finite-Time Improvement for the Kiefer-Wolfowitz Stochastic Approximation Algorithm

Coauthor(s): Mark Broadie, Deniz Cicek.

Abstract:
We consider the Kiefer-Wolfowitz (KW) stochastic approximation algorithm and derive general upper
bounds on its mean-squared error. The bounds are established using an elementary induction argument
and phrased directly in the terms of tuning sequences of the algorithm. From this we deduce the non-
necessity of one of the main assumptions imposed on the tuning sequences in the Kiefer-Wolfowitz paper
and essentially all subsequent literature. The optimal choice of sequences is derived for various cases of
interest, and an adaptive version of the KW algorithm, scaled-and-shifted KW (or SSKW), is proposed
with the aim of improving its nite-time behavior. The key idea is to dynamically scale and shift the
tuning sequences to better match them with characteristics of the unknown function and noise level, and
thus improve algorithm performance. Numerical results are provided which illustrate that the proposed
algorithm retains the convergence properties of the original KW algorithm while dramatically improving its
performance in some cases.