157251-Thumbnail Image.png
Description
This thesis presents a family of adaptive curvature methods for gradient-based stochastic optimization. In particular, a general algorithmic framework is introduced along with a practical implementation that yields an efficient, adaptive curvature gradient descent algorithm. To this end, a theoretical

This thesis presents a family of adaptive curvature methods for gradient-based stochastic optimization. In particular, a general algorithmic framework is introduced along with a practical implementation that yields an efficient, adaptive curvature gradient descent algorithm. To this end, a theoretical and practical link between curvature matrix estimation and shrinkage methods for covariance matrices is established. The use of shrinkage improves estimation accuracy of the curvature matrix when data samples are scarce. This thesis also introduce several insights that result in data- and computation-efficient update equations. Empirical results suggest that the proposed method compares favorably with existing second-order techniques based on the Fisher or Gauss-Newton and with adaptive stochastic gradient descent methods on both supervised and reinforcement learning tasks.


Download restricted.

Details

Title
  • Adaptive Curvature for Stochastic Optimization
Contributors
Date Created
2019
Resource Type
  • Text
  • Collections this item is in
    Note
    • thesis
      Partial requirement for: M.S., Arizona State University, 2019
    • bibliography
      Includes bibliographical references (pages 35-39)
    • Field of study: Computer science

    Citation and reuse

    Statement of Responsibility

    by Trevor Barron

    Machine-readable links