• KELLEY

  • Referenced in 600 articles [sw04829]
  • cases the noise often introduces artificial minimizers. Gradient information, even if available, cannot expected ... methods use finite difference approximations of the gradient, which are adjusted to the noise level...
  • SNOPT

  • Referenced in 507 articles [sw02300]
  • derivatives are available and that the constraint gradients are sparse. We discuss an SQP algorithm...
  • LSQR

  • Referenced in 358 articles [sw00530]
  • equivalent to the standard method of conjugate gradients, but possesses more favorable numerical properties. Reliable ... described comparing LSQR with several other conjugate-gradient algorithms, indicating that LSQR is the most...
  • MINOS

  • Referenced in 441 articles [sw05095]
  • automatic estimation of some or all gradients. Upper and lower bounds on the variables...
  • Adam

  • Referenced in 273 articles [sw22205]
  • introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based ... invariant to diagonal rescaling of the gradients, and is well suited for problems that ... problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically...
  • INTLAB

  • Referenced in 419 articles [sw04004]
  • automatic differentiation (forward mode, vectorized computations, fast) Gradients (to solve systems of nonlinear equations) Hessians...
  • CGS

  • Referenced in 301 articles [sw08225]
  • algorithm (a “squared” conjugate gradient method) with a preconditioning called ILLU (an incomplete line...
  • darch

  • Referenced in 255 articles [sw11086]
  • known training algorithms like backpropagation or conjugate gradient...
  • SHASTA

  • Referenced in 224 articles [sw18281]
  • positivity of actual mass densities so steep gradients and inviscid shocks are handled particularly well...
  • tn

  • Referenced in 130 articles [sw05140]
  • discusses the use of the linear conjugate-gradient method (developed via the Lanczos method ... equivalent Lanczos characterization of the linear conjugate-gradient method may be exploited to define ... direction defined by a nonlinear conjugate-gradient-type method and a modified Newton direction. Numerical...
  • lobpcg.m

  • Referenced in 120 articles [sw24008]
  • locally optimal block preconditioned conjugate gradient method: Toward the optimal preconditioned eigensolver: Locally optimal block ... preconditioned conjugate gradient method. We describe new algorithms of the locally optimal block preconditioned conjugate ... gradient (LOBPCG) method for symmetric eigenvalue problems, based on a local optimization of a three ... algorithm, we advocate the standard preconditioned conjugate gradient method for finding an eigenvector...
  • SCALCG

  • Referenced in 100 articles [sw08453]
  • SCALCG – Scaled conjugate gradient algorithms for unconstrained optimization. In this work we present and analyze ... scaled conjugate gradient algorithm and its implementation, based on an interpretation of the secant equation ... line search conditions. The best spectral conjugate gradient algorithm SCG by Birgin and Martínez ... Beale–Powell. The parameter scaling the gradient is selected as spectral gradient...
  • NASTRAN

  • Referenced in 177 articles [sw04245]
  • behavior, dynamic and vibration response and thermal gradients in real-world systems, MSC Nastran...
  • CONOPT

  • Referenced in 162 articles [sw02791]
  • CONOPT is a generalized reduced-gradient (GRG) algorithm for solving large-scale nonlinear programs involving...
  • SPGL1

  • Referenced in 159 articles [sw08365]
  • complex domain. At each iteration, a spectral gradient-projection method approximately minimizes a least-squares...
  • AdaGrad

  • Referenced in 97 articles [sw22202]
  • ADAGRAD: adaptive gradient algorithm; Adaptive subgradient methods for online learning and stochastic optimization. We present ... earlier iterations to perform more informative gradient-based learning. Metaphorically, the adaptation allows ... which employ proximal functions to control the gradient steps of the algorithm. We describe...
  • TRON

  • Referenced in 109 articles [sw04714]
  • bound-constrained optimization problems. TRON uses a gradient projection method to generate a Cauchy step ... preconditioned conjugate gradient method with an incomplete Cholesky factorization to generate a direction...
  • Pegasos

  • Referenced in 94 articles [sw08752]
  • Pegasos: primal estimated sub-gradient solver for SVM. We describe and analyze a simple ... effective stochastic sub-gradient descent algorithm for solving the optimization problem cast by Support Vector ... example. In contrast, previous analyses of stochastic gradient descent methods for SVMs require...
  • CG_DESCENT

  • Referenced in 105 articles [sw04813]
  • Algorithm 851: CG_DESCENT. A conjugate gradient method with guaranteed descent Recently, a new nonlinear ... conjugate gradient scheme was developed which satisfies the descent condition gTkdk...
  • GradSamp

  • Referenced in 103 articles [sw05270]
  • robust gradient sampling algorithm for nonsmooth, nonconvex optimization The authors describe a practical and robust ... only request formulated is that the gradient of the function is easily computed where...