Adagrad

Adagrad#

class seli.opt.Adagrad(lr: float = 0.01, eps: float = 1e-08)[source]#

Bases: Optimizer

Adaptive Gradient optimizer.

Adapts learning rates per-parameter by scaling with the inverse square root of accumulated squared gradients.

Methods Summary

call_param(key, grad, **_)

Process the gradients of a single parameter.

Methods Documentation

call_param(key: str, grad: Float[Array, '*s'], **_) → Float[Array, '*s'][source]#

Process the gradients of a single parameter. This function is useful for implementing custom optimizers that essentially run the same function for all parameters. This is the case for most well known optimizers.

Parameters:

  • loss (Float[Array, ""]) – The absolute loss value.

  • key (str) – The key of the parameter.

  • grad (Float[Array]) – The gradients of the parameter.

  • param (Float[Array]) – The parameter values.

Returns:

grad – The processed gradients of the parameter.

Return type:

Float[Array]

Contents