LayerNorm

class seli.net.LayerNorm(eps: float = 1e-06, offset: float | int = 1)

Bases: Module

Normalize the input along the last axis. Then add a learnable offset and scale by a learnable weight along the last axis.

Parameters:

• eps (float) – Epsilon value for numerical stability.

• offset (bool) – Whether to add 1 to the scale weight before multiplying. If true, the model is initialized to the identity function. If false, the model is initialized to the constant zero function.

Methods Summary

__call__(x)

Call self as a function.

Methods Documentation

__call__(x: Float[Array, '*batch dim']) → Float[Array, '*batch dim']

Call self as a function.