seli.net

Building neural networks from Modules. Contains many pre-built modules for common neural network layers.

This folder contains the functionality for creating and manipulating networks.

Classes

Add(*modules)	Add the output of a sequence of modules to the input.
Affine(dim)	Apply a learnable linear transformation followed by a learnable bias.
Bias()	Add a learnable bias to the last axis of the input.
Constant(value)	Return a constant value.
CrossAttention(dim, heads_q[, heads_k, ...])	Perform cross-attention between two sequences.
DotProductAttention(dim, heads_q[, heads_k, ...])	Apply dot-product attention to a sequence.
Einsum(pattern)	Wrapper around einops.einsum.
Init()	Base class for all initializers.
InitNormal([init, shift, scale])	Initializes values from a normal distribution.
InitOnes()	Initializes all values to one.
InitOrthogonal([scale])	Initialize weights as an orthogonal matrices.
InitTruncatedNormal([minv, maxv, shift, scale])	Initializes values from a truncated normal distribution.
InitUniform([init, shift, scale])	Initializes values from a uniform distribution.
InitZeros()	Initializes all values to zero.
LayerNorm([eps, offset])	Normalize the input along the last axis.
Linear(dim)	Apply a learnable linear transformation to last axis of the input.
Multiply(*modules)	Multiply the output of a sequence of modules by the input.
Param(*, init[, rngs, value, collection])	Organizes a parameter
RMSNorm([eps, offset])	Scale the input by the reciprocal of the root mean square along the last axis.
Rearrange(pattern, **dims)	Rearrange the input tensor according to the given pattern.
Reduce(pattern, reduction, **dims)	Reduce the input tensor according to the given pattern and the reduction type.
Repeat(pattern, **dims)	Repeat the input tensor according to the given pattern.
Scale([offset])	Scale the last axis of the input by a learnable vector.
Sequential(*modules)	Call a sequence of modules in order.