A rank-2 tensor with all neuron biases in a neural network:

\begin{equation*} \vec{b}^{[i]} = \begin{pmatrix} b^{[i]}_1 \\ \vdots \\ b^{[i]}_p \end{pmatrix} \in \mathbb{R}^p \end{equation*}

where

• \(b^{[i]}\) are all biases of the \(i\)-th layer
• \(b^{[i]}_j\) is the bias of the \(j\)-th neuron in the \(i\)-th layer

and

• \(i\) is the layer number
• \(p\) is the number of neurons in the \(i\)-th layer.