---

Weight tensor

A rank-3 tensor with all neuron weights in a neural network:

\[\begin{equation*}
  W^{[i]} =
  \begin{pmatrix}
    w^{[i](1)}_{1} & \cdots & w^{[i](n)}_{1} \\
    \vdots & \ddots & \vdots \\
    w^{[i](1)}_{p} & \cdots & w^{[i](n)}_{p}
  \end{pmatrix}
  \in \mathbb{R}^{p \times n}
\end{equation*}
\]

where

• \(w^{[i]}\) are all weights in the \(i\)-th layer
• \(w^{[i]}_j\) are all weights in the \(j\)-th neuron in the \(i\)-th layer
• \(w^{[i](k)}_j\) is the \(k\)-th weight in the \(j\)-th neuron in the \(i\)-th layer

so

• \(w^{[i](k)}_j\) is the weight of the path from
  • the \(k\)-th neuron in the \((i - 1)\)-th layer to
  • the \(j\)-th neuron in the \(i\)-th layer

and

• \(i\) is the layer number
• \(p\) is the number of neurons in \(i\)-th layer
• \(n\) is the number of neurons in the \((i - 1)\)-th layer ²⁴

• Ranks
• Drawing of rank-3 network weights
• Drawing of rank-2 layer weights
• Drawing of rank-1 neuron weights
• Drawing of rank-0 weight