A rank-3 tensor with all neuron weights in a neural network:

\begin{equation*} W^{[i]} = \begin{pmatrix} w^{[i](1)}_{1} & \cdots & w^{[i](n)}_{1} \\ \vdots & \ddots & \vdots \\ w^{[i](1)}_{p} & \cdots & w^{[i](n)}_{p} \end{pmatrix} \in \mathbb{R}^{p \times n} \end{equation*}

where

• \(w^{[i]}\) are all weights in the \(i\)-th layer
• \(w^{[i]}_j\) are all weights in the \(j\)-th neuron in the \(i\)-th layer
• \(w^{[i](k)}_j\) is the \(k\)-th weight in the \(j\)-th neuron in the \(i\)-th layer

so

• \(w^{[i](k)}_j\) is the weight of the path from
  • the \(k\)-th neuron in the \((i - 1)\)-th layer to
  • the \(j\)-th neuron in the \(i\)-th layer

and

• \(i\) is the layer number
• \(p\) is the number of neurons in \(i\)-th layer
• \(n\) is the number of neurons in the \((i - 1)\)-th layer ¹

---

• Ranks
• Drawing of rank-3 network weights
• Drawing of rank-2 layer weights
• Drawing of rank-1 neuron weights
• Drawing of rank-0 weight