---

Ranks

The ranks of the weight tensor are:

• rank-3 tensor \(W\)
  all weights in the neural network
• rank-2 matrix \(W^{[i]}\)
  all weights of all neurons in the \(i\)-th layer
• rank-1 row vector²⁵ \(w^{[i]}_{j}\)
  all weights of the \(j\)-th neuron in the \(i\)-th layer
• rank-0 scalar \(w^{[i](k)}_{j}\)
  the \(k\)-th weight of the \(j\)-th neuron in the \(i\)-th layer.