Rudolf Adamkovič Personal site


Dimensionality check: Recursive case

Check the forward propagation dimensions for the hidden layer

\[\begin{equation*}
  \boxed{A^{[2]}_{p^{[2]} \times n^{[2]}}}
  \stackrel{?}{=}
  \boxed{A^{[2]}_{p^{[2]} \times p^{[1]}}}
\end{equation*}
\]

with

\[\begin{equation*}
  p^{[1]} = n^{[1]}
\end{equation*}
\]

by

\[\begin{align*}
  \boxed{A^{[2]}_{p^{[2]} \times n^{[2]}}}
  & =
  g^{[2]}
  \big(
  W^{[2]}_{p^{[2]} \times n^{[2]}}
  A^{[1]}_{p^{[1]} \times n^{[1]}}
  + \vec{b}^{[2]}_{p^{[2]} \times 1}
  \big)
  && \text{by definition of \(A^{[i]}\)}
  \\
  & =
  g^{[2]}
  \big(
  W^{[2]}_{p^{[2]} \times p^{[1]}}
  A^{[1]}_{p^{[1]} \times n^{[1]}}
  + \vec{b}^{[2]}_{p^{[2]} \times 1}
  \big)
  && \text{substitute \(n^{[2]}\) with \(p^{[1]}\)}
  \\
  & =
  g^{[2]}
  \big(
  T^{[2]}_{p^{[2]} \times n^{[1]}}
  + \vec{b}^{[2]}_{p^{[2]} \times 1}
  \big)
  && \text{multiply \(W^{[2]} A^{[1]}\)}
  \\
  & =
  g^{[2]}
  \big(
  U^{[2]}_{p^{[2]} \times n^{[1]}}
  \big)
  && \text{add \(T^{[2]} + \vec{b}^{[2]}\)}
  \\
  & =
  g^{[2]}
  \big(
  U^{[2]}_{p^{[2]} \times p^{[1]}}
  \big)
  && \text{substitute \(n^{[1]}\) with \(p^{[1]}\)}
  \\
  & =
  \boxed{A^{[2]}_{p^{[2]} \times p^{[1]}}}
  && \text{apply \(g^{[2]}\).}
\end{align*}
\]

© 2025 Rudolf Adamkovič under GNU General Public License version 3.
Made with Emacs and secret alien technologies of yesteryear.