Personal website Rudolf Adamkovič

Home / Deep learning / Forward propagation


Dimensionality check: Recursive case

Check the forward propagation dimensions for the hidden layer

\begin{equation*} \boxed{A^{[2]}_{p^{[2]} \times n^{[2]}}} \stackrel{?}{=} \boxed{A^{[2]}_{p^{[2]} \times p^{[1]}}} \end{equation*}

with

\begin{equation*} p^{[1]} = n^{[1]} \end{equation*}

by

\begin{align*} \boxed{A^{[2]}_{p^{[2]} \times n^{[2]}}} & = g^{[2]} \big( W^{[2]}_{p^{[2]} \times n^{[2]}} A^{[1]}_{p^{[1]} \times n^{[1]}} + \vec{b}^{[2]}_{p^{[2]} \times 1} \big) && \text{by definition of \(A^{[i]}\)} \\ & = g^{[2]} \big( W^{[2]}_{p^{[2]} \times p^{[1]}} A^{[1]}_{p^{[1]} \times n^{[1]}} + \vec{b}^{[2]}_{p^{[2]} \times 1} \big) && \text{substitute \(n^{[2]}\) with \(p^{[1]}\)} \\ & = g^{[2]} \big( T^{[2]}_{p^{[2]} \times n^{[1]}} + \vec{b}^{[2]}_{p^{[2]} \times 1} \big) && \text{multiply \(W^{[2]} A^{[1]}\)} \\ & = g^{[2]} \big( U^{[2]}_{p^{[2]} \times n^{[1]}} \big) && \text{add \(T^{[2]} + \vec{b}^{[2]}\)} \\ & = g^{[2]} \big( U^{[2]}_{p^{[2]} \times p^{[1]}} \big) && \text{substitute \(n^{[1]}\) with \(p^{[1]}\)} \\ & = \boxed{A^{[2]}_{p^{[2]} \times p^{[1]}}} && \text{apply \(g^{[2]}\).} \end{align*}

© 2025 Rudolf Adamkovič under GNU General Public License version 3.
Made with Emacs and secret alien technologies of yesteryear.