Home / Deep learning / Forward propagation
Dimensionality check: Base case
Check the forward propagation dimensions for
- the input layer \(A^{[0]}\)
- the first hidden layer \(A^{[1]}\)
with
\begin{align*} p^{[0]} & = n^{[0]} \\ p^{[1]} & = n^{[1]} \end{align*}by
\begin{align*} \boxed{A^{[1]}_{p^{[1]} \times n^{[1]}}} & = g^{[1]} \big( W^{[1]}_{p^{[1]} \times n^{[1]}} A^{[0]}_{p^{[0]} \times n^{[0]}} + \vec{b}^{[1]}_{p^{[1]} \times 1} \big) && \text{by definition of \(A^{[i]}\)} \\ & = g^{[1]} \big( W^{[1]}_{p^{[1]} \times p^{[0]}} A^{[0]}_{p^{[0]} \times n^{[0]}} + \vec{b}^{[1]}_{p^{[1]} \times 1} \big) && \text{substitute \(n^{[1]}\) with \(p^{[0]}\)} \\ & = g^{[1]} \big( T^{[1]}_{p^{[1]} \times n^{[0]}} + \vec{b}^{[1]}_{p^{[1]} \times 1} \big) && \text{multiply \(W^{[1]} A^{[0]}\)} \\ & = g^{[1]} \big( U^{[1]}_{p^{[1]} \times n^{[0]}} \big) && \text{add \(T^{[1]} + \vec{b}^{[1]}\)} \\ & = g^{[1]} \big( U^{[1]}_{p^{[1]} \times p^{[0]}} \big) && \text{substitute \(n^{[0]}\) with \(p^{[0]}\)} \\ & = \boxed{A^{[1]}_{p^{[1]} \times p^{[0]}}} && \text{apply \(g^{[1]}\).} \end{align*}