规定
- \(y_{ij}\)为第\(i\)层网络第\(j\)个神经元的输出.
- \(t_i\)为输出层第\(i\)个输出.
- \(n_i\)为第\(i\)层网络的神经元数量.
- 激活函数\(\sigma(x)=Sigmod(x)=\frac{1}{1+e^{-x}}\)，因此\(\frac{\partial \sigma(x)}{\partial x}=\sigma(x)[1-\sigma(x)]\).
- \(E\)代表误差，即\(E=\sum_{i=1}^{2}(y_{3i}-t_i)^{2}\).
- \(\nabla_{ijk}\)为第\(i\)层网络第\(j\)个神经元，它对上一层网络的第\(k\)个神经元的\(\omega\)的梯度值.
公式推导
现在以\(11\rightarrow 21 \rightarrow 31\)该过程为例，同时导出通项公式。
1.\(net_{ij}\)
\(net_{21}=\sum_{i=1}^{2}(\omega_{21i}y_{1i})\).
\(net_{31}=\sum_{i=1}^{3}(\omega_{31i}y_{2i})\).
因此通项公式为：\(net_{ij}=\sum_{k=1}^{n_{i-1}}(\omega_{ijk}y_{i-1,k})\), \(y_{ij}=\sigma(net_{ij})\).
2.\(y_{ij}\)
\(y_{21}=\sigma(net_{21})\).
\(y_{31}=\sigma(net_{31})\).
因此通项公式为：\(y_{ij}=\sigma(net_{ij})\).
3.误差\(E\)
\(E=\sum_{i=1}^{2}(y_{3i}-t_{i})^2\).
4.\(\nabla_{3ij}\)（输出层各\(\omega\)的梯度值）
\(\frac{\partial E}{\partial net_{31}}=\frac{E}{\partial y_{31}}\cdot \frac{\partial y_{31}}{\partial {net_{31}}}=2(y_{31}-t_1)y_{31}(1-y_{31})\).
\(\therefore \frac{E}{\partial net_{3i}}=2(y_{3i}-t_i)y_{3i}(1-y_{3i})\).
\(\nabla_{311}=\frac{\partial E}{\partial net_{31}} \cdot \frac{\partial net_{31}}{\partial \omega_{311}}=\frac{\partial E}{\partial net_{31}}\cdot y_{21}\).
因此通项公式为：\(\nabla_{3ij}=\frac{\partial E}{\partial net_{3i}}\cdot \frac{\partial net_{3i}}{\partial \omega_{3ij}}=2(y_{3i}-t_{i})y_{3i}(1-y_{3i})y_{2i}\).
5.\(\nabla_{2ij}\)（隐藏层各\(\omega\)的梯度值）
\(\nabla_{211}=\frac{\partial E}{\partial \omega_{211}}=\frac{\partial E}{\partial net_{31}}\cdot \frac{\partial net_{31}}{\partial y_{21}}\cdot \frac{\partial y_{21}}{\partial net_{21}}\cdot \frac{\partial{net_{21}}}{\partial \omega_{211}}+\frac{\partial E}{\partial net_{32}}\cdot \frac{\partial net_{32}}{\partial y_{21}}\cdot \frac{\partial y_{21}}{\partial net_{21}}\cdot \frac{\partial{net_{21}}}{\partial \omega_{211}}\\=\sum_{i=1}^{2}(\frac{\partial E}{\partial net_{3i}}\cdot \frac{\partial net_{3i}}{\partial y_{2i}})\cdot \frac{\partial y_{21}}{\partial net_{21}} \cdot \frac{\partial net_{21}}{\partial \omega_{211}}\\=\sum_{i=1}^{2}(\frac{\partial E}{\partial net_{3i}}\cdot \omega_{3i1})\cdot y_{21}(1-y_{21})\omega_{21}\)
因此通项公式为：\(\nabla_{2ij}=\frac{\partial E}{\partial \omega_{2ij}}=\sum_{k=1}^{2}(\frac{\partial E}{\partial net_{3k}}\cdot \omega_{3ki})\cdot y_{2i}(1-y_{2i})\omega_{2ij}\).