python - 神经网络 - 检查节点激活

Question

我参与了一个研究项目，该项目正在研究在安全关键环境中使用神经网络。这项研究的监管框架的一部分针对的是系统中必须没有休眠代码的状态。必须有一条通过系统每个部分的路径，并且该路径必须是可测试/可验证的。

显然，神经网络由许多节点组成。输入/输出节点很容易测试激活，但有人知道测试隐藏层节点激活的方法吗？

显然，激活取决于节点的输入值和激活函数，并且可能有一种数学方法。

最终代码将使用 C/C++，但我们正在使用 Python 进行 NN 开发。因此，任何涉及相关工具集的想法都会受到欢迎。如果这有助于此测试，我还可以将 NN 结构和矩阵导出/导入到另一个包或环境。

希望你们都充满了想法，因为谷歌没有提供任何东西。:(

谢谢。

标签： pythonneural-networkartificial-intelligence

Answer 1

Somehow techytushar's comment nudged my brain into a new line of reasoning, which I think has been very helpful:

So the problem I'm addressing is: 'There can be no dormant code.' Be that lines of C or array elements that are never, and can never, be accessed.

So when the trained NN runs as a compiled C application, the application will calculate the value for each neuron and evaluate its activation function irrespective of the node's input value(s). So actually there is no such thing as dormant code or array elements in this regard. Just a true/false output for that node's activation at that moment. It might change in the next moment. It'll all be re calculated, even if mathmatically the result is always for no activation.

So the question then moves away from this subject, to ensuring that no combinations of node activation can result in the system being in a dangerous state. That's off topic of the original question, so I think I can draw a line under this...?