tensorflow - 这个 Dense 层是如何计算它的维度的？

Question

layer = tf.keras.layers.Dense(10, input_shape=(None, 5))
x = layer(tf.zeros([10,5])) 

x 是 [10,10] 矩阵。这是为什么？为什么不是 [10,5]？它似乎在内部执行 [10,5] * [5,10]。

Dense 层如何具有 [5,10] 的形状？

标签： tensorflow

Answer 1

You're passing in an input of shape (None, 5)

Then you have a dense layer with 10 units. This dense layer have two sets of trainable parameters.

kernel => A (5,10) Matrix
bias => A (10) vector

The dense layer know the correct shape to construct because, you're passing the input_shape parameter.

Then when you call layer(tf.zeros([10,5])), it does the following computation.

matmul(input[10,5], kernel[5, 10]) + bias[1, 10] => x[10,10]

So as you can see, it results in a [10,10] matrix.