tensorflow - 自定义层输出 Keras 的尺寸
问题描述
通常，keras 层的输出维度为(None, c, h, w), or (None, h, w, c)，具体取决于channels_firstorchannels_last配置。
我正在尝试使用带有两个输入的自定义 keras 层。当我打印模型摘要时，它不显示None尺寸。
如何对我的自定义层进行编程以包含此无维度？
我认为这可能是我收到错误的原因
No data provided for "crfrnn". Need data for each key in: ['crfrnn']
crfrnn是我的自定义层的名称
我尝试在方法中重塑输出call()，不幸的是无济于事。
我确保输入数据的形状和准备得当。请注意，在我尝试在图层顶部添加此图层之前，我的代码训练良好plant_output（请参阅下面的摘要）
下面是自定义层的代码，它是从这个 github 存储库中采用的：
class CrfRnnLayer(tf.keras.layers.Layer):
""" Implements the CRF-RNN layer described in:
Conditional Random Fields as Recurrent Neural Networks,
S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang and P. Torr,
ICCV 2015
"""
def __init__(self, image_dims, num_classes,
theta_alpha, theta_beta, theta_gamma,
num_iterations, NCHW, **kwargs):
self.image_dims = image_dims
self.num_classes = num_classes
self.theta_alpha = theta_alpha
self.theta_beta = theta_beta
self.theta_gamma = theta_gamma
self.num_iterations = num_iterations
self.NCHW = NCHW
self.spatial_ker_weights = None
self.bilateral_ker_weights = None
self.compatibility_matrix = None
super(CrfRnnLayer, self).__init__(**kwargs)
def build(self, input_shape):
if not self.NCHW:
channel_axis = -1
if input_shape[channel_axis] is None:
raise ValueError('The channel dimension of the inputs '
'should be defined. Found `None`.')
input_dim = input_shape[0][channel_axis]
self.input_spec = [tf.keras.layers.InputSpec(shape=(None, input_shape[0][1], input_shape[0][2], input_shape[0][3])), tf.keras.layers.InputSpec(shape=(None, input_shape[1][1], input_shape[1][2], input_shape[1][3]))]
else:
channel_axis = 1
if input_shape[channel_axis] is None:
raise ValueError('The channel dimension of the inputs '
'should be defined. Found `None`.')
input_dim = input_shape[0][channel_axis]
self.input_spec = [tf.keras.layers.InputSpec(shape=(None, input_shape[0][1], input_shape[0][2], input_shape[0][3])), tf.keras.layers.InputSpec(shape=(None, input_shape[1][1], input_shape[1][2], input_shape[1][3]))]
# Weights of the spatial kernel
self.spatial_ker_weights = self.add_weight(name='spatial_ker_weights',
shape=(self.num_classes, self.num_classes),
initializer=_diagonal_initializer,
trainable=True)
# Weights of the bilateral kernel
self.bilateral_ker_weights = self.add_weight(name='bilateral_ker_weights',
shape=(self.num_classes, self.num_classes),
initializer=_diagonal_initializer,
trainable=True)
# Compatibility matrix
self.compatibility_matrix = self.add_weight(name='compatibility_matrix',
shape=(self.num_classes, self.num_classes),
initializer=_potts_model_initializer,
trainable=True)
super(CrfRnnLayer, self).build(input_shape)
def call(self, inputs):
unaries = tf.transpose(inputs[0][0, :, :, :], perm=(2, 0, 1))
rgb = tf.transpose(inputs[1][0, :, :, :], perm=(2, 0, 1))
#input is channels first
c, h, w = self.num_classes, self.image_dims[0], self.image_dims[1]
all_ones = np.ones((c, h, w), dtype=np.float32)
# Prepare filter normalization coefficients
spatial_norm_vals = custom_module.high_dim_filter(all_ones, rgb, bilateral=False,
theta_gamma=self.theta_gamma)
bilateral_norm_vals = custom_module.high_dim_filter(all_ones, rgb, bilateral=True,
theta_alpha=self.theta_alpha,
theta_beta=self.theta_beta)
q_values = unaries
for i in range(self.num_iterations):
softmax_out = tf.nn.softmax(q_values, 0)
# Spatial filtering
spatial_out = custom_module.high_dim_filter(softmax_out, rgb, bilateral=False,
theta_gamma=self.theta_gamma)
spatial_out = spatial_out / spatial_norm_vals
# Bilateral filtering
bilateral_out = custom_module.high_dim_filter(softmax_out, rgb, bilateral=True,
theta_alpha=self.theta_alpha,
theta_beta=self.theta_beta)
bilateral_out = bilateral_out / bilateral_norm_vals
# Weighting filter outputs
message_passing = (tf.matmul(self.spatial_ker_weights,
tf.reshape(spatial_out, (c, -1))) +
tf.matmul(self.bilateral_ker_weights,
tf.reshape(bilateral_out, (c, -1))))
# Compatibility transform
pairwise = tf.matmul(self.compatibility_matrix, message_passing)
# Adding unary potentials
pairwise = tf.reshape(pairwise, (c, h, w))
q_values = unaries - pairwise
#output is channels last
return tf.transpose(tf.reshape(q_values, (1, c, h, w)), perm=(0, 2, 3, 1))
def compute_output_shape(self, input_shape):
return input_shape
该层有两个输入，每个输入都有 size (None, 384, 512, 3)。我希望输出是一样的，但是当我编译模型时，产生的摘要如下（请注意，我没有在中间显示很多层，因为我违反了这个平台上的字符限制，最重要的层要查看的是input_node,plant_output和 custom crfrnn):
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_node (InputLayer) (None, 384, 512, 3) 0
__________________________________________________________________________________________________
encoder_conv1 (Conv2D) (None, 384, 512, 32) 128 input_node[0][0]
__________________________________________________________________________________________________
bneck1_dense_encoder1 (Conv2D) (None, 384, 512, 8) 264 encoder_conv1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 384, 512, 8) 32 bneck1_dense_encoder1[0][0]
__________________________________________________________________________________________________
leaky_re_lu (LeakyReLU) (None, 384, 512, 8) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
conv1_dense_encoder1 (Conv2D) (None, 384, 512, 4) 292 leaky_re_lu[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 384, 512, 4) 0 conv1_dense_encoder1[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 384, 512, 36) 0 encoder_conv1[0][0]
dropout[0][0]
__________________________________________________________________________________________________
bneck2_dense_encoder1 (Conv2D) (None, 384, 512, 8) 296 concatenate[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 384, 512, 8) 32 bneck2_dense_encoder1[0][0]
__________________________________________________________________________________________________
leaky_re_lu_1 (LeakyReLU) (None, 384, 512, 8) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2_dense_encoder1 (Conv2D) (None, 384, 512, 4) 292 leaky_re_lu_1[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 384, 512, 4) 0 conv2_dense_encoder1[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 384, 512, 40) 0 concatenate[0][0]
dropout_1[0][0]
__________________________________________________________________________________________________
bneck3_dense_encoder1 (Conv2D) (None, 384, 512, 8) 328 concatenate_1[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 384, 512, 8) 32 bneck3_dense_encoder1[0][0]
__________________________________________________________________________________________________
leaky_re_lu_2 (LeakyReLU) (None, 384, 512, 8) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
conv3_dense_encoder1 (Conv2D) (None, 384, 512, 4) 292 leaky_re_lu_2[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 384, 512, 4) 0 conv3_dense_encoder1[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 384, 512, 12) 0 dropout[0][0]
dropout_1[0][0]
dropout_2[0][0]
__________________________________________________________________________________________________
encoder_concat1 (Concatenate) (None, 384, 512, 44) 0 encoder_conv1[0][0]
concatenate_2[0][0]
__________________________________________________________________________________________________
encoder_bneck1 (Conv2D) (None, 384, 512, 22) 990 encoder_concat1[0][0]
__________________________________________________________________________________________________
encoder_downsample1 (Conv2D) (None, 192, 256, 22) 12122 encoder_bneck1[0][0]
__________________________________________________________________________________________________
bneck1_dense_encoder2 (Conv2D) (None, 192, 256, 8) 184 encoder_downsample1[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 192, 256, 8) 32 bneck1_dense_encoder2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_3 (LeakyReLU) (None, 192, 256, 8) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv1_dense_encoder2 (Conv2D) (None, 192, 256, 4) 292 leaky_re_lu_3[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 192, 256, 4) 0 conv1_dense_encoder2[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 192, 256, 26) 0 encoder_downsample1[0][0]
dropout_3[0][0]
__________________________________________________________________________________________________
bneck2_dense_encoder2 (Conv2D) (None, 192, 256, 8) 216 concatenate_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 192, 256, 8) 32 bneck2_dense_encoder2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_4 (LeakyReLU) (None, 192, 256, 8) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
conv2_dense_encoder2 (Conv2D) (None, 192, 256, 4) 292 leaky_re_lu_4[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout) (None, 192, 256, 4) 0 conv2_dense_encoder2[0][0]
__________________________________________________________________________________________________
concatenate_4 (Concatenate) (None, 192, 256, 30) 0 concatenate_3[0][0]
dropout_4[0][0]
__________________________________________________________________________________________________
bneck3_dense_encoder2 (Conv2D) (None, 192, 256, 8) 248 concatenate_4[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 192, 256, 8) 32 bneck3_dense_encoder2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_5 (LeakyReLU) (None, 192, 256, 8) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
conv3_dense_encoder2 (Conv2D) (None, 192, 256, 4) 292 leaky_re_lu_5[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout) (None, 192, 256, 4) 0 conv3_dense_encoder2[0][0]
__________________________________________________________________________________________________
concatenate_5 (Concatenate) (None, 192, 256, 12) 0 dropout_3[0][0]
dropout_4[0][0]
dropout_5[0][0]
__________________________________________________________________________________________________
encoder_concat2 (Concatenate) (None, 192, 256, 34) 0 encoder_downsample1[0][0]
concatenate_5[0][0]
__________________________________________________________________________________________________
encoder_bneck2 (Conv2D) (None, 192, 256, 17) 595 encoder_concat2[0][0]
__________________________________________________________________________________________________
encoder_downsample2 (Conv2D) (None, 96, 128, 17) 7242 encoder_bneck2[0][0]
__________________________________________________________________________________________________
bneck1_dense_encoder3 (Conv2D) (None, 96, 128, 8) 144 encoder_downsample2[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 96, 128, 8) 32 bneck1_dense_encoder3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_6 (LeakyReLU) (None, 96, 128, 8) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
conv1_dense_encoder3 (Conv2D) (None, 96, 128, 4) 292 leaky_re_lu_6[0][0]
__________________________________________________________________________________________________
dropout_6 (Dropout) (None, 96, 128, 4) 0 conv1_dense_encoder3[0][0]
__________________________________________________________________________________________________
concatenate_6 (Concatenate) (None, 96, 128, 21) 0 encoder_downsample2[0][0]
dropout_6[0][0]
__________________________________________________________________________________________________
bneck2_dense_encoder3 (Conv2D) (None, 96, 128, 8) 176 concatenate_6[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 96, 128, 8) 32 bneck2_dense_encoder3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_7 (LeakyReLU) (None, 96, 128, 8) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
conv2_dense_encoder3 (Conv2D) (None, 96, 128, 4) 292 leaky_re_lu_7[0][0]
__________________________________________________________________________________________________
dropout_7 (Dropout) (None, 96, 128, 4) 0 conv2_dense_encoder3[0][0]
__________________________________________________________________________________________________
concatenate_7 (Concatenate) (None, 96, 128, 25) 0 concatenate_6[0][0]
dropout_7[0][0]
__________________________________________________________________________________________________
bneck3_dense_encoder3 (Conv2D) (None, 96, 128, 8) 208 concatenate_7[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 96, 128, 8) 32 bneck3_dense_encoder3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_8 (LeakyReLU) (None, 96, 128, 8) 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
conv3_dense_encoder3 (Conv2D) (None, 96, 128, 4) 292 leaky_re_lu_8[0][0]
__________________________________________________________________________________________________
dropout_8 (Dropout) (None, 96, 128, 4) 0 conv3_dense_encoder3[0][0]
__________________________________________________________________________________________________
concatenate_8 (Concatenate) (None, 96, 128, 12) 0 dropout_6[0][0]
dropout_7[0][0]
dropout_8[0][0]
__________________________________________________________________________________________________
__________________________________________________________________________________________________
plant_conv (Conv2D) (None, 384, 512, 32) 416 concatenate_20[0][0]
__________________________________________________________________________________________________
plant_output (Conv2D) (None, 384, 512, 3) 99 plant_conv[0][0]
__________________________________________________________________________________________________
crfrnn (CrfRnnLayer) (1, 384, 512, 3) 27 plant_output[0][0]
input_node[0][0]
==================================================================================================
Total params: 41,833
Trainable params: 41,497
Non-trainable params: 336
__________________________________________________________________________________________________
注意 crfrnn 层的形状 (1, 384, 512, 3)。我相信这导致程序不训练，抛出错误：
Epoch 1/2000
No data provided for "crfrnn". Need data for each key in: ['crfrnn']
解决方案
问题有两个方面，第一个更重要的是要分享。
在上面的代码中，在 call() 函数中，您会看到：
unaries = tf.transpose(inputs[0][0, :, :, :], perm=(2, 0, 1))
rgb = tf.transpose(inputs[1][0, :, :, :], perm=(2, 0, 1))
我删除了这个，而是写道：
unaries = inputs[0]
rgb = inputs[1]
这假设通道首先作为通道输入，因此不需要转置。此外，但是像这样阅读它们，keras 能够推断形状，并且摘要现在具有 (None, c, h, w) 作为该层的形状。
这个错误No data provided for "crfrnn". Need data for each key in: ['crfrnn']实际上是我的一个愚蠢的错误，我输入的标签字典有错误的键（应该是 crfrnn）。我在 crfrnn repo 的原始 github 上发布了这个，因为其他人很好奇如何将这一层应用到他们自己的网络中，我已经这样做了。
推荐阅读
- c# - 如何在 Unity 中使用代码淡化色差
- javascript - 在 JavaScript 中添加 n 个空格
- c# - 手动创建的 HtmlDocument.DocumentNode 从 InnerHtml 返回 null
- fstar - 如何解决 FStar 中的这种类型冲突？
- r - 使用 mutate_at 跨嵌套数据框映射
- flutter - 如何处理“在 null 上调用了 getter 'microsecondsSinceEpoch'。” 错误？
- php - 无法正确计算时差
- python - 在 Python 中“收集并执行”的最佳实践
- netlogo - netlogo - 随机转发 -
- javascript - 如何将 DataTables 的本地化更改为德语