python - 使用 Keras TimeDistributed 层时拓扑排序失败

Question

我尝试使用 keras TimeDistributed 层将回顾维度的最后一列点到 4D 张量（样本、时间步长、回顾、特征）上的先前回顾期。该模型可以正常运行，但是当我运行 Model.fit() 时，它会发出一个警告，即图形无法按拓扑顺序排序。

这个论坛说它可以搞砸模型训练。那么我能做些什么来防止这种情况发生呢？

环境：

1. TensorFlow-GPU 1.15.0
2. CUDA V10.0.130
3. 蟒蛇3.6.5
4. 凯拉斯 2.3.1
5. Keras-应用程序 1.0.8
6. Keras-预处理 1.1.0

import numpy as np
from keras.models import Model
from keras.layers import Input, TimeDistributed
import keras
# Dot layer
class Dot(keras.layers.Layer):
    def __init__(self, **kwargs):
        super(Dot, self).__init__(**kwargs)

    def call(self, x):

        ht, hT = x[:,:-1,:],x[:,-1:,:]
        ml = tf.multiply(ht, hT)

        # I believe problem come from reduce_sum
        dot = tf.reduce_sum(ml, axis=-1)
        return dot

    def compute_output_shape(self, input_shape):

        return (None,input_shape[1]-1)

num_fea = 11
num_lookback = 5
time_step = 3
sample = 2

# create model
input = Input(shape=(time_step,num_lookback,num_fea))
dot = Dot()
output = TimeDistributed(dot)(input)

M = Model(inputs=[input], outputs=[output])
M.compile(keras.optimizers.Adam(learning_rate=0.0001), loss='mse')

# create test data
data = np.arange(num_lookback*num_fea).reshape((num_lookback,num_fea))
data = np.broadcast_to(data,shape=(sample,time_step,num_lookback,num_fea))
y = np.ones(shape=(sample,time_step,num_lookback-1))

# fit model to demonstrate error
M.fit(x=data,y=y, batch_size=2, epochs=10)

警告日志

2020-03-05 08:36:17.558396: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:697] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-05 08:36:17.558777: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:533] layout failed: Invalid argument: The graph couldn't be sorted in topological order.
2020-03-05 08:36:17.559302: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:533] model_pruner failed: Invalid argument: MutableGraphView::MutableGraphView error: node 'loss/time_distributed_1_loss/mean_squared_error/weighted_loss/concat' has self cycle fanin 'loss/time_distributed_1_loss/mean_squared_error/weighted_loss/concat'.
2020-03-05 08:36:17.560121: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:533] remapper failed: Invalid argument: MutableGraphView::MutableGraphView error: node 'loss/time_distributed_1_loss/mean_squared_error/weighted_loss/concat' has self cycle fanin 'loss/time_distributed_1_loss/mean_squared_error/weighted_loss/concat'.
2020-03-05 08:36:17.560575: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:533] arithmetic_optimizer failed: Invalid argument: The graph couldn't be sorted in topological order.
2020-03-05 08:36:17.560853: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:697] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-05 08:36:17.561141: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:697] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.

标签： pythontensorflowkerasdeep-learning

Answer 1

您可以考虑使用 TensorFlow 2.x 版本。

我已迁移/升级您的代码并验证它是否适用于 google colab。您可以尝试在此处查看有关如何将代码迁移到 Tensorflow 2.x 的更多信息

请参考以下代码

import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, TimeDistributed
#import keras
# Dot layer
class Dot(tf.keras.layers.Layer):
    def __init__(self, **kwargs):
        super(Dot, self).__init__(**kwargs)

    def call(self, x):

        ht, hT = x[:,:-1,:],x[:,-1:,:]
        ml = tf.multiply(ht, hT)

        # I believe problem come from reduce_sum
        dot = tf.reduce_sum(ml, axis=-1)
        return dot

    def compute_output_shape(self, input_shape):

        return (None,input_shape[1]-1)

num_fea = 11
num_lookback = 5
time_step = 3
sample = 2

# create model
input = Input(shape=(time_step,num_lookback,num_fea))
dot = Dot()
output = TimeDistributed(dot)(input)

M = Model(inputs=[input], outputs=[output])
M.compile(optimizer='adam', loss='mse')

# create test data
data = np.arange(num_lookback*num_fea).reshape((num_lookback,num_fea))
data = np.broadcast_to(data,shape=(sample,time_step,num_lookback,num_fea))
y = np.ones(shape=(sample,time_step,num_lookback-1))

# fit model to demonstrate error
M.fit(x=data,y=y, batch_size=2, epochs=10)