tensorflow - Tensorflow Estimator + 带占位符的数据集 API + Keras 模型

Question

我已经建立了一个自定义 keras 模型（create_model）。然后使用该模型来构建估计器。使用估计器，我希望使用 input_fn 参数 = train_input_fn 训练我的模型。我已经使用数据集 API 构建了这个函数。我收到错误消息：您必须为占位符张量提供一个值。

首先，我按照这个演示构建了 model_fn：https ://github.com/tensorflow/models/blob/master/official/mnist/mnist.py#L102 。

我从这个演示中借用了数据集 API 函数 (dataset_train_input_fn)，只是将数据、num_epochs 和 batch_size 参数添加到函数：https ://gist.github.com/peterroelants/6a7b3cc802f7f855744e3a74a1fab354#file-mnist_new_dataset_hook-py 。

我像这样构建了我的估算器：

model_function = model_fn
classifier = tf.estimator.Estimator(    model_fn = model_function, 
                                        model_dir = OUTDIR )

我尝试像这样训练我的估算器：

train_input_function, train_iterator_hook = dataset_train_input_fn(train_images, train_labels, 10, 128)
classifier.train(input_fn=train_input_function, hooks = [train_iterator_hook])

而我自定义的 Keras 模型如下：

def create_model():
shape = (4,4096)
# define input for each network
inp = tf.keras.layers.Input(shape = shape, dtype = 'float32', name = '4dim_input')
slice0 = tf.keras.layers.Lambda(lambda inp: tf.slice(inp, [0, 0, 0], [-1, 1, 4096]), output_shape = (-1, 1, 4096), name = 'image_1')
slice1 = tf.keras.layers.Lambda(lambda inp: tf.slice(inp, [0, 1, 0], [-1, 1, 4096]), output_shape = (-1, 1, 4096), name = 'image_2')
slice2 = tf.keras.layers.Lambda(lambda inp: tf.slice(inp, [0, 2, 0], [-1, 1, 4096]), output_shape = (-1, 1, 4096), name = 'image_3')
slice3 = tf.keras.layers.Lambda(lambda inp: tf.slice(inp, [0, 3, 0], [-1, 1, 4096]), output_shape = (-1, 1, 4096), name = 'image_4')

# for each output, pass through stages of FC layers
model_2 = tf.keras.models.clone_model(model_stage2_fc_layers)
#model_2 = stage2_fc_layers

out1 = model_2(slice0(inp))
out2 = model_2(slice1(inp))
out3 = model_2(slice2(inp))
out4 = model_2(slice3(inp))

# merging models
out = tf.keras.layers.concatenate([out1, out2, out3, out4])

# stage3_fc_layers
prediction = stage3_fc_layers(out)

# generate model from layers above
model = tf.keras.models.Model(inputs = inp, outputs = prediction)
return model

我不知道这个混合物的哪一部分阻止了 train_input_function 返回张量。

先感谢您。

这是完整的错误跟踪：

    Caused by op '4dim_input', defined at:
  File "tf_greater_london_summary.py", line 343, in <module>
    classifier.train(input_fn=train_input_function, hooks = [train_iterator_hook])
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 354, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 1207, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 1237, in _train_model_default
    features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 1195, in _call_model_fn
    model_fn_results = self._model_fn(features=features, **kwargs)
  File "tf_greater_london_summary.py", line 263, in model_fn
    model = final_model()
  File "tf_greater_london_summary.py", line 188, in final_model
    inp = tf.keras.layers.Input(shape = shape, dtype = 'float32', name = '4dim_input')
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/input_layer.py", line 229, in Input
    input_tensor=tensor)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/input_layer.py", line 112, in __init__
    name=self.name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py", line 1747, in placeholder
    return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 5206, in placeholder
    "Placeholder", dtype=dtype, shape=shape, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor '4dim_input' with dtype float and shape [?,4,4096]
     [[node 4dim_input (defined at tf_greater_london_summary.py:188)  = Placeholder[dtype=DT_FLOAT, shape=[?,4,4096], _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

标签： tensorflowkerastensorflow-datasetstensorflow-estimator