时间戳

首页 > 解决方案 > 尝试在 Tensorflow 高级 API 上编写自己的损失函数

python - 尝试在 Tensorflow 高级 API 上编写自己的损失函数

问题描述

在处理强化学习问题时，我试图提出自己的损失函数。（Open Ai 的“CartPole-v0”游戏。）

inputs = keras.Input(shape=(32,))  # Returns a placeholder tensor

# A layer instance is callable on a tensor, and returns a tensor.
x = keras.layers.Dense(64, activation='relu')(inputs)
x = keras.layers.Dense(64, activation='relu')(x)
output = keras.layers.Dense(10, activation='softmax')(x)

# Instantiate the model given inputs and outputs.
model = keras.Model(inputs=inputs, outputs=output)

chosen_action = tf.argmax(output,1)

#The next six lines establish the training proceedure. We feed the reward and chosen action into the network
#to compute the loss, and use it to update the network.
reward_holder = tf.placeholder(shape=[None],dtype=tf.float32)
action_holder = tf.placeholder(shape=[None],dtype=tf.int32)

#here, basically I am finding the final decisioned outputs of neural network.
indexes = tf.range(0, tf.shape(output)[0]) * tf.shape(output)[1] + action_holder
responsible_outputs = tf.gather(tf.reshape(output, [-1]), indexes)

#this is the loss function that I want to use
myloss = -tf.reduce_mean(tf.log(responsible_outputs)*reward_holder)

直到这里一切都很酷。

model.compile(optimizer=tf.train.AdamOptimizer(), 
              loss=myloss,
              metrics=['accuracy'])

当我这样做时，我得到了这个错误；

TypeError                                 Traceback (most recent call last)
<ipython-input-34-2b1cccfa091b> in <module>()
      1 model.compile(optimizer=tf.train.AdamOptimizer(), 
      2               loss=myloss,
----> 3               metrics=['accuracy'])

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py in compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, weighted_metrics, target_tensors, **kwargs)
    172             `optimizer`, `loss`, `metrics` or `sample_weight_mode`.
    173     """
--> 174     loss = loss or {}
    175     if context.executing_eagerly() and not isinstance(
    176         optimizer, (tf_optimizer_module.Optimizer, optimizers.TFOptimizer)):

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in __bool__(self)
    663       `TypeError`.
    664     """

--> 665     raise TypeError("Using a `tf.Tensor` as a Python `bool` is not allowed. "
    666                     "Use `if t is not None:` instead of `if t:` to test if a "
    667                     "tensor is defined, and use TensorFlow ops such as "

TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

标签： pythontensorflowneural-network

解决方案

我认为问题在于您没有在 AdamOptimizer 中指定学习率。亚当优化器使用自适应动量梯度下降算法来更新神经网络中的权重和偏差。更新速度取决于指定的学习率。

更正是： optimizer=tf.train.AdamOptimizer( learning_rate )

推荐阅读