时间戳

首页 > 解决方案 > Tensorflow 2.0：使用 GradientTape 显式计算和应用渐变时，“numpy.dtype”对象没有属性“is_floating”

tensorflow - Tensorflow 2.0：使用 GradientTape 显式计算和应用渐变时，“numpy.dtype”对象没有属性“is_floating”

问题描述

对于强化学习，我想明确

我用一个简单的策略网络创建了一个代理：

def simple_policy_model(self):        
    inputs = Input(shape=(self.state_size,), name="Input")
    outputs = Dense(self.action_size, activation='softmax', name="Output")(inputs)
    predict_model = Model(inputs=[inputs], outputs=[outputs])
    return predict_model

然后我尝试获得渐变：

agent = REINFORCE_Agent(state_size=env.observation_space.shape[0],
                        action_size=env.action_space.n)
print(agent.predict_model.summary())
state_memory = np.random.uniform(size=(3,4))/10
#state_memory = tf.convert_to_tensor(state_memory)
print(state_memory)
print(agent.predict_model.predict(state_memory))

with tf.GradientTape() as tape:
    probs = agent.predict_model.predict(state_memory)
    ### fails below ###
    grads = tape.gradient(probs, agent.predict_model.trainable_weights)

输出：

Model: "model_18"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
Input (InputLayer)           (None, 4)                 0         
_________________________________________________________________
Output (Dense)               (None, 2)                 10        
=================================================================
Total params: 10
Trainable params: 10
Non-trainable params: 0
_________________________________________________________________
None
state_memory [[0.01130021 0.01476066 0.09524527 0.05552276]
 [0.02018996 0.03127809 0.07232339 0.07146596]
 [0.08925738 0.08890574 0.04845396 0.0056015 ]]
prediction [[0.5127161  0.4872839 ]
 [0.5063317  0.49366832]
 [0.4817074  0.51829267]]
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
...
AttributeError: 'numpy.dtype' object has no attribute 'is_floating'

如果我通过取消注释 convert_to_tensor 将 state_memory 转换为张量，它将在 .predict() 处失败：

ValueError: If your data is in the form of symbolic tensors, you should specify the `steps` argument (instead of the `batch_size` argument, because symbolic tensors are expected to produce batches of input data).

看起来很简单但很卡住，知道获得渐变的正确方法是什么吗？

标签： tensorflowgradienttensorflow2.0

解决方案

问题是，

probs = agent.predict_model.predict(state_memory)

产生一个 numpy 张量作为输出。你不能得到梯度w.r.tnumpy张量。相反，您需要tf.Tensor模型中的 a 。为此，请执行以下操作。

with tf.GradientTape() as tape:
    probs = agent.predict_model(state_memory)
    ### fails below ###
grads = tape.gradient(probs, agent.predict_model.trainable_weights)

推荐阅读