python - RLlib 改变观察的形状，将 [None] 添加到形状元组

Question

RLlib（0.7.3 版）提供了 Box(10, 3) 的观察形状，我想将其与 FCN 代理一起使用。但图书馆似乎为它增加了另一个维度。由于这个添加，RLlib 尝试为代理使用视觉网络。

我如何将它与 FCN 代理一起使用。

在文件 ray/rllib/policy/dynamic_tf_policy.py 的第 108 行。

    if existing_inputs is not None:
        obs = existing_inputs[SampleBatch.CUR_OBS]
        if self._obs_include_prev_action_reward:
            prev_actions = existing_inputs[SampleBatch.PREV_ACTIONS]
            prev_rewards = existing_inputs[SampleBatch.PREV_REWARDS]
    else:
        obs = tf.placeholder(
            tf.float32,
            shape=[None] + list(obs_space.shape), # <----------------
            name="observation")
        if self._obs_include_prev_action_reward:
            prev_actions = ModelCatalog.get_action_placeholder(
                action_space)
            prev_rewards = tf.placeholder(
                tf.float32, [None], name="prev_reward")

    self.input_dict = {
        SampleBatch.CUR_OBS: obs,
        SampleBatch.PREV_ACTIONS: prev_actions,
        SampleBatch.PREV_REWARDS: prev_rewards,
        "is_training": self._get_is_training_placeholder(),
    }

标签： pythonreinforcement-learningrayrllib