python - RL 网络的输入形状差异

Question

我是 DQN 和深度学习的初学者。我正在做的一项练习具有以下网络定义。

model = Sequential()
# https://keras.io/api/layers/reshaping_layers/flatten/
model.add(Flatten(input_shape=(1,) + obs_space))
 
model.add(Dense(16))
model.add(Activation('relu'))
 
model.add(Dense(32))
model.add(Activation('relu'))
 
model.add(Dense(act_space))
model.add(Activation('linear'))
 
print(model.summary())

该网络的摘要是：

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten (Flatten)            (None, 4)                 0         
_________________________________________________________________
dense_5 (Dense)              (None, 16)                80        
_________________________________________________________________
activation (Activation)      (None, 16)                0         
_________________________________________________________________
dense_6 (Dense)              (None, 32)                544       
_________________________________________________________________
activation_1 (Activation)    (None, 32)                0         
_________________________________________________________________
dense_7 (Dense)              (None, 2)                 66        
_________________________________________________________________
activation_2 (Activation)    (None, 2)                 0         
=================================================================
Total params: 690
Trainable params: 690
Non-trainable params: 0

我正在尝试使用这样的功能 API 创建这个网络..

obs_space = env.observation_space.shape
i = Input(shape=obs_space)
x = Dense(16, activation='relu')(i)
x = Dense(32, activation='relu')(x)
o = Dense(env.action_space.n, activation='linear')(x)
 
model = Model (i,o)
model.summary()

这给了我这个总结......

Model: "model_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_4 (InputLayer)         [(None, 4)]               0         
_________________________________________________________________
dense_12 (Dense)             (None, 16)                80        
_________________________________________________________________
dense_13 (Dense)             (None, 32)                544       
_________________________________________________________________
dense_14 (Dense)             (None, 2)                 66        
=================================================================
Total params: 690
Trainable params: 690
Non-trainable params: 0

但这不起作用......显然输入形状不一样

顺序（无，4）
功能[（无，4）]

我如何使用功能 API 实现顺序模型形状，以及具体做什么

model.add(Flatten(input_shape=(1,) + obs_space))

做？

标签： pythonkerasreinforcement-learning