python - 使用预训练的权重初始化密集张量流层

Question

我正在尝试使用预训练的权重和偏差来初始化权tensorflow.keras.layers.Dense重和偏差。我尝试了两种方法，首先

        dims = [960,480,200,75,25]
        dense_params['weights'] = (weights['w1'], biases['b1']) # [960, 480], 960
        dense2_params['weights'] = (weights['w2'], biases['b2']) # [480, 200], 480
        dense3_params['weights'] = (weights['w3'], biases['b3']) # [200, 75], 200
        dense4_params['weights'] = (weights['w4'], biases['b4']) # [75, 25], 75
    
        # define and initialize MLP model
        mlp = Sequential([
            Dense(dims[1], input_shape=(dims[0],), # 480
                  kernel_regularizer=regularizers.l2(mlp_l2),
                  kernel_initializer=weights['w1'],
                  bias_initializer=biases['b1']),
            Activation('sigmoid'),
            Dense(dims[2], # 200
                  kernel_regularizer=regularizers.l2(mlp_l2),
                  kernel_initializer=weights['w2'],
                  bias_initializer=biases['b2']),
            Activation('sigmoid'),
            Dense(dims[3], # 75
                  kernel_regularizer=regularizers.l2(mlp_l2),
                  kernel_initializer=weights['w3'],
                  bias_initializer=biases['b3']),
            Activation('sigmoid'),
            Dense(dims[4], # 25
                  kernel_regularizer=regularizers.l2(mlp_l2),
                  kernel_initializer=weights['w4'],
                  bias_initializer=biases['b4']),
            Activation('sigmoid'),
            Dense(4, kernel_initializer=glorot_uniform(seed=5555)),
            Activation('softmax'),
        ])
        mlp.compile(optimizer=Adam(lr=lr),
                    loss='categorical_crossentropy',
                    metrics=['accuracy'])

这给出了例外

ValueError: Could not interpret initializer identifier: tf.Tensor(
[[-0.2785529   0.4547253   0.893165   ... -0.73468804 -0.9307138
   0.52965516]
 [-0.31190312  0.5024654   0.35986277 ... -0.61041874 -0.39063498
  -0.8553045 ]
 [ 0.8878816  -0.75287884 -0.63842094 ... -0.38421196 -0.16647834
  -0.26172128]
 ...

我试过的第二种方法是

    dims = [960,480,200,75,25]
    dense_params['weights'] = (weights['w1'], biases['b1']) # [960, 480], 960
    dense2_params['weights'] = (weights['w2'], biases['b2']) # [480, 200], 480
    dense3_params['weights'] = (weights['w3'], biases['b3']) # [200, 75], 200
    dense4_params['weights'] = (weights['w4'], biases['b4']) # [75, 25], 75
    mlp = Sequential([
    Dense(dims[1], input_shape=(dims[0],), # 480
          kernel_regularizer=regularizers.l2(mlp_l2),
          kernel_initializer=glorot_uniform(seed=3333),
          **dense_params),
    Activation('sigmoid'),
    Dense(dims[2], # 200
          kernel_regularizer=regularizers.l2(mlp_l2),
          kernel_initializer=glorot_uniform(seed=4444),
          **dense2_params),
    Activation('sigmoid'),
    Dense(dims[3], # 75
          kernel_regularizer=regularizers.l2(mlp_l2),
          kernel_initializer=glorot_uniform(seed=4444),
          **dense3_params),
    Activation('sigmoid'),
    Dense(dims[4], # 25
          kernel_regularizer=regularizers.l2(mlp_l2),
          kernel_initializer=glorot_uniform(seed=4444),
          **dense4_params),
    Activation('sigmoid'),
    Dense(4, kernel_initializer=glorot_uniform(seed=5555)),
    Activation('softmax'),
])
mlp.compile(optimizer=Adam(lr=lr),
            loss='categorical_crossentropy',
            metrics=['accuracy'])

这使ValueError: Layer weight shape (480,) not compatible with provided weight shape (960, 1)

我也尝试了最后一种方法，没有input_shape它会给出错误 ValueError: Layer weight shape (200,) not compatible with provided weight shape (480, 1)

权重是二维张量，尺寸显示在评论中，偏差是一维张量，也显示在评论中。如何初始化权重和偏差？

标签： pythontensorflowkerasneural-networkpre-trained-model