tensorflow - 如何在适合 keras 的过程中均衡减重?
问题描述
我的模型:
class MultiAutoencoder:
def decoder_category_branch(encoder_GRU):
decoded = RepeatVector(max_len_str, name = 'bottleneck-2')(encoder_GRU)
decoder_BLSTM = Bidirectional(LSTM(25, return_sequences=True, name='Decoder-BLSTM'))(decoded)
decoder_drop = Dropout(0.1)(decoder_BLSTM)
decoder_GRU = GRU(100, activation='tanh', return_sequences=False, name='Decoder-GRU')(decoder_drop)
decoder_output = Dense(max_len_str, activation='softmax', name='decoder-output')(decoder_GRU)
return decoder_output
def encoder_binary_branch(emb_norm):
encoder_BLSTM = Bidirectional(LSTM(25, activation='tanh', return_sequences=True, name='Encoder-LSTM'))(emb_norm)
encoder_drop = Dropout(0.1)(encoder_BLSTM)
encoder_GRU = GRU(25, activation='tanh', return_sequences=False, name = 'bottleneck-1')(encoder_drop)
encoder_dense_1 = Dense(20, activation='relu')(encoder_GRU)
encoder_dense_2 = Dense(10, activation='relu')(encoder_dense_1)
encoder_dense_3 = Dense(5, activation='relu')(encoder_dense_2)
encoder_output = Dense(1, activation='sigmoid', name = 'encoder-output')(encoder_dense_3)
return encoder_GRU, encoder_output
def build():
encoder_inputs = Input(shape=(max_len_str,), name='Encoder-Input')
emb_layer = Embedding(input_dim=len(word_index)+1, output_dim=100, input_length = max_len_str, name='Embedding', mask_zero=False)(encoder_inputs)
emb_drop = SpatialDropout1D(0.1)(emb_layer)
emb_norm = BatchNormalization()(emb_drop)
encoder_GRU, encoder_output = MultiAutoencoder.encoder_binary_branch(emb_norm)
decoder_output = MultiAutoencoder.decoder_category_branch(encoder_GRU)
model = Model(inputs=encoder_inputs, outputs=[encoder_output, decoder_output], name="MultiAutoencoder")
return model
和损失权:
losses = {
"decoder-output": "categorical_crossentropy",
"encoder-output": "binary_crossentropy",
}
loss_weights = {
'decoder-output': 0.001,
'encoder-output': 70
}
metrics={'decoder-output': [keras.metrics.CategoricalAccuracy()],
'encoder-output': [keras.metrics.BinaryAccuracy()]}
我的目标是提高编码器的准确性。但是在学习的过程中,解码器输出的损失会增加:
Epoch 23/30
169/169 [==============================] - 55s 327ms/step - loss: 29.3175 - encoder-output_loss: 0.0190 - decoder-output_loss: 27989.1639 - encoder-output_binary_accuracy: 0.9970 - decoder-output_categorical_accuracy: 0.0451 - val_loss: 30.8555 - val_encoder-output_loss: 0.0358 - val_decoder-output_loss: 28350.3809 - val_encoder-output_binary_accuracy: 0.9937 - val_decoder-output_categorical_accuracy: 0.0407
Epoch 24/30
169/169 [==============================] - 55s 326ms/step - loss: 29.9731 - encoder-output_loss: 0.0238 - decoder-output_loss: 28307.4726 - encoder-output_binary_accuracy: 0.9959 - decoder-output_categorical_accuracy: 0.0464 - val_loss: 31.3778 - val_encoder-output_loss: 0.0356 - val_decoder-output_loss: 28886.6445 - val_encoder-output_binary_accuracy: 0.9937 - val_decoder-output_categorical_accuracy: 0.0407
Epoch 25/30
169/169 [==============================] - 56s 329ms/step - loss: 30.5244 - encoder-output_loss: 0.0222 - decoder-output_loss: 28972.7803 - encoder-output_binary_accuracy: 0.9964 - decoder-output_categorical_accuracy: 0.0448 - val_loss: 31.9176 - val_encoder-output_loss: 0.0355 - val_decoder-output_loss: 29430.2871 - val_encoder-output_binary_accuracy: 0.9936 - val_decoder-output_categorical_accuracy: 0.0407
Epoch 26/30
169/169 [==============================] - 56s 333ms/step - loss: 31.2009 - encoder-output_loss: 0.0236 - decoder-output_loss: 29546.3366 - encoder-output_binary_accuracy: 0.9960 - decoder-output_categorical_accuracy: 0.0465 - val_loss: 32.3901 - val_encoder-output_loss: 0.0353 - val_decoder-output_loss: 29916.2539 - val_encoder-output_binary_accuracy: 0.9938 - val_decoder-output_categorical_accuracy: 0.0407
Epoch 27/30
169/169 [==============================] - 56s 330ms/step - loss: 31.4490 - encoder-output_loss: 0.0227 - decoder-output_loss: 29856.7499 - encoder-output_binary_accuracy: 0.9961 - decoder-output_categorical_accuracy: 0.0442 - val_loss: 32.9726 - val_encoder-output_loss: 0.0352 - val_decoder-output_loss: 30508.6621 - val_encoder-output_binary_accuracy: 0.9935 - val_decoder-output_categorical_accuracy: 0.0407
Epoch 28/30
169/169 [==============================] - 56s 330ms/step - loss: 32.5321 - encoder-output_loss: 0.0212 - decoder-output_loss: 31044.8867 - encoder-output_binary_accuracy: 0.9966 - decoder-output_categorical_accuracy: 0.0457 - val_loss: 33.4694 - val_encoder-output_loss: 0.0344 - val_decoder-output_loss: 31062.6191 - val_encoder-output_binary_accuracy: 0.9938 - val_decoder-output_categorical_accuracy: 0.0407
Epoch 29/30
169/169 [==============================] - 56s 332ms/step - loss: 33.0040 - encoder-output_loss: 0.0235 - decoder-output_loss: 31359.5384 - encoder-output_binary_accuracy: 0.9961 - decoder-output_categorical_accuracy: 0.0454 - val_loss: 33.9589 - val_encoder-output_loss: 0.0347 - val_decoder-output_loss: 31532.6406 - val_encoder-output_binary_accuracy: 0.9936 - val_decoder-output_categorical_accuracy: 0.0407
Epoch 30/30
169/169 [==============================] - 56s 330ms/step - loss: 33.4066 - encoder-output_loss: 0.0252 - decoder-output_loss: 31645.2180 - encoder-output_binary_accuracy: 0.9956 - decoder-output_categorical_accuracy: 0.0443 - val_loss: 34.6286 - val_encoder-output_loss: 0.0355 - val_decoder-output_loss: 32143.4551 - val_encoder-output_binary_accuracy: 0.9936 - val_decoder-output_categorical_accuracy: 0.0407
如何在训练期间校准损失权重?请帮助完成代码...
解决方案
推荐阅读
- javascript - Rails 6 和 Webpacker 上的签名板
- python - 为什么我的“分组依据”过滤器不显示选择字段中的字符串?
- flutter - Flutter - 使用固定小部件滚动
- python - 无法通过 pip 安装 pyfftw3
- python - 我想使用 sshtunnel 重新连接并使用 pythonanywhere 数据库查询数据库
- android - 需要在 Recyclerview 水平方向内一次显示两个项目
- git - 无法从 git 镜像触发 Jenkins MultiBranch Pipeline 作业
- vue.js - 访问状态时:未知突变类型
- java - Java:PrintWriter 是否提供了某种形式的关于请求源平台首选行结尾的“线索”?
- react-native - 如何在 React Native 0.63 中滚动到滚动视图的底部?