python - 为 Tensorflow 2.0 适配 Tensorflow RNN Seq2Seq 模型代码
问题描述
我对 Tensorflow 非常陌生,并且一直在通过这个链接搞一个简单的聊天机器人构建项目。
有很多警告说,Tensorflow 2.0 将弃用某些东西,我应该升级,所以我做了。然后,我使用自动Tensorflow 代码升级程序将所有必要的文件更新到 2.0。这有一些错误。
在处理 model.py 文件时,它返回了以下警告:
133:20: WARNING: tf.nn.sampled_softmax_loss requires manual check. `partition_strategy` has been removed from tf.nn.sampled_softmax_loss. The 'div' strategy will be used by default.
148:31: WARNING: Using member tf.contrib.rnn.DropoutWrapper in deprecated module tf.contrib.rnn. (Manual edit required) tf.contrib.rnn.* has been deprecated, and widely used cells/functions will be moved to tensorflow/addons repository. Please check it there and file Github issues if necessary.
148:31: ERROR: Using member tf.contrib.rnn.DropoutWrapper in deprecated module tf.contrib. tf.contrib.rnn.DropoutWrapper cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
171:33: ERROR: Using member tf.contrib.legacy_seq2seq.embedding_rnn_seq2seq in deprecated module tf.contrib. tf.contrib.legacy_seq2seq.embedding_rnn_seq2seq cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
197:27: ERROR: Using member tf.contrib.legacy_seq2seq.sequence_loss in deprecated module tf.contrib. tf.contrib.legacy_seq2seq.sequence_loss cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
我遇到的主要问题是使用现在不存在的contrib
模块中的代码。如何调整以下三个代码块以便它们在 TensorFlow 2.0 中工作?
# Define the network
# Here we use an embedding model, it takes integer as input and convert them into word vector for
# better word representation
decoderOutputs, states = tf.contrib.legacy_seq2seq.embedding_rnn_seq2seq(
self.encoderInputs, # List<[batch=?, inputDim=1]>, list of size args.maxLength
self.decoderInputs, # For training, we force the correct output (feed_previous=False)
encoDecoCell,
self.textData.getVocabularySize(),
self.textData.getVocabularySize(), # Both encoder and decoder have the same number of class
embedding_size=self.args.embeddingSize, # Dimension of each word
output_projection=outputProjection.getWeights() if outputProjection else None,
feed_previous=bool(self.args.test) # When we test (self.args.test), we use previous output as next input (feed_previous)
)
# Finally, we define the loss function
self.lossFct = tf.contrib.legacy_seq2seq.sequence_loss(
decoderOutputs,
self.decoderTargets,
self.decoderWeights,
self.textData.getVocabularySize(),
softmax_loss_function= sampledSoftmax if outputProjection else None # If None, use default SoftMax
)
encoDecoCell = tf.contrib.rnn.DropoutWrapper(
encoDecoCell,
input_keep_prob=1.0,
output_keep_prob=self.args.dropout
)
解决方案
tf.contrib
基本上是 TensorFlow 社区做出的贡献,它的工作原理如下。
- 社区成员可以提交代码,然后将其与标准 TensorFlow 包一起分发。他们的代码由 TensorFlow 团队审查并作为 TensorFlow 测试的一部分进行测试。
现在在 tensorflow 2 中,Tensorflow 删除了 contrib,现在 contrib 中的每个项目都有其未来的三个选项之一:移至核心;移动到单独的存储库;或删除。
您可以从此链接查看属于哪个类别的所有项目列表。
来到替代解决方案,将代码从 Tensorflow 1 迁移到 Tensorflow 2 不会自动发生,您必须手动更改。
您可以改用以下替代方法。
tf.contrib.rnn.DropoutWrapper
你可以把它改成tf.compat.v1.nn.rnn_cell.DropoutWrapper
对于序列到序列,您可以使用TensorFlow Addons
.
TensorFlow Addons 项目包括许多序列到序列工具,可让您轻松构建生产就绪的编码器-解码器。
例如,您可以使用如下所示的内容。
import tensorflow_addons as tfa
encoder_inputs = keras.layers.Input(shape=[None], dtype=np.int32)
decoder_inputs = keras.layers.Input(shape=[None], dtype=np.int32)
sequence_lengths = keras.layers.Input(shape=[], dtype=np.int32)
embeddings = keras.layers.Embedding(vocab_size, embed_size)
encoder_embeddings = embeddings(encoder_inputs)
decoder_embeddings = embeddings(decoder_inputs)
encoder = keras.layers.LSTM(512, return_state=True)
encoder_outputs, state_h, state_c = encoder(encoder_embeddings)encoder_state = [state_h, state_c]
sampler = tfa.seq2seq.sampler.TrainingSampler()
decoder_cell = keras.layers.LSTMCell(512)
output_layer = keras.layers.Dense(vocab_size)
decoder = tfa.seq2seq.basic_decoder.BasicDecoder(decoder_cell, sampler,
output_layer=output_layer)
final_outputs, final_state, final_sequence_lengths = decoder(
decoder_embeddings, initial_state=encoder_state,
sequence_length=sequence_lengths)
Y_proba = tf.nn.softmax(final_outputs.rnn_output)
model = keras.Model(inputs=[encoder_inputs, decoder_inputs,
sequence_lengths],
outputs=[Y_proba])
同样,您需要将所有使用 tf.contrib 的方法更改为兼容的方法。
我希望这能回答你的问题。
推荐阅读
- angular - 错误 - 此命令只能在 CLI 项目内运行
- android - Android Wear:无法通过 wifi 或蓝牙连接手表
- android - Android studio 无法将 apk 安装到 Xiaomi Note5A (Redmi) 中(安装失败并显示 INSTALL_FAILED_USER_RESTRICTED 消息:)
- objective-c - Swift 4:将 Objective-c 转换为 Swift 时,非标称类型“T”不支持显式初始化
- jmeter - 如何使用后端侦听器将 jmeter 结果推送到多个数据库?
- php - 在 Woocommerce 的单个产品页面中添加产品备注字段
- macos - 在 macOS 上安装 VMware Player 时出现问题
- android - 如何使颤振应用程序响应?
- r - 扫描一列以查找超过某个值的值,并将整个列移动到 R 中的新数据框中
- spring - Spring Framework:由于浏览器中的多个选项卡而获取错误的 processFormSubmission() 对象