python - 了解 Trax 中有关变压器的介绍性示例
问题描述
我的目标是了解 Trax 中有关转换器的介绍性示例,可在https://trax-ml.readthedocs.io/en/latest/notebooks/trax_intro.html找到:
import trax
# Create a Transformer model.
# Pre-trained model config in gs://trax-ml/models/translation/ende_wmt32k.gin
model = trax.models.Transformer(
input_vocab_size=33300,
d_model=512, d_ff=2048,
n_heads=8, n_encoder_layers=6, n_decoder_layers=6,
max_len=2048, mode='predict')
# Initialize using pre-trained weights.
model.init_from_file('gs://trax-ml/models/translation/ende_wmt32k.pkl.gz',
weights_only=True)
# Tokenize a sentence.
sentence = 'It is nice to learn new things today!'
tokenized = list(trax.data.tokenize(iter([sentence]), # Operates on streams.
vocab_dir='gs://trax-ml/vocabs/',
vocab_file='ende_32k.subword'))[0]
# Decode from the Transformer.
tokenized = tokenized[None, :] # Add batch dimension.
tokenized_translation = trax.supervised.decoding.autoregressive_sample(
model, tokenized, temperature=0.0) # Higher temperature: more diverse results.
# De-tokenize,
tokenized_translation = tokenized_translation[0][:-1] # Remove batch and EOS.
translation = trax.data.detokenize(tokenized_translation,
vocab_dir='gs://trax-ml/vocabs/',
vocab_file='ende_32k.subword')
print(translation)
该示例工作得很好。但是,当我尝试使用初始化模型翻译另一个示例时,例如
sentence = 'I would like to try another example.'
tokenized = list(trax.data.tokenize(iter([sentence]),
vocab_dir='gs://trax-ml/vocabs/',
vocab_file='ende_32k.subword'))[0]
tokenized = tokenized[None, :]
tokenized_translation = trax.supervised.decoding.autoregressive_sample(
model, tokenized, temperature=0.0)
tokenized_translation = tokenized_translation[0][:-1]
translation = trax.data.detokenize(tokenized_translation,
vocab_dir='gs://trax-ml/vocabs/',
vocab_file='ende_32k.subword')
print(translation)
!
我在本地机器和 Google Colab 上都得到了输出。其他示例也是如此。
当我构建和初始化一个新模型时,一切正常。
这是一个错误吗?如果没有,这里发生了什么,我该如何避免/修复这种行为?
标记化和去标记化似乎运作良好,我调试了它。事情似乎出错/出乎意料trax.supervised.decoding.autoregressive_sample
。
解决方案
我自己发现了...需要重置模型的state
. 所以下面的代码对我有用:
def translate(model, sentence, vocab_dir, vocab_file):
empty_state = model.state # save empty state
tokenized_sentence = next(trax.data.tokenize(iter([sentence]), vocab_dir=vocab_dir,
vocab_file=vocab_file))
tokenized_translation = trax.supervised.decoding.autoregressive_sample(
model, tokenized_sentence[None, :], temperature=0.0)[0][:-1]
translation = trax.data.detokenize(tokenized_translation, vocab_dir=vocab_dir,
vocab_file=vocab_file)
model.state = empty_state # reset state
return translation
# Create a Transformer model.
# Pre-trained model config in gs://trax-ml/models/translation/ende_wmt32k.gin
model = trax.models.Transformer(input_vocab_size=33300, d_model=512, d_ff=2048, n_heads=8,
n_encoder_layers=6, n_decoder_layers=6, max_len=2048,
mode='predict')
# Initialize using pre-trained weights.
model.init_from_file('gs://trax-ml/models/translation/ende_wmt32k.pkl.gz',
weights_only=True)
print(translate(model, 'It is nice to learn new things today!',
vocab_dir='gs://trax-ml/vocabs/', vocab_file='ende_32k.subword'))
print(translate(model, 'I would like to try another example.',
vocab_dir='gs://trax-ml/vocabs/', vocab_file='ende_32k.subword'))
推荐阅读
- vscode-settings - 使用自定义键盘快捷键导航 vscode 智能感知
- python - PyQt5 功能问题
- html - 页面调整大小时有很多空白
- javascript - 如何从 Google 日历上的一个事件创建连锁事件?
- sql - 为什么这个 pg 查询这么慢?我怎样才能让它更快?
- vuejs2 - 在 nuxt 布局中使用多个 v-app 时是否有任何问题
- asp.net-core - 从角度 7 传递到我的 Asp.net 核心 WebApi 控制器的参数全部为空
- postgresql - 表 oid 错误。在 postgresql 中选择查询,在 postgres 查询工具消息中返回 table_oid 并显示异常列不存在
- c# - XUnit CollectionFixture 多次实例化
- javascript - “检查后表达式已更改”的控制台错误