python - 运行BERT模型后如何解决属性错误
问题描述
运行 BERT 模型后收到错误。至此,代码运行成功。我收到的错误是 AttributeError:“str”对象没有属性“shape”。代码之前的上一步是创建自定义数据生成器。使用它创建了模型。为了提供上下文,我使用的模型可以在网站https://keras.io/examples/nlp/semantic_similarity_with_bert/上找到,我用它来解释我自己的数据。
from ipywidgets import IntProgress
strategy = tf.distribute.MirroredStrategy()
with strategy.scope():
# Encoded token ids from BERT tokenizer.
input_ids = tf.keras.layers.Input(
shape=(max_length,), dtype=tf.int32, name="input_ids"
)
# Attention masks indicates to the model which tokens should be attended to.
attention_masks = tf.keras.layers.Input(
shape=(max_length,), dtype=tf.int32, name="attention_masks"
)
# Token type ids are binary masks identifying different sequences in the model.
token_type_ids = tf.keras.layers.Input(
shape=(max_length,), dtype=tf.int32, name="token_type_ids"
)
# Loading pretrained BERT model.
bert_model = transformers.TFBertModel.from_pretrained("bert-base-uncased")
# Freeze the BERT model to reuse the pretrained features without modifying them.
bert_model.trainable = False
sequence_output, pooled_output = bert_model(
input_ids, attention_mask=attention_masks, token_type_ids=token_type_ids
)
# Add trainable layers on top of frozen layers to adapt the pretrained features on the new data.
bi_lstm = tf.keras.layers.Bidirectional(
tf.keras.layers.LSTM(64, return_sequences=True)
)(sequence_output)
# Applying hybrid pooling approach to bi_lstm sequence output.
avg_pool = tf.keras.layers.GlobalAveragePooling1D()(bi_lstm)
max_pool = tf.keras.layers.GlobalMaxPooling1D()(bi_lstm)
concat = tf.keras.layers.concatenate([avg_pool, max_pool])
dropout = tf.keras.layers.Dropout(0.3)(concat)
output = tf.keras.layers.Dense(3, activation="softmax")(dropout)
model = tf.keras.models.Model(
inputs=[input_ids, attention_masks, token_type_ids], outputs=output
)
model.compile(
optimizer=tf.keras.optimizers.Adam(),
loss="categorical_crossentropy",
metrics=["acc"],
)
print(f"Strategy: {strategy}")
model.summary()
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-35-e6d50369bfa4> in <module>
25 )
26 # Add trainable layers on top of frozen layers to adapt the pretrained features on the new data.
---> 27 bi_lstm = tf.keras.layers.Bidirectional(
28 tf.keras.layers.LSTM(64, return_sequences=True)
29 )(sequence_output)
~\anaconda3\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\layers\wrappers.py in __call__(self, inputs, initial_state, constants, **kwargs)
528
529 if initial_state is None and constants is None:
--> 530 return super(Bidirectional, self).__call__(inputs, **kwargs)
531
532 # Applies the same workaround as in `RNN.__call__`
~\anaconda3\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in __call__(self, *args, **kwargs)
980 with ops.name_scope_v2(name_scope):
981 if not self.built:
--> 982 self._maybe_build(inputs)
983
984 with ops.enable_auto_cast_variables(self._compute_dtype_object):
~\anaconda3\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in _maybe_build(self, inputs)
2615 # Check input assumptions set before layer building, e.g. input rank.
2616 if not self.built:
-> 2617 input_spec.assert_input_compatibility(
2618 self.input_spec, inputs, self.name)
2619 input_list = nest.flatten(inputs)
~\anaconda3\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
164 spec.min_ndim is not None or
165 spec.max_ndim is not None):
--> 166 if x.shape.ndims is None:
167 raise ValueError('Input ' + str(input_index) + ' of layer ' +
168 layer_name + ' is incompatible with the layer: '
AttributeError: 'str' object has no attribute 'shape'
解决方案
模型的输入不能是字符串。它应该是一个 numpy 数组或张量。您应该对字符串进行编码,将字符转换为 numpy 数组。
从https://keras.io/examples/nlp/semantic_similarity_with_bert/也有用于标记化的代码示例:
self.tokenizer = transformers.BertTokenizer.from_pretrained(
"bert-base-uncased", do_lower_case=True
)
# With BERT tokenizer's batch_encode_plus batch of both the sentences are
# encoded together and separated by [SEP] token.
encoded = self.tokenizer.batch_encode_plus(
sentence_pairs.tolist(),
add_special_tokens=True,
max_length=max_length,
return_attention_mask=True,
return_token_type_ids=True,
pad_to_max_length=True,
return_tensors="tf",
)
# Convert batch of encoded features to numpy array.
input_ids = np.array(encoded["input_ids"], dtype="int32")
attention_masks = np.array(encoded["attention_mask"], dtype="int32")
token_type_ids = np.array(encoded["token_type_ids"], dtype="int32")
推荐阅读
- python - 我对 tkinter 按钮命令功能有问题
- python - 如何对没有 predict_proba 或 decision_function 的模型使用 CalibratedClassifierCV
- laravel - 发生异常。FormatException (FormatException: 意外字符 (在字符 1) ^ )
- java - 以下程序的输出将是什么?5.3f和5.3有什么区别
- prepared-statement - 在芭蕾舞女演员中准备好的陈述
- optimization - 如何提高z3的优化速度?
- vue.js - 在 Vue 模板中使用三元
- c# - 通过单击浏览器返回阻止用户进入支付网关
- javascript - 用 ObjectLoader 加载的三个.js 对象无法更改不透明度
- javascript - 更改背景跟随操作表单