tensorflow - 使用拥抱脸变压器运行 trainer.train() 失败

Question

我正在尝试为问答项目设置 TensorFlow 微调框架。以拥抱脸/变形金刚为原型，但无法通过训练器运行。

实验在 Databricks 进行，加载的预训练模型是 base-bert，训练和开发集是从拥抱人脸示例 SQUAD 2.0 https://github.com/huggingface/transformers/tree/master/examples/question下载的- 回答

错误日志抱怨意外的关键字参数“ is_impossible ”，这是 SQUAD 2 数据格式功能。

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
model = TFAutoModelForQuestionAnswering.from_pretrained('bert-base-uncased')

processor = SquadV2Processor()

train_examples = processor.get_train_examples(data_dir, train_file_name)
eval_examples = processor.get_dev_examples(data_dir, dev_file_name)

train_dataset = (squad_convert_examples_to_features(
            examples=train_examples,
            tokenizer=tokenizer,
            max_seq_length=max_seq_length,
            doc_stride=doc_stride,
            max_query_length=max_query_length,
            is_training=True,
            return_dataset="tf"
        )) 

eval_dataset = (squad_convert_examples_to_features(
            examples=eval_examples,
            tokenizer=tokenizer,
            max_seq_length=max_seq_length,
            doc_stride=doc_stride,
            max_query_length=max_query_length,
            is_training=False,
            return_dataset="tf"
        )) 

training_args = (TFTrainingArguments(
    output_dir=output_dir, 
    num_train_epochs=2,
    do_train=True, 
    per_device_train_batch_size = 8, 
    per_device_eval_batch_size = 16,
    logging_steps=10, 
    learning_rate=3e-5))

trainer = TFTrainer(model=model, args=training_args, train_dataset=train_dataset, eval_dataset=eval_dataset)

trainer.train()

错误日志如下：

---

----> 1 trainer.train() 中的 TypeError Traceback (last last call last)

/databricks/python/lib/python3.7/site-packages/transformers/trainer_tf.py in train(self) 410 if self.args.past_index >= 0: 411 self._past = None --> 412 for step, training_loss in enumerate(self._training_steps(train_ds, optimizer)): 413 self.global_step = iterations.numpy() 414 self.epoch_logging = epoch_iter - 1 + (step + 1) / steps_per_epoch

/databricks/python/lib/python3.7/site-packages/transformers/trainer_tf.py in _training_steps(self, ds, optimizer) 457 在训练步骤上返回生成器（即参数更新）。458 """ --> 459 for i, loss in enumerate(self._accumulate_next_gradients(ds)): 460 if i % self.args.gradient_accumulation_steps == 0: 461 self._apply_gradients(optimizer)

/databricks/python/lib/python3.7/site-packages/transformers/trainer_tf.py in _accumulate_next_gradients(self, ds) 490 while True: 491 try: --> 492 yield _accumulate_next() 493 tf.errors.OutOfRangeError 除外： 494 休息

/local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in call (self, *args, **kwds) 566 xla_context .Exit() 567 else: --> 568 result = self._call(*args, **kwds) 569 570 if tracking_count == self._get_tracing_count():

/local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in _call(self, *args, **kwds) 613 #这是 call 的第一次调用，所以我们必须初始化。614 initializers = [] --> 615 self._initialize(args, kwds, add_initializers_to=initializers) 616 finally: 617 #此时我们知道初始化完成（或更少

/local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to) 495 self. _concrete_stateful_fn = ( 496 self._stateful_fn._get_concrete_function_internal_garbage_collected(

pylint：禁用=受保护的访问

--> 497 *args, **kwds)) 498 499 def invalid_creator_scope(*unused_args, **unused_kwds):

/local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs) 2387 args , kwargs = None, None 2388 with self._lock: -> 2389 graph_function, _, _ = self._maybe_define_function(args, kwargs) 2390 return graph_function 2391

/local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py in _maybe_define_function(self, args, kwargs) 2701 2702 self._function_cache .missed.add(call_context_key) -> 2703 graph_function = self._create_graph_function(args, kwargs) 2704 self._function_cache.primary[cache_key] = graph_function 2705 return graph_function, args, kwargs

/local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes) 2591 arg_names= arg_names, 2592 override_flat_arg_shapes=override_flat_arg_shapes, -> 2593 capture_by_value=self._capture_by_value), 2594 self._function_attributes, 2595 # 告诉 ConcreteFunction 一旦它退出就清理它的图

/local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py in func_graph_from_py_func（名称，python_func，args，kwargs，签名，func_graph , autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes) 976 convert_func) 977 --> 978 func_outputs = python_func(*func_args, **func_kwargs) 979 980 # invariant:func_outputs只包含张量，复合张量，

/local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py in Wrapped_fn(*args, **kwds) 437 # Wrapped允许AutoGraph 以交换转换后的函数。我们给 438 # 函数一个对自身的弱引用以避免引用循环。--> 439 返回weak_wrapped_fn()。包装（*args，**kwds）440 weak_wrapped_fn = weakref.ref（wrapped_fn）441

/local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py in wrapper(*args, **kwargs) 966 例外为e: # pylint:disable=broad-except 967 if hasattr(e, "ag_error_metadata"): --> 968 raise e.ag_error_metadata.to_exception(e) 969 else: 970 raise

TypeError：在转换后的代码中：

/databricks/python/lib/python3.7/site-packages/transformers/trainer_tf.py:488

_accumulate_next * return self._accumulate_gradients(per_replica_features, per_replica_labels) /databricks/python/lib/python3.7/site-packages/transformers/trainer_tf.py:498 _accumulate_gradients * per_replica_loss = self.args.strategy.experimental_run_v2( /local_disk0/pythonVirtualEnvDirs/ virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/distribute/one_device_strategy.py:180experimental_run_v2 return super(OneDeviceStrategy, self).experimental_run_v2(fn, args, kwargs) /databricks/python/lib/python3.7/site-packages/transformers/trainer_tf.py:511 _forward * per_example_loss, _ = self._run_model(features, labels, True) /databricks/python/lib/python3.7/site -packages/transformers/trainer_tf.py:532 _run_model * 输出 = self.model(features, training=training, **labels)[:2] /local_disk0/pythonVirtualEnvDirs/virtualEnv-f56565e5-e45b-447a-b7df-50daf9109495/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py:778 调用 输出 = call_fn(cast_inputs, *args, **kwargs)

TypeError: tf__call() got an unexpected keyword argument 'is_impossible'

标签： tensorflowbert-language-modelhuggingface-transformerssquad