首页 > 解决方案 > 在微调变压器中无法获取锁(缓存文件)错误

问题描述

我正在使用 jupyter 微调 T5 变压器。我尝试运行从 github 获取的代码,但是当我尝试训练模型时遇到问题,这是一段代码model = T5FineTuner(args) ,但出现此错误

ValueError: Cannot acquire lock, caching file might be used by another process, you should setup a unique 'experiment_id' for this run.

错误的详细信息如下

---------------------------------------------------------------------------
Timeout                                   Traceback (most recent call last)
~\anaconda3\lib\site-packages\nlp\metric.py in __init__(self, name, experiment_id,                     
process_id, num_process, data_dir, in_memory, hash, seed, **kwargs)
127         try:
--> 128             self.filelock.acquire(timeout=1)
129         except Timeout:

~\anaconda3\lib\site-packages\filelock.py in acquire(self, timeout, poll_intervall)
277                     logger().debug('Timeout on acquiring lock %s on %s', lock_id, 
lock_filename)
--> 278                     raise Timeout(self._lock_file)
279                 else:

Timeout: The file lock 
  'C:\Users\Dhuha\.cache\huggingface\metrics\rouge\default\1.0.0\06783dbed5f6b6a5413f84d2a5f0d9dc9cb871f1aeb3787f2c90a8e3fe60b1c1\cache-rouge-0.arrow.lock' could not be acquired.

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-30-10e7c6aa1f29> in <module>
----> 1 model = T5FineTuner(args)

<ipython-input-21-e14a03cde08f> in __init__(self, hparams)
  5         self.model = 
T5ForConditionalGeneration.from_pretrained(hparams.model_name_or_path)
  6         self.tokenizer = 
T5Tokenizer.from_pretrained(hparams.tokenizer_name_or_path)
----> 7         self.rouge_metric = load_metric('rouge')
  8 
  9         if self.hparams.freeze_embeds:

 ~\anaconda3\lib\site-packages\nlp\load.py in load_metric(path, name, process_id, 
num_process, data_dir, experiment_id, in_memory, download_config, **metric_init_kwargs)
  442     module_path, hash = prepare_module(path, download_config=download_config, 
 dataset=False)
443     metric_cls = import_main_class(module_path, dataset=False)
--> 444     metric = metric_cls(
445         name=name,
446         hash=hash,

 ~\anaconda3\lib\site-packages\nlp\metric.py in __init__(self, name, experiment_id, process_id, num_process, data_dir, in_memory, hash, seed, **kwargs)
128             self.filelock.acquire(timeout=1)
129         except Timeout:
--> 130             raise ValueError(
131                 "Cannot acquire lock, caching file might be used by another 
process, "
132                 "you should setup a unique 'experiment_id' for this run."

ValueError: Cannot acquire lock, caching file might be used by another process, you 
should setup a unique 'experiment_id' for this run.

我是初学者,所以在我第一次尝试运行fine_tune 模型时尝试帮助我。
谢谢

标签: pythonmodeljupytertransformerfine-tune

解决方案


推荐阅读