huggingface-transformers - 在 docker 上运行变压器
问题描述
为什么这在 google colab 中有效,但在 docker 上无效?
所以这是我的 Dockerfile。
FROM python:3.7
RUN pip install -q transformers tensorflow
RUN pip install ipython
ENTRYPOINT ["/bin/bash"]
我正在执行这个。
from transformers import *
nlp = pipeline(
'question-answering',
model='mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es',
tokenizer=(
'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es',
{"use_fast": False}
)
)
但我得到这个错误
...:
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 465/465 [00:00<00:00, 325kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 242k/242k [00:00<00:00, 796kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 70.1kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 135/135 [00:00<00:00, 99.6kB/s]
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
/usr/local/lib/python3.7/site-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
461 if resolved_archive_file is None:
--> 462 raise EnvironmentError
463 except EnvironmentError:
OSError:
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-1-1f9fed95967a> in <module>
5 tokenizer=(
6 'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es',
----> 7 {"use_fast": False}
8 )
9 )
/usr/local/lib/python3.7/site-packages/transformers/pipelines.py in pipeline(task, model, config, tokenizer, framework, **kwargs)
1882 "Trying to load the model with Tensorflow."
1883 )
-> 1884 model = model_class.from_pretrained(model, config=config, **model_kwargs)
1885
1886 return task_class(model=model, tokenizer=tokenizer, modelcard=modelcard, framework=framework, task=task, **kwargs)
/usr/local/lib/python3.7/site-packages/transformers/modeling_tf_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1207 for config_class, model_class in TF_MODEL_FOR_QUESTION_ANSWERING_MAPPING.items():
1208 if isinstance(config, config_class):
-> 1209 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
1210 raise ValueError(
1211 "Unrecognized configuration class {} for this kind of TFAutoModel: {}.\n"
/usr/local/lib/python3.7/site-packages/transformers/modeling_tf_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
467 f"- or '{pretrained_model_name_or_path}' is the correct path to a directory containing a file named one of {TF2_WEIGHTS_NAME}, {WEIGHTS_NAME}.\n\n"
468 )
--> 469 raise EnvironmentError(msg)
470 if resolved_archive_file == archive_file:
471 logger.info("loading weights file {}".format(archive_file))
OSError: Can't load weights for 'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es'. Make sure that:
- 'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es' is a correct model identifier listed on 'https://huggingface.co/models'
- or 'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.
然而,这在google colab中完美运行。这个 Google Colab 不需要运行 GPU,那么为什么它不能在 docker 中运行呢?我可能会缺少哪些依赖项?它没有在错误消息中看到依赖项可能丢失,不仅仅是模型不存在,而是看:是的,这个模型存在"mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es"
于 hugging.co
解决方案
推荐阅读
- sql - 根据字段值限制查询结果
- javascript - 使用 javascript 在 mysql 上进行更新
- flutter - Flutter StreamBuilder在Firestore的子集合中查询文档的子集合
- mongodb - 当键为数字时,子文档中的 Mongo 查询
- android - Flutter 是否适用于任何 Android 平台版本?
- linux - 从 arm32v7 golang 映像构建 docker 映像时出现 exec 格式错误
- keras - 分层 KFold 交叉验证 (Keras) ValueError:找到暗淡为 4 的数组。预计估计器 <= 2
- c++ - 不属于任何类型的结构中的成员?和一个结构来保存一个单独的变量
- sql - 将 SQL 服务器查询转换为 SQLite
- .net - IdentityServer4 总是返回“错误”:“invalid_scope”