时间戳

首页 > 解决方案 > “cdqa”的问题：无法从“joblib._memmapping_reducer”导入名称“TemporaryResourcesManager”

python - “cdqa”的问题：无法从“joblib._memmapping_reducer”导入名称“TemporaryResourcesManager”

问题描述

我正在尝试使用 cdqa 库（https://github.com/cdqa-suite/cdQA）。我使用“git clone”克隆了 GitHub 存储库，并在“requirements.txt”文件中为我的版本更改了 pandas 和 pytorch 的指定版本（因为我在安装这两个库请求的特定版本时遇到问题，并且根据“在那个 github 存储库中，有几个人设法通过更改“requirements.txt”中的版本来安装 cdqa）。然后，我使用了：

cd cdQA
pip install -e .

现在，我正在运行该库的 GitHub 存储库中的示例：

import pandas as pd
from ast import literal_eval

from cdqa.utils.filters import filter_paragraphs
from cdqa.utils.download import download_model, download_bnpp_data
from cdqa.pipeline.cdqa_sklearn import QAPipeline
#from cdqa.pipeline import QAPipeline

# Download data and models
download_bnpp_data(dir='./data')
download_model(model='bert-squad_1.1', dir='./Models')

# Loading data and filtering / preprocessing the documents
df = pd.read_csv('data/bnpp_newsroom-v1.1.csv', converters={'paragraphs': literal_eval})
df = filter_paragraphs(df)

# Loading QAPipeline with CPU version of BERT Reader pretrained on SQuAD 1.1
cdqa_pipeline = QAPipeline(reader='Models/bert_qa.joblib')

但它返回以下错误：

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-29-596fa5ffc73b> in <module>
      4 from cdqa.utils.filters import filter_paragraphs
      5 from cdqa.utils.download import download_model, download_bnpp_data
----> 6 from cdqa.pipeline.cdqa_sklearn import QAPipeline
      7 #from cdqa.pipeline import QAPipeline
      8 

~/Documents/GIT/QA/cdQA/cdqa/pipeline/__init__.py in <module>
----> 1 from .cdqa_sklearn import QAPipeline
      2 
      3 __all__ = ["QAPipeline"]

~/Documentos/Morning_Star_Consulting/GIT/QA/cdQA/cdqa/pipeline/cdqa_sklearn.py in <module>
----> 1 import joblib
      2 import warnings
      3 
      4 import pandas as pd
      5 import numpy as np

~/anaconda3/lib/python3.8/site-packages/joblib/__init__.py in <module>
    118 from .numpy_pickle import load
    119 from .compressor import register_compressor
--> 120 from .parallel import Parallel
    121 from .parallel import delayed
    122 from .parallel import cpu_count

~/anaconda3/lib/python3.8/site-packages/joblib/parallel.py in <module>
     24 from .logger import Logger, short_format_time
     25 from .disk import memstr_to_bytes
---> 26 from ._parallel_backends import (FallbackToBackend, MultiprocessingBackend,
     27                                  ThreadingBackend, SequentialBackend,
     28                                  LokyBackend)

~/anaconda3/lib/python3.8/site-packages/joblib/_parallel_backends.py in <module>
     17     from .pool import MemmappingPool
     18     from multiprocessing.pool import ThreadPool
---> 19     from .executor import get_memmapping_executor
     20 
     21     # Compat between concurrent.futures and multiprocessing TimeoutError

~/anaconda3/lib/python3.8/site-packages/joblib/executor.py in <module>
     10 
     11 from ._memmapping_reducer import get_memmapping_reducers
---> 12 from ._memmapping_reducer import TemporaryResourcesManager
     13 from .externals.loky.reusable_executor import _ReusablePoolExecutor
     14 

ImportError: cannot import name 'TemporaryResourcesManager' from 'joblib._memmapping_reducer' (/home/user/anaconda3/lib/python3.8/site-packages/joblib/_memmapping_reducer.py)

我怎样才能修复那个我认为与joblib有关的错误？非常感谢。

标签： pythonnlpjoblib

解决方案

确保您的 python 版本 >=3.8.0

修改这个文件

python3.8/site-packages/joblib/externals/cloudpickle/cloudpickle.py

第 135 行

改成

co.co_kwonlyargcount,

0,

co.co_nlocals，

推荐阅读