python - pip install wheel 随机失败
问题描述
我有一个 python 项目,我在本地打包为一个轮子,我想在一个基于 python:alpine3.7 的 docker 容器中的虚拟环境中安装这个轮子。我的问题是,有时我工作,有时它无法解决轮子中的依赖关系,似乎是随机的。
轮子的依赖项是:
_install_requires = [
'numpy>=1.15.0',
'pandas>=0.21.0',
'scipy>=1.0.0',
'scikit-learn>=0.19.2'
]
我的dockerfile如下:
FROM python:alpine3.7
# install dependencies
# the lapack package is only in the community repository
RUN echo "http://dl-4.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --update add --no-cache --virtual .build-deps \
gcc \
freetype-dev
RUN apk add --update lapack-dev python py-pip python-dev &&\
apk add --update linux-headers
# Install dependencies
RUN apk add --no-cache --virtual .build-deps \
gfortran \
musl-dev \
g++ \
make &&\
ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install --upgrade pip && pip install virtualenv
RUN virtualenv -p python3 ./venv
ADD dist/project-as-wheel-py3-none-any.whl .
RUN . ./venv/bin/activate && pip install ./project-as-wheel-py3-none-any.whl
在构建 docker 容器时,有时会失败,典型的日志是:
Processing /project-as-wheel-py3-none-any.whl
Collecting scikit-learn>=0.19.2 (from project-as-wheel==0.2.6)
Downloading https://files.pythonhosted.org/packages/25/b6/454cf208be93efa3db50ce06b732328c57ede005d1dcfa71d9a1548530b0/scikit-learn-0.19.2.tar.gz (9.7MB)
Collecting pandas>=0.21.0 (from project-as-wheel==0.2.6)
Downloading https://files.pythonhosted.org/packages/3e/56/82c4d4c049294f87ebd05b65fdcbc9ed68bd23fb0a7e4469caf9a75d199f/pandas-0.23.3.tar.gz (10.5MB)
Collecting numpy>=1.15.0 (from project-as-wheel==0.2.6)
Downloading https://files.pythonhosted.org/packages/3a/20/c81632328b1a4e1db65f45c0a1350a9c5341fd4bbb8ea66cdd98da56fe2e/numpy-1.15.0.zip (4.5MB)
Collecting scipy>=1.0.0 (from project-as-wheel==0.2.6)
Downloading https://files.pythonhosted.org/packages/07/76/7e844757b9f3bf5ab9f951ccd3e4a8eed91ab8720b0aac8c2adcc2fdae9f/scipy-1.1.0.tar.gz (15.6MB)
Collecting python-dateutil>=2.5.0 (from pandas>=0.21.0->project-as-wheel==0.2.6)
Downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kB)
Collecting pytz>=2011k (from pandas>=0.21.0->project-as-wheel==0.2.6)
Downloading https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl (510kB)
Collecting six>=1.5 (from python-dateutil>=2.5.0->pandas>=0.21.0->project-as-wheel==0.2.6)
Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Building wheels for collected packages: scikit-learn, pandas, numpy, scipy
Running setup.py bdist_wheel for scikit-learn: started
Running setup.py bdist_wheel for scikit-learn: finished with status 'error'
Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scikit-learn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-ogmdp3dv --python-tag cp37:
Partial import of sklearn during the build process.
Traceback (most recent call last):
File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 168, in get_numpy_status
import numpy
ModuleNotFoundError: No module named 'numpy'
Traceback (most recent call last):
File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 148, in get_scipy_status
import scipy
ModuleNotFoundError: No module named 'scipy'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 269, in <module>
setup_package()
File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 249, in setup_package
.format(numpy_req_str, instructions))
ImportError: Numerical Python (NumPy) is not installed.
scikit-learn requires NumPy >= 1.8.2.
Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html
----------------------------------------
Failed building wheel for scikit-learn
Running setup.py clean for scikit-learn
Running setup.py bdist_wheel for pandas: started
Running setup.py bdist_wheel for pandas: still running...
Running setup.py bdist_wheel for pandas: still running...
Running setup.py bdist_wheel for pandas: still running...
Running setup.py bdist_wheel for pandas: still running...
Running setup.py bdist_wheel for pandas: still running...
Running setup.py bdist_wheel for pandas: still running...
Running setup.py bdist_wheel for pandas: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/8d/e7/58/7a882c73419e95e6709d925abb4b56c719c9be59ef826fe01c
Running setup.py bdist_wheel for numpy: started
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/77/d2/00/03180c22fc40b6f17f6a5c6ea0fa1689734a9cf4cf54179a1c
Running setup.py bdist_wheel for scipy: started
Running setup.py bdist_wheel for scipy: finished with status 'error'
Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-83ktidr8 --python-tag cp37:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-439jjmrz/scipy/setup.py", line 474, in <module>
setup_package()
File "/tmp/pip-install-439jjmrz/scipy/setup.py", line 450, in setup_package
from numpy.distutils.core import setup
ModuleNotFoundError: No module named 'numpy'
----------------------------------------
Failed building wheel for scipy
Running setup.py clean for scipy
Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all:
`setup.py clean` is not supported, use one of the following instead:
- `git clean -xdf` (cleans all files)
- `git clean -Xdf` (cleans all versioned files, doesn't touch
files that aren't checked into the git repo)
Add `--force` to your command to use it anyway if you must (unsupported).
----------------------------------------
Failed cleaning build dir for scipy
Successfully built pandas numpy
Failed to build scikit-learn scipy
Installing collected packages: scikit-learn, six, python-dateutil, pytz, numpy, pandas, scipy, project-as-wheel
Running setup.py install for scikit-learn: started
Running setup.py install for scikit-learn: finished with status 'error'
Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scikit-learn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-dwajrs1d/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python3.7/scikit-learn:
Partial import of sklearn during the build process.
Traceback (most recent call last):
File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 168, in get_numpy_status
import numpy
ModuleNotFoundError: No module named 'numpy'
Traceback (most recent call last):
File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 148, in get_scipy_status
import scipy
ModuleNotFoundError: No module named 'scipy'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 269, in <module>
setup_package()
File "/tmp/pip-install-439jjmrz/scikit-learn/setup.py", line 249, in setup_package
.format(numpy_req_str, instructions))
ImportError: Numerical Python (NumPy) is not installed.
scikit-learn requires NumPy >= 1.8.2.
Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html
----------------------------------------
Command "/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-439jjmrz/scikit-learn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-dwajrs1d/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python3.7/scikit-learn" failed with error code 1 in /tmp/pip-install-439jjmrz/scikit-learn/
让我感到困惑的是 pip 设法安装 numpy 和 pandas,然后无法安装 scipy 声称“没有名为 'numpy' 的模块”。
编辑:日志上的 TL;DR 是:
- pip 尝试安装 scikit-learn,由于未安装 numpy 而失败。
- pip 尝试安装 pandas,成功。
- pip 尝试安装 numpy,成功。
- pip 尝试安装 scipy,由于未安装 numpy 而失败(但是在上面的步骤 3 中已成功安装!)。
- pip 声明它成功安装了 pandas 和 numpy,但未能安装 scikit-learn 和 scipy。
- pip 再次尝试安装 scipy,再次失败,抱怨未安装 numpy。
- 点放弃。
Edit2:根据 Nuts 的建议,我将 dockerfile 中的相关行更改为
RUN . ./venv/bin/activate && pip install numpy && pip install scipy && pip install ./project-as-wheel-py3-none-any.whl
不太令人满意,因为我无法从 setup.py 中获取版本要求,但它可以工作。
解决方案
将要求更改为此
_install_requires = [
'numpy>= 1.8.2',
'pandas>=0.21.0',
'scipy>=1.0.0',
'scikit-learn>=0.19.2'
]
推荐阅读
- python-3.x - 如何监控 EC2 中托管的 API(Flask)
- r - 删除重复行并对数据框中最后一列的相应值求和
- python - 在图像中的一组点周围寻找不同的边界
- entity-framework - 使用附加或单一的实体框架更新
- django - html问题中的Django循环
- python - 每次函数什么都不返回时,这是编写 -> None 的好习惯吗?
- javascript - 传单图例添加和删除到连接的 layerGroup
- hyperledger-fabric - fabric-ca-client 找不到 ./utils.js
- c++ - 我可以从 C++ 中的某一行开始代码吗?
- javascript - 理解 java/apex 复杂的三元运算符