python - 无法从 Amazon EC2 中的私有 pypi 服务器获取包
问题描述
出于几个原因,我们决定在 Amazon S3 上托管我们所有的私有和公共 Python 依赖项(及其依赖项)。我们打算仅从 S3 下载/安装软件包,而不是其他任何地方。
我按照https://stackoverflow.com/a/57552988/3007402中提到的步骤(我写了答案)在 S3 上设置 pypi 服务器。
要将公共包上传到 S3,我将首先使用
pip download numpy==1.14.2
pip download statsmodels==0.6.1
要安装我会使用的任何软件包
pip install pandas --index-url=http://<s3_endpoint> --trusted-host=<s3_endpoint> --no-cache-dir
.whl
对于作为文件下载的包,一切正常。这样的包(例如pandas
)能够毫无问题地安装它们自己和它们的依赖项(numpy
如果是)。pandas
问题在于非 whl 软件包,例如statsmodels-0.6.1.tar.gz
. Whilepip
用于安装statsmodels
,安装依赖项,statsmodels
使用easy_install
.
pip arg--index-url
不被使用easy_install
,它会numpy
从 pypi.org 下载依赖项。
为了解决这个问题(仅从 S3 下载),我提取statsmodels-0.6.1.tar.gz
、编辑setup.cfg
、重新打包并上传到 S3。以下是 的内容setup.cfg
:
[egg_info]
tag_build =
tag_date = 0
tag_svn_revision = 0
# lines below are added by me
[easy_install]
index_url = http://<s3_link>
find_links = http://<s3_link>
通过该更改从 S3statsmodels
获取依赖项并成功安装它。numpy
出于某种奇怪的原因,这仅适用于 Ubuntu(本地和运行 Ubuntu 的 EC2),但在运行 Amazon Linux 的 EC2 上失败。下面是我使用--log <file>
pip 的参数保存的日志。为简洁起见,我删除了时间戳。
Created temporary directory: /tmp/pip-ephem-wheel-cache-7SD5Bu
Created temporary directory: /tmp/pip-req-tracker-du4AEi
Created requirements tracker '/tmp/pip-req-tracker-du4AEi'
Created temporary directory: /tmp/pip-install-G2qw36
Looking in indexes: http://<s3_link>
Collecting statsmodels
1 location(s) to search for versions of statsmodels:
* http://<s3_link>/statsmodels/
Getting page http://<s3_link>/statsmodels/
Found index url http://<s3_link>
Analyzing links from page http://<s3_link>/statsmodels/
Found link http://<s3_link>/statsmodels/statsmodels-0.6.1.tar.gz (from http://<s3_link>/statsmodels/), version: 0.6.1
Given no hashes to check 1 links for project 'statsmodels': discarding no candidates
Using version 0.6.1 (newest of versions: 0.6.1)
Created temporary directory: /tmp/pip-unpack-r8lKU4
Found index url http://<s3_link>
Downloading http://<s3_link>/statsmodels/statsmodels-0.6.1.tar.gz (7.1MB)
Downloading from URL http://<s3_link>/statsmodels/statsmodels-0.6.1.tar.gz (from http://<s3_link>/statsmodels/)
Added statsmodels from http://<s3_link>/statsmodels/statsmodels-0.6.1.tar.gz to build tracker '/tmp/pip-req-tracker-du4AEi'
Running setup.py (path:/tmp/pip-install-G2qw36/statsmodels/setup.py) egg_info for package statsmodels
Running command python setup.py egg_info
No local packages or download links found for numpy
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-G2qw36/statsmodels/setup.py", line 449, in <module>
**setuptools_kwargs)
File "/usr/lib64/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/home/ec2-user/tempenv/local/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "/home/ec2-user/tempenv/local/lib/python2.7/site-packages/setuptools/dist.py", line 311, in fetch_build_eggs
replace_conflicting=True,
File "/home/ec2-user/tempenv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 797, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/home/ec2-user/tempenv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1047, in best_match
return self.obtain(req, installer)
File "/home/ec2-user/tempenv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1059, in obtain
return installer(requirement)
File "/home/ec2-user/tempenv/local/lib/python2.7/site-packages/setuptools/dist.py", line 378, in fetch_build_egg
return cmd.easy_install(req)
File "/home/ec2-user/tempenv/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 617, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('numpy')
cat /etc/os-release
(Amazon Linux 详细信息)的输出:
NAME="Amazon Linux AMI"
VERSION="2017.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2017.03"
PRETTY_NAME="Amazon Linux AMI 2017.03"
解决方案
显然,运行 Amazon Linux 的 EC2 具有旧版本的setuptools
.
我升级到最新版本,我的安装很顺利。
推荐阅读
- android - 如何使用 Kotlin 使用嵌套 JSON 从一个 RecyclerView 项发送数据以加载另一个 RecyclerView
- sql - ORACLE SQL:如何按列对数据进行分组和计数?
- javascript - 一种在 javascript 中获取规范名称的方法
- java - 请求正文正在控制台中打印,但不是响应
- powershell - 如何在使用 powershell 脚本导入 CSV 后保留空格?
- c++ - OpenCV 4.5.1 与 3.4.2 的性能差异
- r - 组合来自两个不同列的属性以创建单个属性
- mysql - MySQL 查询给了我不正确或旧的数据
- java - 字符串字段的 jpametamodel null SingularAttribute
- javascript - 模拟 util 函数导入