python-3.x - 无法在 AWS Elastic Beanstalk 上安装 pycurl==7.43.0.5
问题描述
大家好,我一直在尝试在运行 python 3.7 的 EB 上添加 pycurl,但是当我尝试启动我的 celery 应用程序时,它返回以下错误:
“curl 客户端需要 pycurl 库”
正在执行的命令是:
celery -A sportspot worker -B
[2020-06-23 07:36:14,815: CRITICAL/MainProcess] Unrecoverable error: ImportError('The curl client requires the pycurl library.')
2020-06-23 07:36:16,508 P8649 [INFO] Traceback (most recent call last):
2020-06-23 07:36:16,508 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/http/__init__.py", line 20, in get_client
2020-06-23 07:36:16,508 P8649 [INFO] return hub._current_http_client
2020-06-23 07:36:16,508 P8649 [INFO] AttributeError: 'Hub' object has no attribute '_current_http_client'
2020-06-23 07:36:16,508 P8649 [INFO]
2020-06-23 07:36:16,508 P8649 [INFO] During handling of the above exception, another exception occurred:
2020-06-23 07:36:16,508 P8649 [INFO]
2020-06-23 07:36:16,508 P8649 [INFO] Traceback (most recent call last):
2020-06-23 07:36:16,508 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/worker/worker.py", line 208, in start
2020-06-23 07:36:16,508 P8649 [INFO] self.blueprint.start(self)
2020-06-23 07:36:16,508 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
2020-06-23 07:36:16,508 P8649 [INFO] step.start(parent)
2020-06-23 07:36:16,508 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/bootsteps.py", line 369, in start
2020-06-23 07:36:16,508 P8649 [INFO] return self.obj.start()
2020-06-23 07:36:16,508 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
2020-06-23 07:36:16,509 P8649 [INFO] blueprint.start(self)
2020-06-23 07:36:16,509 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/bootsteps.py", line 119, in start
2020-06-23 07:36:16,509 P8649 [INFO] step.start(parent)
2020-06-23 07:36:16,509 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/worker/consumer/consumer.py", line 599, in start
2020-06-23 07:36:16,509 P8649 [INFO] c.loop(*c.loop_args())
2020-06-23 07:36:16,509 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/celery/worker/loops.py", line 83, in asynloop
2020-06-23 07:36:16,509 P8649 [INFO] next(loop)
2020-06-23 07:36:16,509 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/hub.py", line 301, in create_loop
2020-06-23 07:36:16,509 P8649 [INFO] item()
2020-06-23 07:36:16,509 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/vine/promises.py", line 170, in __call__
2020-06-23 07:36:16,509 P8649 [INFO] return self.throw()
2020-06-23 07:36:16,509 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/vine/promises.py", line 167, in __call__
2020-06-23 07:36:16,509 P8649 [INFO] retval = fun(*final_args, **final_kwargs)
2020-06-23 07:36:16,509 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/transport/SQS.py", line 391, in _schedule_queue
2020-06-23 07:36:16,509 P8649 [INFO] queue, callback=promise(self._loop1, (queue,)),
2020-06-23 07:36:16,509 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/transport/SQS.py", line 407, in _get_bulk_async
2020-06-23 07:36:16,510 P8649 [INFO] return self._get_async(queue, maxcount, callback=callback)
2020-06-23 07:36:16,510 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/transport/SQS.py", line 417, in _get_async
2020-06-23 07:36:16,510 P8649 [INFO] qname, count=count, connection=self.asynsqs(queue=qname),
2020-06-23 07:36:16,510 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/transport/SQS.py", line 571, in asynsqs
2020-06-23 07:36:16,510 P8649 [INFO] region=self.region
2020-06-23 07:36:16,510 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/aws/sqs/connection.py", line 27, in __init__
2020-06-23 07:36:16,510 P8649 [INFO] **kwargs
2020-06-23 07:36:16,510 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/aws/connection.py", line 198, in __init__
2020-06-23 07:36:16,510 P8649 [INFO] **http_client_params)
2020-06-23 07:36:16,510 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/aws/connection.py", line 151, in __init__
2020-06-23 07:36:16,510 P8649 [INFO] self._httpclient = http_client or get_client()
2020-06-23 07:36:16,510 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/http/__init__.py", line 22, in get_client
2020-06-23 07:36:16,510 P8649 [INFO] client = hub._current_http_client = Client(hub, **kwargs)
2020-06-23 07:36:16,510 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/http/__init__.py", line 13, in Client
2020-06-23 07:36:16,510 P8649 [INFO] return CurlClient(hub, **kwargs)
2020-06-23 07:36:16,511 P8649 [INFO] File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/kombu/asynchronous/http/curl.py", line 43, in __init__
2020-06-23 07:36:16,511 P8649 [INFO] raise ImportError('The curl client requires the pycurl library.')
2020-06-23 07:36:16,511 P8649 [INFO] ImportError: The curl client requires the pycurl library.
2020-06-23 07:36:16,511 P8649 [INFO]
2020-06-23 07:36:16,511 P8649 [INFO] -------------- celery@ip-172-31-17-183.ec2.internal v4.4.5 (cliffs)
2020-06-23 07:36:16,511 P8649 [INFO] --- ***** -----
2020-06-23 07:36:16,511 P8649 [INFO] -- ******* ---- Linux-4.14.177-139.254.amzn2.x86_64-x86_64-with-glibc2.2.5 2020-06-23 07:36:14
2020-06-23 07:36:16,511 P8649 [INFO] - *** --- * ---
2020-06-23 07:36:16,511 P8649 [INFO] - ** ---------- [config]
2020-06-23 07:36:16,511 P8649 [INFO] - ** ---------- .> app: sportspot:0x7f1916c1c290
2020-06-23 07:36:16,511 P8649 [INFO] - ** ---------- .> transport: sqs://localhost//
2020-06-23 07:36:16,511 P8649 [INFO] - ** ---------- .> results: disabled://
2020-06-23 07:36:16,511 P8649 [INFO] - *** --- * --- .> concurrency: 1 (prefork)
2020-06-23 07:36:16,511 P8649 [INFO] -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
2020-06-23 07:36:16,511 P8649 [INFO] --- ***** -----
2020-06-23 07:36:16,511 P8649 [INFO] -------------- [queues]
2020-06-23 07:36:16,512 P8649 [INFO] .> celery exchange=celery(direct) key=celery
但是我尝试了不同的解决方法来解决这个问题,但没有运气,其中一些喜欢在我的 requierements.txt 中指定 pycurl 的不同变体,如下所示(当然我一次只部署一个选项,而不是全部):
pycurl==7.43.0.5 #no luck here
pycurl==7.43.0.5 --global-option="--with-nss" #no luck here
pycurl==7.43.0.5 --global-option="--with-openssl" #no luck here
pycurl==7.19.3 #no luck dowgrading
我还尝试从 eb packages 命令安装包,如下所示:
packages:
yum:
libcurl-devel: []
libcurl4-openssl-dev: []
libssl-dev: []
python-pycurl: [7.19.3]
如果我 ssh 进入 eb 实例然后转到 venv 并导入库我得到以下
[ec2-user@ip-172-31-17-183 ~]$ source /var/app/venv/staging-LQM1lest/bin/activate
(staging) [ec2-user@ip-172-31-17-183 ~]$ python3 -c 'import pycurl'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
(staging) [ec2-user@ip-172-31-17-183 ~]$ python -c 'import pycurl'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
(staging) [ec2-user@ip-172-31-17-183 ~]$
同样出于某种原因,eb 没有将虚拟环境存储在经典的 opt/python 路径上,因此它从一开始就使事情变得更加复杂,因为实际上只有 0 个关于它的文档。每个路径都不同,例如迁移或收集静态文件,我必须更改路径才能激活虚拟环境。
即使在亚马逊的文档中,所有教程都使用旧的“选择”路径......
在这一点上,我真的不知道还能尝试什么,因为当我在 localhost 上执行时它可以完美地工作。
我已经设法在没有 celery 的情况下部署它,但执行该部分对我们来说也非常重要。
如果您阅读了整个问题,我非常感谢您花时间分析这个问题!:)
解决方案
通过执行以下操作,我设法为在 python 3.7 上运行的 EB 实例解决了这个问题:
首先我通过 ssh 进入实例,然后运行以下命令:
sudo su
cd /var/app/
source /venv/staging-LQM1lest/bin/activate
(staging): pip uninstall pycurl
(staging): export CPPFLAGS=-I/usr/local/opt/openssl/include
(staging): export LDFLAGS=-L/usr/local/opt/openssl/lib
(staging): pip install pycurl --global-option="--with-openssl"
由于此命令运行良好,因此我将它们转换为 ebconfig 文件上的单个命令,如下所示:
首先从 pycurl 中删除,requirements.txt
然后将以下内容添加到您的 eb 扩展文件中:
container_commands:
.
. some other commands...
.
04_installpycurl:
command: "source /var/app/venv/staging-LQM1lest/bin/activate && export CPPFLAGS=-I/usr/local/opt/openssl/include && export LDFLAGS=-L/usr/local/opt/openssl/lib && pip install pycurl --global-option='--with-openssl'"
请注意,我尝试从 requirements.txt 文件中安装 pycurl --global-option='--with-openssl' 但没有结果,似乎导出这两个变量对安装有影响,我不知道为什么或这些事情做了什么,但解决了问题:)
最后请注意,我的命令首先转到 venv 并激活它,所以如果您的 venv 路径不同,您必须更改它!
推荐阅读
- sql-server - 从同一网络域 PC 从 Access 2013 连接到 SQL Server 2017 时,多步 OLE DB 操作生成错误
- database - 我可以在浏览器中创建数据库吗?除了 localstorage/indexedDB/session 存储
- wordpress - 如何使用 getChildHtml() 从 Magento 获取块并将其插入 Wordpress 标题?
- reactjs - 有没有办法从父组件访问formik的提交函数、值和其他道具
- sql-server - Hibernate 一对多准备好的语句非常慢
- c++ - 最后打印所有类实例方法的打印语句?
- python - 有没有办法说“如果有什么==什么”?(Python)
- audio - FFMPEG:无法从 RTSP 摄像机 IP:V380 Pro 获取音频流
- python - 拆分字符串,中间有 2 个空格
- matlab - 以下代码的矢量化有什么好处吗?