celery - 气流工作人员未达到 sqs

Question

我正在运行气流工作者服务。该服务无法连接到 sqs

调度程序能够到达并写入队列

环境：Amazon Linux Python 3.54 Airflow 1.10.1 Celery 4.1.1

代理很好；我已经在 python 2.7 和 3.5 相同的问题中实现了这个

我已经为区域设置了 celery_transport_options 气流配置有 celeryExecutor、SQS 代理和 postgres 数据库作为后端

部分日志

[2018-08-30 15:43:58,779: CRITICAL/MainProcess] Unrecoverable error: Exception('Request Empty body HTTP 599 Failed to connect to eu-west-1.queue.amazonaws.com port 443: Connection timed out (None)',)
Traceback (most recent call last):
 File "/usr/local/lib/python3.5/site-packages/celery/worker/worker.py", line 207, in start
 self.blueprint.start(self)
 File "/usr/local/lib/python3.5/site-packages/celery/bootsteps.py", line 119, in start
 step.start(parent)
 File "/usr/local/lib/python3.5/site-packages/celery/bootsteps.py", line 370, in start
 return self.obj.start()
 File "/usr/local/lib/python3.5/site-packages/celery/worker/consumer/consumer.py", line 316, in start
 blueprint.start(self)
 File "/usr/local/lib/python3.5/site-packages/celery/bootsteps.py", line 119, in start
 step.start(parent)
 File "/usr/local/lib/python3.5/site-packages/celery/worker/consumer/consumer.py", line 592, in start
 c.loop(*c.loop_args())
 File "/usr/local/lib/python3.5/site-packages/celery/worker/loops.py", line 91, in asynloop
 next(loop)
 File "/usr/local/lib/python3.5/site-packages/kombu/asynchronous/hub.py", line 354, in create_loop
 cb(*cbargs)
 File "/usr/local/lib/python3.5/site-packages/kombu/asynchronous/http/curl.py", line 114, in on_writable
 return self._on_event(fd, _pycurl.CSELECT_OUT)
 File "/usr/local/lib/python3.5/site-packages/kombu/asynchronous/http/curl.py", line 124, in _on_event
 self._process_pending_requests()
 File "/usr/local/lib/python3.5/site-packages/kombu/asynchronous/http/curl.py", line 132, in _process_pending_requests
 self._process(curl, errno, reason)
 File "/usr/local/lib/python3.5/site-packages/kombu/asynchronous/http/curl.py", line 178, in _process
 buffer=buffer, effective_url=effective_url, error=error,
 File "/usr/local/lib/python3.5/site-packages/vine/promises.py", line 150, in __call__
 svpending(*ca, **ck)
 File "/usr/local/lib/python3.5/site-packages/vine/promises.py", line 143, in __call__
 return self.throw()
 File "/usr/local/lib/python3.5/site-packages/vine/promises.py", line 140, in __call__
 retval = fun(*final_args, **final_kwargs)
 File "/usr/local/lib/python3.5/site-packages/vine/funtools.py", line 100, in _transback
 return callback(ret)
 File "/usr/local/lib/python3.5/site-packages/vine/promises.py", line 143, in __call__
 return self.throw()

File "/usr/local/lib/python3.5/site-packages/vine/promises.py", line 140, in __call__
 retval = fun(*final_args, **final_kwargs)
 File "/usr/local/lib/python3.5/site-packages/vine/funtools.py", line 98, in _transback
 callback.throw()
 File "/usr/local/lib/python3.5/site-packages/vine/funtools.py", line 96, in _transback
 ret = filter_(*args + (ret,), **kwargs)
 File "/usr/local/lib/python3.5/site-packages/kombu/asynchronous/aws/connection.py", line 233, in _on_list_ready
 raise self._for_status(response, response.read())
Exception: Request Empty body HTTP 599 Failed to connect to eu-west-1.queue.amazonaws.com port 443: Connection timed out (None)

标签： celeryamazon-sqsairflowworker