时间戳

首页 > 解决方案 > 触发气流后立即杀死任务

airflow - 触发气流后立即杀死任务

问题描述

我在 MySQL 中使用带有 LocalExecutor 和后端的 Airflow 2.0。

我的问题是，每次气流正确安排我的一项任务时，它都会自动终止。这是日志：

[2021-03-07 21:01:05,855] {bash.py:158} INFO - Running command: sudo /d/Airflow/pdi-ce-8.3.0.0-371/data-integration/pan.sh -rep:"penta_repo" -user:dummy -pass:dummy -dir:BDN_DCC/ALIMENTADOR -trans:"ALIMENTADOR_1" -level:Basic
[2021-03-07 21:01:05,942] {bash.py:169} INFO - Output:
[2021-03-07 21:01:10,897] {local_task_job.py:169} WARNING - State of this instance has been externally set to None. Terminating instance.
[2021-03-07 21:01:10,922] {process_utils.py:95} INFO - Sending Signals.SIGTERM to GPID 19956
[2021-03-07 21:01:10,923] {taskinstance.py:1214} ERROR - Received SIGTERM. Terminating subprocesses.
[2021-03-07 21:01:10,924] {bash.py:185} INFO - Sending SIGTERM signal to bash process group
[2021-03-07 21:01:10,965] {taskinstance.py:1396} ERROR - [Errno 1] Operation not permitted
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/airflow/models/taskinstance.py", line 1086, in _run_raw_task
    self._prepare_and_execute_task_with_callbacks(context, task)
  File "/usr/local/lib/python3.8/dist-packages/airflow/models/taskinstance.py", line 1260, in _prepare_and_execute_task_with_callbacks
    result = self._execute_task(context, task_copy)
  File "/usr/local/lib/python3.8/dist-packages/airflow/models/taskinstance.py", line 1300, in _execute_task
    result = task_copy.execute(context=context)
  File "/usr/local/lib/python3.8/dist-packages/airflow/operators/bash.py", line 171, in execute
    for raw_line in iter(self.sub_process.stdout.readline, b''):
  File "/usr/local/lib/python3.8/dist-packages/airflow/models/taskinstance.py", line 1215, in signal_handler
    task_copy.on_kill()
  File "/usr/local/lib/python3.8/dist-packages/airflow/operators/bash.py", line 187, in on_kill
    os.killpg(os.getpgid(self.sub_process.pid), signal.SIGTERM)
PermissionError: [Errno 1] Operation not permitted
[2021-03-07 21:01:10,968] {taskinstance.py:1433} INFO - Marking task as UP_FOR_RETRY. dag_id=ALIMENTADOR_JOB, task_id=Alimentador, execution_date=20210307T230100, start_date=20210308T000105, end_date=20210308T000110
[2021-03-07 21:01:11,337] {process_utils.py:61} INFO - Process psutil.Process(pid=19956, status='terminated', exitcode=1, started='21:01:05') (19956) terminated with exit code 1
[2021-03-07 21:01:49,679] {process_utils.py:61} INFO - Process psutil.Process(pid=20807, status='terminated', started='21:01:06') (20807) terminated with exit code None
[2021-03-07 21:01:49,680] {process_utils.py:61} INFO - Process psutil.Process(pid=20792, status='terminated', started='21:01:06') (20792) terminated with exit code None
[2021-03-07 21:01:49,681] {process_utils.py:61} INFO - Process psutil.Process(pid=20815, status='terminated', started='21:01:06') (20815) terminated with exit code None
[2021-03-07 21:01:49,682] {process_utils.py:61} INFO - Process psutil.Process(pid=19972, status='terminated', started='21:01:05') (19972) terminated with exit code None
[2021-03-07 21:01:49,682] {process_utils.py:61} INFO - Process psutil.Process(pid=19994, status='terminated', started='21:01:05') (19994) terminated with exit code None
[2021-03-07 21:01:49,683] {process_utils.py:61} INFO - Process psutil.Process(pid=20778, status='terminated', started='21:01:06') (20778) terminated with exit code None
[2021-03-07 21:01:49,684] {process_utils.py:61} INFO - Process psutil.Process(pid=20817, status='terminated', started='21:01:06') (20817) terminated with exit code None
[2021-03-07 21:01:49,684] {process_utils.py:61} INFO - Process psutil.Process(pid=20033, status='terminated', started='21:01:06') (20033) terminated with exit code None
[2021-03-07 21:01:49,685] {process_utils.py:61} INFO - Process psutil.Process(pid=20804, status='terminated', started='21:01:06') (20804) terminated with exit code None
[2021-03-07 21:01:49,685] {local_task_job.py:118} INFO - Task exited with return code 1

我已经多次查看气流配置文件，试图增加超时但没有任何效果。直接在命令提示符下运行 bash 命令可以正常工作。关于这里发生的事情有什么想法吗？

谢谢！

标签： airflowairflow-scheduler

解决方案

[2021-03-07 21:01:10,897] {local_task_job.py:169} 警告 - 此实例的状态已在外部设置为无。终止实例。[2021-03-07 21:01:10,922] {process_utils.py:95} INFO - 将 Signals.SIGTERM 发送到 GPID 19956 [2021-03-07 21:01:10,923] {taskinstance.py:1214} 错误 - 收到SIGTERM。终止子进程。[2021-03-07 21:01:10,924] {bash.py:185} INFO - 向 bash 进程组发送 SIGTERM 信号 [2021-03-07 21:01:10,965] {taskinstance.py:1396} 错误 - [ Errno 1] 不允许操作

这些行表示您有 OOM 问题，请检查您的 CPU 和 RAM

推荐阅读