时间戳

首页 > 解决方案 > 在 Json 中格式化 Airflow 日志 - 问题

logging - 在 Json 中格式化 Airflow 日志 - 问题

问题描述

按照Format Airflow Logs in JSON和其中提到的指南，我将 log_config 复制到了 airflow/config 文件夹。但是当我运行网络服务器或调度程序时，我得到“模块未定义”（init .py 在配置文件夹中定义）并且 PYTHONPATH 被设置为气流/配置

错误是

  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/airflow/logging_config.py", line 40, in configure_logging
    logging_config = import_string(logging_class_path)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/airflow/utils/module_loading.py", line 33, in import_string
    module = import_module(module_path)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'airflow.config'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/bin/airflow", line 25, in <module>
    from airflow.configuration import conf
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/airflow/__init__.py", line 47, in <module>
    settings.initialize()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/airflow/settings.py", line 374, in initialize
    LOGGING_CLASS_PATH = configure_logging()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/airflow/logging_config.py", line 52, in configure_logging
    'Unable to load custom logging from {} due to {}'.format(logging_class_path,err), isinstance('airflow',dict), isinstance('airflow.config',dict), type('airflow.config')
ImportError: ("Unable to load custom logging from airflow.config.log_config.LOGGING_CONFIG due to No module named 'airflow.config'",

现在它回来了， ImportError: Unable to load custom logging from airflow.config.log_config.LOGGING_CONFIG due to section/key [logging/logging_level] not found in config

标签： loggingairflowairflow-scheduler

解决方案

如果你还没有弄清楚。看起来加载的代码logging_level正在寻找它位于名为的部分logging中airflow.cfg file

但是，默认情况下，此属性实际上位于coresection 下。因此，如果您将代码更改LOG_LEVEL: str = conf.get("core", "LOGGING_LEVEL").upper()
为此应该为您解决，或者您可以[logging]在 airflow.cfg 中添加另一个部分。（需要重新排序）

推荐阅读