时间戳

首页 > 解决方案 > 全新安装后未运行 Ubuntu 20.04 上的 Apache Airflow

airflow - 全新安装后未运行 Ubuntu 20.04 上的 Apache Airflow

问题描述

我按照Running Airflow on Ubuntu 20.04 (TypeError: required field "type_ignores" missing from Module```)中的步骤操作，并收到以下信息：

(airflow-uGvev7QO) root@testing2:/opt/airflow# airflow db init
DB: sqlite:////root/airflow/airflow.db
[2021-03-30 21:17:43,978] {db.py:674} INFO - Creating tables
/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:3454 SAWarning: relationship 'DagRun.serialized_dag' will copy column serialized_dag.dag_id to column dag_run.dag_id, which conflicts with relationship(s): 'TaskInstance.dag_run' (copies task_instance.dag_id to dag_run.dag_id), 'DagRun.task_instances' (copies task_instance.dag_id to dag_run.dag_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards.   The 'overlaps' parameter may be used to remove this warning.
/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:3454 SAWarning: relationship 'SerializedDagModel.dag_runs' will copy column serialized_dag.dag_id to column dag_run.dag_id, which conflicts with relationship(s): 'TaskInstance.dag_run' (copies task_instance.dag_id to dag_run.dag_id), 'DagRun.task_instances' (copies task_instance.dag_id to dag_run.dag_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards.   The 'overlaps' parameter may be used to remove this warning.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/bin/airflow", line 8, in <module>
    sys.exit(main())
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main
    args.func(args)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
    return func(*args, **kwargs)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/cli/commands/db_command.py", line 31, in initdb
    db.initdb()
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/utils/db.py", line 549, in initdb
    upgradedb()
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/utils/db.py", line 684, in upgradedb
    command.upgrade(config, 'heads')
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/command.py", line 294, in upgrade
    script.run_env()
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/script/base.py", line 490, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
    module = load_module_py(module_id, path)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/util/compat.py", line 182, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/migrations/env.py", line 108, in <module>
    run_migrations_online()
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/runtime/migration.py", line 548, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/command.py", line 283, in upgrade
    return script._upgrade_revs(revision, rev)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/script/base.py", line 365, in _upgrade_revs
    revs = list(revs)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/script/revision.py", line 916, in _iterate_revisions
    uppers = util.dedupe_tuple(self.get_revisions(upper))
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/script/revision.py", line 457, in get_revisions
    resolved_id, branch_label = self._resolve_revision_number(id_)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/script/revision.py", line 640, in _resolve_revision_number
    self._revision_map
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/util/langhelpers.py", line 234, in __get__
    obj.__dict__[self.__name__] = result = self.fget(obj)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/script/revision.py", line 156, in _revision_map
    for revision in self._generator():
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/script/base.py", line 115, in _load_revisions
    script = Script._from_filename(self, vers, file_)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/script/base.py", line 904, in _from_filename
    module = util.load_python_file(dir_, filename)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
    module = load_module_py(module_id, path)
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/alembic/util/compat.py", line 182, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/migrations/versions/2c6edca13270_resource_based_permissions.py", line 29, in <module>
    from airflow.www.app import create_app
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/www/app.py", line 38, in <module>
    from airflow.www.extensions.init_views import (
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/www/extensions/init_views.py", line 29, in <module>
    from airflow.www.views import lazy_add_provider_discovered_options_to_connection_form
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/www/views.py", line 96, in <module>
    from airflow.www import auth, utils as wwwutils
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/airflow/www/utils.py", line 27, in <module>
    from flask_appbuilder.models.sqla.interface import SQLAInterface
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/flask_appbuilder/models/sqla/interface.py", line 16, in <module>
    from sqlalchemy_utils.types.uuid import UUIDType
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/sqlalchemy_utils/__init__.py", line 1, in <module>
    from .aggregates import aggregated  # noqa
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/sqlalchemy_utils/aggregates.py", line 372, in <module>
    from .functions.orm import get_column_key
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/sqlalchemy_utils/functions/__init__.py", line 1, in <module>
    from .database import (  # noqa
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/sqlalchemy_utils/functions/database.py", line 11, in <module>
    from .orm import quote
  File "/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/sqlalchemy_utils/functions/orm.py", line 14, in <module>
    from sqlalchemy.orm.query import _ColumnEntity
ImportError: cannot import name '_ColumnEntity' from 'sqlalchemy.orm.query' (/root/.local/share/virtualenvs/airflow-uGvev7QO/lib/python3.7/site-packages/sqlalchemy/orm/query.py)
(airflow-uGvev7QO) root@testing2:/opt/airflow#

欢迎咨询。

谢谢

标签： airflow

解决方案

我能够重现该错误，原因似乎是 sqlalchemy 1.4 的发布引入了重大更改。Airflow 默认依赖于最新版本的 sqlalchemy，但不能使用 1.4 版本。一种解决方法是将 sqlalchem 降级到 < 1.4.0 的版本：

pipenv uninstall sqlalchemy
pipenv install 'sqlalchemy < 1.4.0'

安装 Airflow 新鲜时：

pipenv install 'sqlalchemy < 1.4.0' apache-airflow

推荐阅读