首页
学习
活动
专区
圈层
工具
发布
首页
学习
活动
专区
圈层
工具
MCP广场
社区首页 >问答首页 >无法在本地运行气流,接收各种错误(InvalidToken,admin已经存在,webserver已经在运行)

无法在本地运行气流,接收各种错误(InvalidToken,admin已经存在,webserver已经在运行)
EN

Stack Overflow用户
提问于 2021-10-11 17:39:41
回答 1查看 815关注 0票数 0

我很难在当地运行我们的气流部署。以下是我们目前收到的日志:

代码语言:javascript
运行
复制
webserver_1  |   ____________       _____________
webserver_1  |  ____    |__( )_________  __/__  /________      __
webserver_1  | ____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
webserver_1  | ___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
webserver_1  |  _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
webserver_1  | [2021-10-11 17:30:43,668] {dagbag.py:496} INFO - Filling up the DagBag from /dev/null
webserver_1  | [2021-10-11 17:30:44,099] {manager.py:573} WARNING - Refused to delete permission view, assoc with role exists View Menus.can_create Admin
webserver_1  | [2021-10-11 17:30:44,115] {manager.py:573} WARNING - Refused to delete permission view, assoc with role exists View Menus.can_edit Admin
webserver_1  | [2021-10-11 17:30:44,126] {manager.py:573} WARNING - Refused to delete permission view, assoc with role exists View Menus.can_delete Admin
webserver_1  | [2021-10-11 17:30:44,461] {manager.py:573} WARNING - Refused to delete permission view, assoc with role exists DAG Runs.clear Admin
webserver_1  | Traceback (most recent call last):
webserver_1  |   File "/usr/local/bin/airflow", line 8, in <module>
webserver_1  |     sys.exit(main())
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main
webserver_1  |     args.func(args)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
webserver_1  |     return func(*args, **kwargs)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 91, in wrapper
webserver_1  |     return f(*args, **kwargs)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/webserver_command.py", line 368, in webserver
webserver_1  |     check_if_pidfile_process_is_running(pid_file=pid_file, process_name="webserver")
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/process_utils.py", line 267, in check_if_pidfile_process_is_running
webserver_1  |     raise AirflowException(f"The {process_name} is already running under PID {pid}.")
webserver_1  | airflow.exceptions.AirflowException: The webserver is already running under PID 330.
postgres_1   | LOG:  unexpected EOF on client connection with an open transaction
cbb-airflow_webserver_1 exited with code 1
webserver_1  | Requirement already satisfied: google-cloud-core==1.4.1 in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 1)) (1.4.1)
webserver_1  | Requirement already satisfied: google-cloud-datastore==1.15.0 in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 2)) (1.15.0)
webserver_1  | Requirement already satisfied: gcsfs==0.6.1 in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 3)) (0.6.1)
webserver_1  | Requirement already satisfied: flatten-dict in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 4)) (0.4.2)
webserver_1  | Requirement already satisfied: bigquery_schema_generator in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 5)) (1.4)
webserver_1  | Requirement already satisfied: backoff in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 6)) (1.11.1)
webserver_1  | Requirement already satisfied: six==1.13.0 in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 7)) (1.13.0)
webserver_1  | Requirement already satisfied: ndjson in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 8)) (0.3.1)
webserver_1  | Requirement already satisfied: pymongo in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 10)) (3.12.0)
webserver_1  | Requirement already satisfied: SQLAlchemy==1.3.15 in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 14)) (1.3.15)
webserver_1  | Requirement already satisfied: pandas==1.3.1 in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 15)) (1.3.1)
webserver_1  | Requirement already satisfied: numpy==1.21.1 in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 16)) (1.21.1)
webserver_1  | Requirement already satisfied: billiard in /usr/local/lib/python3.7/site-packages (from -r /requirements.txt (line 17)) (3.6.4.0)
webserver_1  | Requirement already satisfied: google-api-core<2.0.0dev,>=1.19.0 in /usr/local/lib/python3.7/site-packages (from google-cloud-core==1.4.1->-r /requirements.txt (line 1)) (1.31.2)
webserver_1  | Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.7/site-packages (from gcsfs==0.6.1->-r /requirements.txt (line 3)) (2021.7.0)
webserver_1  | Requirement already satisfied: google-auth-oauthlib in /usr/local/lib/python3.7/site-packages (from gcsfs==0.6.1->-r /requirements.txt (line 3)) (0.4.5)
webserver_1  | Requirement already satisfied: decorator in /usr/local/lib/python3.7/site-packages (from gcsfs==0.6.1->-r /requirements.txt (line 3)) (5.0.9)
webserver_1  | Requirement already satisfied: requests in /usr/local/lib/python3.7/site-packages (from gcsfs==0.6.1->-r /requirements.txt (line 3)) (2.26.0)
webserver_1  | Requirement already satisfied: google-auth>=1.2 in /usr/local/lib/python3.7/site-packages (from gcsfs==0.6.1->-r /requirements.txt (line 3)) (1.35.0)
webserver_1  | Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/site-packages (from pandas==1.3.1->-r /requirements.txt (line 15)) (2021.1)
webserver_1  | Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/site-packages (from pandas==1.3.1->-r /requirements.txt (line 15)) (2.8.2)
webserver_1  | Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/site-packages (from flatten-dict->-r /requirements.txt (line 4)) (4.6.4)
webserver_1  | Requirement already satisfied: protobuf>=3.12.0 in /usr/local/lib/python3.7/site-packages (from google-api-core<2.0.0dev,>=1.19.0->google-cloud-core==1.4.1->-r /requirements.txt (line 1)) (3.17.3)
webserver_1  | Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.7/site-packages (from google-api-core<2.0.0dev,>=1.19.0->google-cloud-core==1.4.1->-r /requirements.txt (line 1)) (57.4.0)
webserver_1  | Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.7/site-packages (from google-api-core<2.0.0dev,>=1.19.0->google-cloud-core==1.4.1->-r /requirements.txt (line 1)) (1.53.0)
webserver_1  | Requirement already satisfied: packaging>=14.3 in /usr/local/lib/python3.7/site-packages (from google-api-core<2.0.0dev,>=1.19.0->google-cloud-core==1.4.1->-r /requirements.txt (line 1)) (21.0)
webserver_1  | Requirement already satisfied: grpcio<2.0dev,>=1.29.0 in /usr/local/lib/python3.7/site-packages (from google-api-core<2.0.0dev,>=1.19.0->google-cloud-core==1.4.1->-r /requirements.txt (line 1)) (1.39.0)
webserver_1  | Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/site-packages (from google-auth>=1.2->gcsfs==0.6.1->-r /requirements.txt (line 3)) (4.7.2)
webserver_1  | Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/site-packages (from google-auth>=1.2->gcsfs==0.6.1->-r /requirements.txt (line 3)) (4.2.2)
webserver_1  | Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/site-packages (from google-auth>=1.2->gcsfs==0.6.1->-r /requirements.txt (line 3)) (0.2.8)
webserver_1  | Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/site-packages (from packaging>=14.3->google-api-core<2.0.0dev,>=1.19.0->google-cloud-core==1.4.1->-r /requirements.txt (line 1)) (2.4.7)
webserver_1  | Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth>=1.2->gcsfs==0.6.1->-r /requirements.txt (line 3)) (0.4.8)
webserver_1  | Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.7/site-packages (from requests->gcsfs==0.6.1->-r /requirements.txt (line 3)) (2.0.4)
webserver_1  | Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from requests->gcsfs==0.6.1->-r /requirements.txt (line 3)) (2021.5.30)
webserver_1  | Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests->gcsfs==0.6.1->-r /requirements.txt (line 3)) (3.2)
webserver_1  | Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/site-packages (from requests->gcsfs==0.6.1->-r /requirements.txt (line 3)) (1.26.6)
webserver_1  | Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/site-packages (from google-auth-oauthlib->gcsfs==0.6.1->-r /requirements.txt (line 3)) (1.3.0)
webserver_1  | Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib->gcsfs==0.6.1->-r /requirements.txt (line 3)) (3.1.1)
webserver_1  | Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/site-packages (from importlib-metadata->flatten-dict->-r /requirements.txt (line 4)) (3.10.0.0)
webserver_1  | Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/site-packages (from importlib-metadata->flatten-dict->-r /requirements.txt (line 4)) (3.5.0)
postgres_1   | LOG:  incomplete startup packet
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The hide_sensitive_variable_fields option in [admin] has been moved to the hide_sensitive_var_conn_fields option in [core] - the old setting has been used, but please update your config.
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The default_queue option in [celery] has been moved to the default_queue option in [operators] - the old setting has been used, but please update your config.
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The default_queue option in [celery] has been moved to the default_queue option in [operators] - the old setting has been used, but please update your config.
webserver_1  | DB: postgresql+psycopg2://airflow:***@postgres:5432/airflow
webserver_1  | [2021-10-11 17:30:59,666] {db.py:702} INFO - Creating tables
webserver_1  | INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
webserver_1  | INFO  [alembic.runtime.migration] Will assume transactional DDL.
webserver_1  | Traceback (most recent call last):
webserver_1  |   File "/usr/local/bin/airflow", line 8, in <module>
webserver_1  |     sys.exit(main())
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main
webserver_1  |     args.func(args)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
webserver_1  |     return func(*args, **kwargs)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/db_command.py", line 31, in initdb
webserver_1  |     db.initdb()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 576, in initdb
webserver_1  |     create_default_connections()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/session.py", line 70, in wrapper
webserver_1  |     return func(*args, session=session, **kwargs)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 335, in create_default_connections
webserver_1  |     session,
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/session.py", line 67, in wrapper
webserver_1  |     return func(*args, **kwargs)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 61, in merge_conn
webserver_1  |     if not session.query(Connection).filter(Connection.conn_id == conn.conn_id).first():
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3298, in first
webserver_1  |     ret = list(self[0:1])
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3076, in __getitem__
webserver_1  |     return list(res)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 101, in instances
webserver_1  |     cursor.close()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
webserver_1  |     exc_value, with_traceback=exc_tb,
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
webserver_1  |     raise exception
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 81, in instances
webserver_1  |     rows = [proc(row) for row in fetch]
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 81, in <listcomp>
webserver_1  |     rows = [proc(row) for row in fetch]
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 599, in _instance
webserver_1  |     state.manager.dispatch.load(state, context)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/event/attr.py", line 322, in __call__
webserver_1  |     fn(*args, **kw)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/mapper.py", line 3353, in _event_on_load
webserver_1  |     instrumenting_mapper._reconstructor(state.obj())
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/models/connection.py", line 150, in on_db_load
webserver_1  |     if self.password:
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 356, in __get__
webserver_1  |     retval = self.descriptor.__get__(instance, owner)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/models/connection.py", line 235, in get_password
webserver_1  |     return fernet.decrypt(bytes(self._password, 'utf-8')).decode()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/cryptography/fernet.py", line 194, in decrypt
webserver_1  |     raise InvalidToken
webserver_1  | cryptography.fernet.InvalidToken
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The hide_sensitive_variable_fields option in [admin] has been moved to the hide_sensitive_var_conn_fields option in [core] - the old setting has been used, but please update your config.
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The default_queue option in [celery] has been moved to the default_queue option in [operators] - the old setting has been used, but please update your config.
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The default_queue option in [celery] has been moved to the default_queue option in [operators] - the old setting has been used, but please update your config.
webserver_1  | [2021-10-11 17:31:05,551] {manager.py:573} WARNING - Refused to delete permission view, assoc with role exists View Menus.can_create Admin
webserver_1  | [2021-10-11 17:31:05,561] {manager.py:573} WARNING - Refused to delete permission view, assoc with role exists View Menus.can_edit Admin
webserver_1  | [2021-10-11 17:31:05,571] {manager.py:573} WARNING - Refused to delete permission view, assoc with role exists View Menus.can_delete Admin
webserver_1  | [2021-10-11 17:31:05,808] {manager.py:573} WARNING - Refused to delete permission view, assoc with role exists DAG Runs.clear Admin
webserver_1  | admin already exist in the db
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The hide_sensitive_variable_fields option in [admin] has been moved to the hide_sensitive_var_conn_fields option in [core] - the old setting has been used, but please update your config.
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The hide_sensitive_variable_fields option in [admin] has been moved to the hide_sensitive_var_conn_fields option in [core] - the old setting has been used, but please update your config.
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The default_queue option in [celery] has been moved to the default_queue option in [operators] - the old setting has been used, but please update your config.
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The default_queue option in [celery] has been moved to the default_queue option in [operators] - the old setting has been used, but please update your config.
webserver_1  | /usr/local/lib/python3.7/site-packages/airflow/configuration.py:345 DeprecationWarning: The default_queue option in [celery] has been moved to the default_queue option in [operators] - the old setting has been used, but please update your config.

我认为有问题的行是:

  • airflow.exceptions.AirflowException: we服务器已经运行在PID 330下。cryptography.fernet.InvalidToken
  • admin已经存在于与代码137一起退出的db
  • cbb-airflow_webserver_1中(没有在上面显示,但我们在其他当前日志中也收到了这个)

我完全不知道是什么引起了这些问题。气流在当地正常工作,在我触发DAG在本地运行后,它立即停止工作。当我访问localhost:8082,localhost:8082/home,localhost:8082/admin,都没有工作,都显示了标准的“网站无法到达”错误页面。

对于为什么会出现这种情况,以及我们如何解决这个问题,有什么想法吗?如果这有帮助的话,我可以共享我们的docker-compose.yml文件。如果这有助于了解,我们最近也更新了从气流v1到v2,尽管气流在触发DAG之前曾在v2上工作,导致了所有的故障。

EN

回答 1

Stack Overflow用户

发布于 2021-10-11 20:04:53

很可能你同时运行Airlfow 1和气流2。你应该确保你清理你的环境,一次只使用一个气流版本/对接-合成。

您可以使用通常的停靠器和坞-组合命令来杀死/停止运行坞容器。您可以很容易地找到正确的命令。例如

docker kill $(docker ps -q)

会杀死所有码头集装箱。

票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/69530399

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档