python - 气流：Google Cloud Composer：TypeError：“字节”类型的对象不是 JSON 可序列化的

Question

我正在使用 MySqlToGoogleCloudStorageOperator，如下所示。

Dag 任务详情

def gen_export_table_task(table_config):
    export_task = MySqlToGoogleCloudStorageOperator(task_id='export_dag_run_to_gcs',
                                                    dag=dag,
                                                    sql='SELECT * FROM dag_run LIMIT 10',
                                                    bucket='gs://aca-composer-bq-test-vishesh',
                                                    filename="cloudsql_to_bigquery_file.json",
                                                    google_cloud_storage_conn_id='google_cloud_storage_default',
                                                    mysql_conn_id='airflow_db')
    return export_task

请在下面找到我在执行 DAG 时遇到的错误详细信息：

[2019-12-19 11:32:29,649] {models.py:1796} ERROR - Object of type 'bytes' is not JSON serializable
Traceback (most recent call last)
  File "/usr/local/lib/airflow/airflow/models.py", line 1659, in _run_raw_tas
    result = task_copy.execute(context=context
  File "/usr/local/lib/airflow/airflow/contrib/operators/mysql_to_gcs.py", line 106, in execut
    files_to_upload = self._write_local_data_files(cursor
  File "/usr/local/lib/airflow/airflow/contrib/operators/mysql_to_gcs.py", line 153, in _write_local_data_file
    s = json.dumps(row_dict
  File "/opt/python3.6/lib/python3.6/json/__init__.py", line 231, in dump
    return _default_encoder.encode(obj
  File "/opt/python3.6/lib/python3.6/json/encoder.py", line 199, in encod
    chunks = self.iterencode(o, _one_shot=True
  File "/opt/python3.6/lib/python3.6/json/encoder.py", line 257, in iterencod
    return _iterencode(o, 0
  File "/opt/python3.6/lib/python3.6/json/encoder.py", line 180, in defaul
    o.__class__.__name__
TypeError: Object of type 'bytes' is not JSON serializabl

标签： pythonairflowgoogle-cloud-composer