时间戳

首页 > 解决方案 > Pentaho 数据集成 Google BigQuery Loader 异常

google-bigquery - Pentaho 数据集成 Google BigQuery Loader 异常

问题描述

我使用 Pentaho 数据集成通过“Google BigQuery Loader”步骤创建将数据从 Google Cloud Storage 加载到 Google BigQuery 的作业。虽然该步骤成功将数据加载到 BigQuery 数据集表中（由 BigQuery 作业日志和表数据检查），但它会引发 NPE：

2019/10/24 10:21:31 - Job 1 - Starting entry [Google BigQuery Loader]
2019/10/24 10:21:31 - Job 1 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : java.lang.NullPointerException
2019/10/24 10:21:31 - Job 1 -   at com.pentaho.di.job.entries.google.bigquery.JobEntryBigQueryLoader.execute(JobEntryBigQueryLoader.java:383)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:680)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:821)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:497)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.run(Job.java:384)
2019/10/24 10:21:31 - Job 1 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : A serious error occurred during job execution: 
2019/10/24 10:21:31 - Job 1 - Unexpected error occurred while launching entry [Google BigQuery Loader.0]
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.run (Job.java:384)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:497)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:821)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:680)
2019/10/24 10:21:31 - Job 1 -  at com.pentaho.di.job.entries.google.bigquery.JobEntryBigQueryLoader.execute (JobEntryBigQueryLoader.java:383)
2019/10/24 10:21:31 - Job 1 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : org.pentaho.di.core.exception.KettleException: 
2019/10/24 10:21:31 - Job 1 - Unexpected error occurred while launching entry [Google BigQuery Loader.0]
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.run (Job.java:384)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:497)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:821)
2019/10/24 10:21:31 - Job 1 -  at org.pentaho.di.job.Job.execute (Job.java:680)
2019/10/24 10:21:31 - Job 1 -  at com.pentaho.di.job.entries.google.bigquery.JobEntryBigQueryLoader.execute (JobEntryBigQueryLoader.java:383)
2019/10/24 10:21:31 - Job 1 - 
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:824)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:497)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.run(Job.java:384)
2019/10/24 10:21:31 - Job 1 - Caused by: java.lang.NullPointerException
2019/10/24 10:21:31 - Job 1 -   at com.pentaho.di.job.entries.google.bigquery.JobEntryBigQueryLoader.execute(JobEntryBigQueryLoader.java:383)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:680)
2019/10/24 10:21:31 - Job 1 -   at org.pentaho.di.job.Job.execute(Job.java:821)
2019/10/24 10:21:31 - Job 1 -   ... 2 more
2019/10/24 10:21:31 - Spoon - Job has ended.

PDI 8.1 和 8.2 显示相同的结果。PDI 8.3 没有 Google BigQuery Loader 步骤。

感谢您提供任何帮助或解决方法。

标签： google-bigquerypentahopentaho-data-integration

解决方案

我得到了 PDI Shell Script step 和gcloud bq cli的解决方法。 Pentaho bugtracker 中的问题。

推荐阅读