时间戳

首页 > 解决方案 > Not being able to export a Big Query table

php - Not being able to export a Big Query table

问题描述

I am trying to export a Big Query table into cloud storage using NEWLINE_DELIMITED_JSON as destination format but I keep on receiving the following error: Error running job: Operation cannot be performed on a nested schema.

I know Big Query allows to export nested tables if you choose the json format (as it is stated both on the documentation and on this question), so I don't understand how come I am getting this error... I am using the PHP SDK and this is the code I've been using:

extract_table($projectId, $datasetId, $tableId, $bucketName, $objectName);


function extract_table($projectId, $datasetId, $tableId, $bucketName, $objectName, $format = 'NEWLINE_DELIMITED_JSON'){
    $bigQuery = new BigQueryClient(['projectId' => $projectId,]);
    $dataset = $bigQuery->dataset($datasetId);
    $table = $dataset->table($tableId);
    // load the storage object
    $storage = new StorageClient([
      'projectId' => $projectId,
    ]);
    $destinationObject = $storage->bucket($bucketName)->object($objectName);
    // create the extract job
    $options = ['destinationFormat' => $format];
    $extractConfig = $table->extract($destinationObject, $options);

    $job = $table->runJob($extractConfig);
    // poll the job until it is complete
    $backoff = new ExponentialBackoff(10);
    try {
       $backoff->execute(function () use ($job) {
           print('Waiting for job to complete' . PHP_EOL);
           $job->reload();
           if (!$job->isComplete()) {
               throw new Exception('Job has not yet completed', 500);
           }
        });
    } catch (Exception $e) {
    }
    // check if the job has errors
    if (isset($job->info()['status']['errorResult'])) {
        $error = $job->info()['status']['errorResult']['message'];
        printf('Error running job: %s' . PHP_EOL, $error);
    } else {
        print('Data extracted successfully' . PHP_EOL);
    }
}

标签： phpgoogle-bigquery

解决方案

您的问题是您使用了错误的options字段配置。按照有关该方法的PHP 文档Table.extract，您将看到该方法中的options字段extract必须是一个configuration对象，例如BigQuery 文档中提供的对象。

您将options对象定义为：

$options = ['destinationFormat' => $format];

相反，您应该使用以下格式：

$options = ['configuration' => ['extract' => ['destinationFormat' => $format]]];

这是因为该destinationFormat字段嵌套在configurationand下extract。

推荐阅读