java - 同步到 Java 中的 BigQuery“在数组外添加了重复记录”

Question

我正在尝试使用 java 向 BigQuery 插入一行。我插入的实体有一个双重嵌套的字段。

生成适合 BigQuery 的实体：

ObjectMapper mapper = new ObjectMapper();
String barcodeDetailsJSON = order.getBarcodeDetailsJSON();
List<StateForBQ> stateForBQList = new ArrayList<StateForBQ>();
for (State state : order.getStates()) {
  StateForBQ stateForBQ = new StateForBQ(state);
  stateForBQ.setSetOn(new Date(stateForBQ.getSetOn().getTime()/1000));
  stateForBQList.add(stateForBQ);
}
  List<BarcodeDetailForBQ> barcodeDetailForBQList = getBarcodeDetailsFromBarcodeDetailsJSON(barcodeDetailsJSON, order.getIsGrouped());

如果没有以下内容，状态将设置为空。（状态是嵌套实体）

  List<Map<String, Object>> stateMap = 
      mapper.convertValue(stateForBQList, new TypeReference<List<Map<String, Object>>>() {});

如果没有以下内容，barcodeDetails 将被设置为 null。（BarcodeDetails 是双重嵌套实体）

  List<Map<String, Object>> barcodeMapList = 
      mapper.convertValue(barcodeDetailForBQList, new TypeReference<List<Map<String, Object>>>() {});

如果没有以下内容，productPriceDetails、productDetails、cgst、sgst 将被设置为 null

  for (Map<String, Object> barcodeMap : barcodeMapList) {
      barcodeMap.put("productPriceDetails", mapper.convertValue(barcodeMap.get("productPriceDetails"), new TypeReference<Map<String, Object>>() {}));
      barcodeMap.put("productDetails", mapper.convertValue(barcodeMap.get("productDetails"), new TypeReference<Map<String, Object>>() {}));
      barcodeMap.put("cgst", mapper.convertValue(barcodeMap.get("cgst"), new TypeReference<Map<String, Object>>() {}));
      barcodeMap.put("sgst", mapper.convertValue(barcodeMap.get("sgst"), new TypeReference<Map<String, Object>>() {}));
}

准备行内容

  Map<String, Object> rowContent = new HashMap<>();
  rowContent.put("orderId", order.getOrderId());
  rowContent.put("customerId", order.getCustomerId());
  rowContent.put("barcodeDetails", barcodeMapList);
  rowContent.put("states", stateMap);

插入到 BigQuery

Gson gson = new Gson();
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
String datasetName = "Latest_Data";
String tableName= "ORDER_TEMP";
  // [START bigquery_table_insert_rows]
  TableId tableId = TableId.of(datasetName, tableName);
  // Values of the row to insert

String barcodeDetailsJSON = order.getBarcodeDetailsJSON();
  List<BarcodeDetailForBQ> barcodeDetailForBQList = new OrderForBQ().getBarcodeDetailsFromBarcodeDetailsJSON(barcodeDetailsJSON, order.getIsGrouped());

  String recordsContentString = gson.toJson(rowContent);

  InsertAllResponse response =
      bigquery.insertAll(
          InsertAllRequest.newBuilder(tableId)
              .addRow(""+orderId, rowContent)
              // More rows can be added in the same RPC by invoking .addRow() on the builder
              .build());

  if (response.hasErrors()) {
    // If any of the insertions failed, this lets you inspect the errors
    for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
      // inspect row error
    }
  }

以下是我得到的回应。

{
  insertErrors: {
    0: [
      {
        reason: "invalid",
        location: "",
        message: "Repeated record added outside of an array."
      }
    ]
  }
}

标签： javagoogle-bigquery