json - Jolt generates null values as transformation output and can't be removed

Question

I have this JSON as input:

{
  "BusinessInfo": {
    "MemberInformation": {
      "Item": [
        {
          "Relation": {
            "Item": [
              {
                "Partner1": "0072938063",
                "Partner2": "0072938064",
                "CategoryId": "BUR001"
              },
              {
                "Partner1": "0072938063",
                "Partner2": "0072937669",
                "CategoryId": "ZCRM06"
              },
              {
                "Partner1": "0072938063",
                "Partner2": "3000011685",
                "CategoryId": "ZCRM06"
              },
              {
                "Partner1": "1020002423",
                "Partner2": "0072938063",
                "CategoryId": "ZCRM01"
              },
              {
                "Partner1": "0072938067",
                "Partner2": "0072938063",
                "CategoryId": "ZCRM04"
              },
              {
                "Partner1": "0072938063",
                "Partner2": "0072937669",
                "CategoryId": "ZCRM04"
              },
              {
                "Partner1": "0072938063",
                "Partner2": "0072938065",
                "CategoryId": "ZCRM04"
              }
            ]
          }
        }
      ]
    }
  }
}

And this Jolt spec:

[
  {
    "operation": "shift",
    "spec": {
      "BusinessInfo": {
        "MemberInformation": {
          "Item": {
            "*": {
              "Relation": {
                "Item": {
                  "*": {
                    "CategoryId": {
                      "ZCRM04": {
                        "@(2,Partner1)": "[&3].Partner1",
                        "@(2,Partner2)": "[&3].Partner2"
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
]

Having as result, this:

[ null, null, null, null, {
  "Partner1" : "0072938067",
  "Partner2" : "0072938063"
}, {
  "Partner1" : "0072938063",
  "Partner2" : "0072937669"
}, {
  "Partner1" : "0072938063",
  "Partner2" : "0072938065"
} ]

The problem is with the null values generated. I need the same result but without them. I tried remove them adding this operations to the spec:

  {
    "operation": "default",
    "spec": {
      "*": "TRASH"
    }
  },
  {
    "operation": "remove",
    "spec": {
      "TRASH": ""
    }
  }

But doesn't work, the result is almost the same, only now instead of null appears "TRASH":

[ "TRASH", "TRASH", "TRASH", "TRASH", {
  "Partner1" : "0072938067",
  "Partner2" : "0072938063"
}, {
  "Partner1" : "0072938063",
  "Partner2" : "0072937669"
}, {
  "Partner1" : "0072938063",
  "Partner2" : "0072938065"
} ]

What could be wrong? In the first transformation or in the next two operations added. It is possible to avoid this from the first transformation?

标签： jsonjolt

Answer 1

您的输出数组中有空值的原因是您的输入“Item”数组中有 7 个元素。您正在使用这 7 个数组索引来确定您的输出数组。

特别是这些行

"Item": {
  "*": {
    "CategoryId": {
      "ZCRM04": {
        "@(2,Partner1)": "[&3].Partner1",
        "@(2,Partner2)": "[&3].Partner2"

您在示例输入数据中，“ZCRM04”是 Item 数组元素 5、6、7 中的 CategoryId。

所以前 0 到 4 个元素不匹配，但元素 5 匹配，并且规范说将数据从“@(2,Partner1)”复制到输出数组的第 5 个索引。

"[&3].Partner1"  -->  "[5].Partner1"  

因此，在这种情况下，shift 将生成一个包含 6 个元素的输出数组，其中元素 0 到 4 为空。

使用 Jolt，如果您想“过滤”数组元素并修改/更改这些数组元素的内容，您需要执行两个步骤。

规格

[
  {
    "operation": "shift",
    "spec": {
      "BusinessInfo": {
        "MemberInformation": {
          "Item": {
            "*": {
              "Relation": {
                "Item": {
                  "*": {
                    "CategoryId": {
                      "ZCRM04": {
                        // in this first shift
                        //  just "identify" all the 
                        //  Item entries needs
                        "@2": "temp[]"
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "temp": {
        "*": {
          // now for each original "Item" 
          //  "fix" it.
          // In this case only pass thru 
          //  Partner1 and 2
          "Partner1": "[&1].Partner1",
          "Partner2": "[&1].Partner2"
        }
      }
    }
  }
]