azure - Azure 数据工厂中使用数据流的舍入转换无法正常工作

问题描述

我目前正在使用通过 DontNet.SDK 进行舍入转换的派生列创建数据流。我使用的 Source 和 Sink 数据集是参数化的。我在运行时通过管道参数为它们分配值。请参考下面两个数据流的 json 文件。

我有一个场景是将工资从小数点后三位四舍五入到小数点后两位。当我在 ADF 中手动创建它时，它已成功舍入。下面是转换的输出结果文件

但是当我使用 .net SDK 创建它时，它不起作用。我没有按预期获得列名，但值正确出现。下面是 .SDK 输出

下面是我创建的数据流的 Json 格式

{
"name": "Rounding_Auto__Transformation",
"properties": {
    "type": "MappingDataFlow",
    "typeProperties": {
        "sources": [
            {
                "dataset": {
                    "referenceName": "defaultdataflowSourcedataset",
                    "type": "DatasetReference"
                },
                "name": "source"
            }
        ],
        "sinks": [
            {
                "dataset": {
                    "referenceName": "defaultdataflowSinkdataset",
                    "type": "DatasetReference"
                },
                "name": "sink"
            }
        ],
        "transformations": [
            {
                "name": "DerivedColumn0"
            }
        ],
        "script": "source(output(\n\t\tid as string,\n\t\tsal as string,\n\t\tgender as string,\n\t\tname as string,\n\t\tisMarried as string,\n\t\ttags as string,\n\t\taddress as string\n\t),\n\tallowSchemaDrift: true,\n\tvalidateSchema: false,\n\tignoreNoFilesFound: false) ~> source\nsource derive(NewSal = round(toFloat(sal),2,2)) ~> DerivedColumn0\nDerivedColumn0 sink(allowSchemaDrift: true,\n\tvalidateSchema: false,\n\tpartitionFileNames:['customer_post_with_round.csv'],\n\tpartitionBy('hash', 1),\n\tskipDuplicateMapInputs: true,\n\tskipDuplicateMapOutputs: true) ~> sink"
    }
}

}

我还直接在 ADF 中比较了为手册创建的 json（因为它有效）——这里是手册的

{
"name": "Rounding_Manually",
"properties": {
    "type": "MappingDataFlow",
    "typeProperties": {
        "sources": [
            {
                "dataset": {
                    "referenceName": "SourcDS",
                    "type": "DatasetReference"
                },
                "name": "source1"
            }
        ],
        "sinks": [
            {
                "dataset": {
                    "referenceName": "SinkDS",
                    "type": "DatasetReference"
                },
                "name": "sink1"
            }
        ],
        "transformations": [
            {
                "name": "DerivedColumn1"
            }
        ],
        "script": "source(output(\n\t\tid as string,\n\t\tsal as string,\n\t\tgender as string,\n\t\tname as string,\n\t\tisMarried as string,\n\t\ttags as string,\n\t\taddress as string\n\t),\n\tallowSchemaDrift: true,\n\tvalidateSchema: false,\n\tignoreNoFilesFound: false) ~> source1\nsource1 derive(NewSal = round(toFloat(sal),2,2)) ~> DerivedColumn1\nDerivedColumn1 sink(allowSchemaDrift: true,\n\tvalidateSchema: false,\n\tpartitionFileNames:['customer_post_with_round.csv'],\n\tpartitionBy('hash', 1),\n\tskipDuplicateMapInputs: true,\n\tskipDuplicateMapOutputs: true) ~> sink1"
    }
}

}

请帮忙。

标签： azureazure-data-factorydataflowazure-sdk-.net

azure - Azure 数据工厂中使用数据流的舍入转换无法正常工作

问题描述

解决方案

推荐阅读