apache-spark - 从 CsvSink 测量 Rate 源的指标

Question

我试图测量火花结构化流的性能指标。但我无法在指标日志文件中看到任何数据。我的输入源是速率源，它以每秒指定的行数生成数据。这是我的代码和metrics.properties文件的链接。

当我使用 运行上述代码spark-submit时，我看到指标日志（例如，run_1.driver.spark.streaming.aggregates.inputRate-total.csv）是在指定目录下创建的，但大多数值为 0。下面是 inputeRate-total.csv 文件的一部分：

1529634585,0.0
1529634595,0.0
1529634605,0.0
1529634615,0.0
1529634625,0.0
1529634635,0.0
1529634645,0.0
1529634655,0.0
1529634665,0.0
1529634675,0.0
1529634685,0.0
1529634695,0.0
1529634705,0.0
1529634715,0.0

有什么理由说明为什么会发生这种情况？如果有帮助，很高兴分享更多信息。

标签： apache-sparkspark-streaming