时间戳

首页 > 解决方案 > 运行数据流时出现问题

google-cloud-dataflow - 运行数据流时出现问题

问题描述

运行数据流作业时出现以下错误。我正在尝试将现有的梁版本更新为 2.11.0，但在运行时出现错误。

java.lang.IncompatibleClassChangeError：类 org.apache.beam.model.pipeline.v1.RunnerApi$StandardPTransforms$Primitives 未在 org.apache.beam.runners.core.construction 实现请求的接口 com.google.protobuf.ProtocolMessageEnum。 org.apache.beam.runners.core.construction.PTransformTranslation.(PTransformTranslation.java:58) 在 org.apache.beam.runners.core.construction.UnconsumedReads$1.visitValue( UnconsumedReads.java:49) 在 org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:666) 在 org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java: 649）在 org.apache.beam.sdk.runners.TransformHierarchy$Node 的 org.apache.beam.sdk.runners.TransformHierarchy$Node.visit（TransformHierarchy.java:649）。在 org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java) 访问（TransformHierarchy.java:649） :245) 在 org.apache.beam.runners.core.construction.UnconsumedReads.ensureAllReadsConsumed(UnconsumedReads.java:40) 在 org.apache.Pipeline.traverseTopologically(Pipeline.java:458)。 org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:660) 上 org.apache.beam.runners.dataflow.DataflowRunner 上的 beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java:868)。在 org.apache.beam.sdk.Pipeline.run(Pipeline.java:313) 上运行 (DataflowRunner.java:173) 在 org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311) at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245) at org.apache.beam.sdk.Pipeline .traverseTopologically(Pipeline.java:458) at org.apache.beam.runners.core.construction.UnconsumedReads.ensureAllReadsConsumed(UnconsumedReads.java:40) at org.apache.beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java :868) 在 org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:173) 在 org.apache. org.apache.beam.sdk.Pipeline.run(Pipeline.java:299) 上的 beam.sdk.Pipeline.run(Pipeline.java:313)beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311) at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245) at org.apache.beam.sdk.Pipeline .traverseTopologically(Pipeline.java:458) at org.apache.beam.runners.core.construction.UnconsumedReads.ensureAllReadsConsumed(UnconsumedReads.java:40) at org.apache.beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java :868) 在 org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:173) 在 org.apache. org.apache.beam.sdk.Pipeline.run(Pipeline.java:299) 上的 beam.sdk.Pipeline.run(Pipeline.java:313)runners.TransformHierarchy.visit(TransformHierarchy.java:245) at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458) at org.apache.beam.runners.core.construction.UnconsumedReads.ensureAllReadsConsumed(UnconsumedReads. java:40) at org.apache.beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java:868) at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:660) at org.apache .beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:173) 在 org.apache.beam.sdk.Pipeline.run(Pipeline.java:313) 在 org.apache.beam.sdk.Pipeline.run(Pipeline .java:299)runners.TransformHierarchy.visit(TransformHierarchy.java:245) at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458) at org.apache.beam.runners.core.construction.UnconsumedReads.ensureAllReadsConsumed(UnconsumedReads. java:40) at org.apache.beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java:868) at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:660) at org.apache .beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:173) 在 org.apache.beam.sdk.Pipeline.run(Pipeline.java:313) 在 org.apache.beam.sdk.Pipeline.run(Pipeline .java:299)ensureAllReadsConsumed(UnconsumedReads.java:40) 在 org.apache.beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java:868) 在 org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:660)在 org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:173) 在 org.apache.beam.sdk.Pipeline.run(Pipeline.java:313) 在 org.apache.beam.sdk.Pipeline .run(管道.java:299)ensureAllReadsConsumed(UnconsumedReads.java:40) 在 org.apache.beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java:868) 在 org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:660)在 org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:173) 在 org.apache.beam.sdk.Pipeline.run(Pipeline.java:313) 在 org.apache.beam.sdk.Pipeline .run(管道.java:299)299)299)

标签： google-cloud-dataflowapache-beam

解决方案

我没有足够的代表发表评论，但我遇到了这个问题，后来发现我的问题是我的 pom.xml 中有不同的梁版本。有些有 2.19，有些有 2.20。

我会在 pom 或 gradle 文件中快速搜索您的版本，以确保它们都相同。

推荐阅读