使用DataFlowRunner时未实现波束/ DataFlow意外错误ProtocolMessageEnum
使用DataFlowRunner时未实现波束/ DataFlow意外错误ProtocolMessageEnum
提问于 2019-01-14 18:58:21
当在本地运行我的Beam管道时，它完全按预期工作，但是当尝试在DataflowRunner上运行它时，我突然得到下面的错误。老实说，我甚至不知道从哪里开始评估它，因为DataflowRunner似乎是一个黑匣子。
Jan 14, 2019 11:26:51 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 165 files. Enable logging at DEBUG level to see which files will be staged.
Exception in thread "main" java.lang.IncompatibleClassChangeError: Class org.apache.beam.model.pipeline.v1.RunnerApi$StandardPTransforms$Primitives does not implement the requested interface com.google.protobuf.ProtocolMessageEnum
at org.apache.beam.runners.core.construction.BeamUrns.getUrn(BeamUrns.java:27)
at org.apache.beam.runners.core.construction.PTransformTranslation.<clinit>(PTransformTranslation.java:58)
at org.apache.beam.runners.core.construction.UnconsumedReads$1.visitValue(UnconsumedReads.java:49)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:666)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
at org.apache.beam.runners.core.construction.UnconsumedReads.ensureAllReadsConsumed(UnconsumedReads.java:40)
at org.apache.beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java:868)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:660)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:173)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
at (my code: pipe.run().waitUntilFinish();) 回答 1
Stack Overflow用户
发布于 2019-02-12 00:13:05
检查beam等的版本，并在可能的情况下升级依赖项。
我也有同样的错误，在看到你得到这个错误后，我认为这一定是一个依赖冲突，因为它以前不存在。
我使用scio来部署到数据流，只是引用了他们正在使用的内容。我还更新了芭乐和https://github.com/spotify/scio/blob/v0.7.1/build.sbt。
我知道你用的是java，但是试着把beam更新到2.9.0，也许还有芭乐，protobuf...
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/54180175
复制相关文章
相似问题
腾讯云开发者
