首页 > 解决方案 > Jackson 数据绑定依赖版本错误 spark 和 scala

问题描述

我正在尝试使用火花流从 kafka 获取数据,并且我使用的是 jackson databind 版本 2.8.7,但它说它需要 jackson databind 版本 >=2.8.0 和 <2.9.0

val stream = KafkaUtils.createDirectStream[String, String](
  scc,
  PreferConsistent,
  Subscribe[String, String](topics, kafkaParams)
)

在这里,在 Subscribe[String, String](topics, kafkaParams) 行显示错误

    Exception in thread "main" java.lang.ExceptionInInitializerError
    at org.apache.spark.SparkContext.withScope(SparkContext.scala:699)
    at org.apache.spark.SparkContext.makeRDD(SparkContext.scala:803)
    at SparkConsumer2$.main(SparkConsumer2.scala:31)
    at SparkConsumer2.main(SparkConsumer2.scala)
    Caused by: com.fasterxml.jackson.databind.JsonMappingException: 
    Scala module 2.8.7 requires Jackson Databind version >= 2.8.0 and 
    < 2.9.0 at 
 com.fasterxml.jackson.module.scala.JacksonModule.setupModule(JacksonM 
   odule.scala:66)
    at 
 com.fasterxml.jackson.module.scala.JacksonModule.setupModule$(JacksonModule.scala:51)
    at com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:18)
    at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:751)
    at org.apache.spark.rdd.RDDOperationScope$.<init>(RDDOperationScope.scala:82)
    at org.apache.spark.rdd.RDDOperationScope$.<clinit>(RDDOperationScope.scala)
    ... 4 more
19/07/24 14:27:10 INFO SparkContext: Invoking stop() from shutdown hook

标签: scalaapache-sparkapache-kafka

解决方案


推荐阅读