时间戳

首页 > 解决方案 > sbt 卡住加载项目

scala - sbt 卡住加载项目

问题描述

当我运行以下命令时：

 sbt -mem 2048 update

sbt 卡住加载项目：

[info] welcome to sbt 1.4.0 (Oracle Corporation Java 1.8.0_265)
[info] loading settings for project my-project-build from plugins.sbt ...
[info] loading project definition from /builds/my-project/project
[debug] not up to date. inChanged = true, force = false
[debug] Updating ProjectRef(uri("file:/builds/my-project/project/"), "my-project-build")...

我在启动任何 sbt 命令（版本、编译、运行……）时都有同样的问题

等待超过 1 小时，加载项目步骤完成，但 24 小时后作业仍未终止，这是我的 build.sbt：

name := "batch_spark"
version := "0.1"
scalaVersion := "2.11.12"

logLevel := Level.Debug

val sparkVersion = "2.3.2"
val hbaseVersion = "1.1.7"
val clusterVersion = "4.7.0.2.6.5.0-292"

checksums in update := Nil
useCoursier := false

libraryDependencies ++= Seq(

  "org.apache.spark" %% "spark-core" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-sql" % sparkVersion % "provided",

  "org.apache.phoenix" % "phoenix-core" % "4.7.0.2.6.5.102-5",
  "org.apache.phoenix" % "phoenix-spark2" % clusterVersion,

  "org.apache.hbase" % "hbase" % hbaseVersion ,
  "org.apache.hbase" % "hbase-common" % hbaseVersion,
  "org.apache.hbase" % "hbase-server" % hbaseVersion,
  "org.apache.hbase" % "hbase-client" % hbaseVersion,

  "org.apache.hadoop" % "hadoop-hdfs" % "2.9.0",
  "org.apache.hadoop" % "hadoop-common" % "2.9.0",
  "org.apache.hadoop" % "hadoop-client" % "2.7.3.2.6.5.0-292",
  "com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.8.8",
  "org.apache.thrift" % "libfb303" % "0.9.2",
  "com.github.pureconfig" %% "pureconfig" % "0.13.0"

)
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.8.8"
dependencyOverrides += "com.google.protobuf" % "protobuf-java" % "2.5.0"
resolvers += "Hortonworks repository" at "https://repo.hortonworks.com/content/repositories/releases"
resolvers += "conjars" at "https://conjars.org/repo/"
resolvers += "miroir-artifactory" at "https://artifactory-toto.fr/artifactory/"

mainClass in assembly := Some("fr.company.service.SparkPhoenix")

assemblyMergeStrategy in assembly := {
  // META-INF discarding
  case PathList("META-INF", xs @ _*) => MergeStrategy.discard
  case x => MergeStrategy.first
}

你有什么主意吗？

标签： scalasbt

解决方案

推荐阅读