smile - SMILE 未能训练回归模型

Question

我有 scala 2.11 和 JDK1.8，在 CDH 上运行微笑 2.4.0。我想用微笑训练一个回归模型，下面是代码。

import smile.regression.{gbm,randomForest,cart,ols}
import smile.data.DataFrame
import smile.data.formula._
import smile.util._

val arrX=lisSamp.map(x=>Array(x._4.toDouble,x._5.toDouble,x._6.toDouble,x._7.toDouble,x._8.toDouble)).toArray
val arrY=lisSamp.map(x=>Array(x._3)).toArray
val sdfX=DataFrame.of(arrX)
val sdfY=DataFrame.of(arrY,"Y")
val sdfFull=sdfX.merge(sdfY)

//train model
val forForm:Formula="Y"~"V1"+"V2"+"V3"
randomForest(forForm,sdfFull,50,3,10,50,100,1)
cart(forForm,sdfFull,10,100,5)
gbm(forForm, sdfFull)
ols(forForm, sdfFull,"qr")

导入smile.util._时出现以下错误，但在第二次导入时出现错误。

error: missing or invalid dependency detected while loading class file 'package.class'.
Could not access term typesafe in package com,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'package.class' was compiled against an incompatible version of com.
error: missing or invalid dependency detected while loading class file 'package.class'.
Could not access type LazyLogging in value com.scalalogging,
because it (or its dependencies) are missing. Check your build definition for
missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
A full rebuild may help if 'package.class' was compiled against an incompatible version of com.scalalogging.

微笑数据框看起来像这样

scala> sdfFull
res40: smile.data.DataFrame =
[V1: double, V2: double, V3: double, V4: double, V5: double, Y: double]
+---+---+---+---+---+---+
| V1| V2| V3| V4| V5|  Y|
+---+---+---+---+---+---+
|  2|  0| 21|  3| 65|  1|
|  2|  0| 21|  3| 70|  1|
|  2|  0| 21|  3| 75|  1|
|  2|  0| 21|  3| 80|  1|
|  2|  0| 21|  3| 85|  1|
|  2|  0| 21|  3| 90|  1|
|  2|  0| 21|  3| 95|  1|
|  2|  0| 21|  3|100|  1|
|  2|  0| 21|  3|105|  1|
|  2|  0| 21|  3|110|  1|
+---+---+---+---+---+---+
156 more rows...

我在尝试运行 gbm、cart、randomForest、ols 回归时收到此错误消息。我做错什么了吗？看起来它没有找到一些包。

java.lang.NoClassDefFoundError: smile/util/package$
  at smile.util.package$time$.apply(package.scala:64)
  at smile.regression.package$.cart(package.scala:249)
...

我运行它来启动 spark-shell

spark-shell  --master yarn   --deploy-mode client 
--jars "/.../jar/lib/smile-core-2.4.0.jar,/.../jar/lib/smile-scala_2.11-2.4.0.jar,/.../jar/lib/smile-data-2.4.0.jar,/.../jar/lib/smile-math-2.4.0.jar"

标签： smile