java - Sparklyr 库中的 FPGrowth() Java 错误

Question

在 Google Cloud 中运行以下代码行，我得到一个 JAVA 错误，但我不知道如何解决它，请帮助我。

我使用 Sparklyr 查找频繁项集，JAVA 版本如下

〜$ java -版本

openjdk 版本“1.8.0_252”

OpenJDK 运行时环境 (build 1.8.0_252-8u252-b09-1~16.04-b09)

OpenJDK 64 位服务器 VM（内部版本 25.252-b09，混合模式）

代码

library(sparklyr)
sc <- spark_connect(master = "local")
* Using Spark: 2.1.0
dataset01 <- read.csv(file='data.csv')
dataset02 <- data.frame(dataset01)
dataset_tbl <- sdf_copy_to(sc, dataset02)
sdf_dim(dataset_tbl)
[1] 23144    33

dataset_tbl
# Source: spark<dataset02> [?? x 33]
   ALT   AST   Acid  BASO_ BASO_1 Cholesterol Creatinin EOS_  EOS_1 Glucose
   <chr> <chr> <chr> <chr> <chr>  <chr>       <chr>     <chr> <chr> <chr>
 1 [0.0… [9.0… [150… [0.0… [0.0-… [1.7-3.9]   [53.0-12… [0.0… [0.0… [3.9-6…
 2 [0.0… [9.0… [150… [0.0… [0.0-… [3.9-5.2]   [53.0-12… [0.0… [0.0… [3.9-6…
 3 [0.0… [9.0… [150… [0.0… [0.0-… [5.2-8.3]   [53.0-12… [0.0… [0.0… [3.9-6…
 4 [40.… [9.0… [150… [0.0… [0.0-… [3.9-5.2]   [53.0-12… [0.0… [0.0… [3.9-6…
 5 [0.0… [9.0… [150… [0.0… [0.0-… [3.9-5.2]   [53.0-12… [0.0… [0.0… [3.9-6…
..........

fp_model <- ml_fpgrowth(dataset_tbl, min_confidence=0.8, min_support=0.3)

Error: java.lang.ClassNotFoundException: org.apache.spark.ml.fpm.FPGrowth
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at sparklyr.StreamHandler.handleMethodCall(stream.scala:106)
        at sparklyr.StreamHandler.read(stream.scala:61)
        at sparklyr.BackendHandler$$anonfun$channelRead0$1.apply$mcV$sp(handler.scala:58)
        at scala.util.control.Breaks.breakable(Breaks.scala:38)
        at sparklyr.BackendHandler.channelRead0(handler.scala:38)
        at sparklyr.BackendHandler.channelRead0(handler.scala:14)
        at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:367)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:353)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:346)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:367)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:353)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:346)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:367)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:353)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:346)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:367)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:353)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:652)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:575)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:489)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:451)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Thread.java:748)

谢谢你

标签： javaapache-spark