scala - Livy Spark 互动环节

Question

我正在尝试使用 livy 创建 spark 交互式会话。我需要在 hdfs 中添加一个类似 jar 的库（请参阅我的代码）。但是会话已经死了，日志在下面。

代码 ：

client = LivyClient('http://sandbox.c4e.kyomei.fr:10500')
session = client.create_session(SessionKind.SPARK , jars = ['hdfs://sandbox-hdp.hortonworks.com:8020/tmp/tsa2-assembly-0.1.jar'] )

日志 ：

> 20/05/09 01:43:48 INFO LineBufferedStream: Exception in thread "main"
> scala.reflect.internal.FatalError: object Predef does not have a
> member classOf                    20/05/09 01:43:48 WARN RSCClient:
> Error stopping RPC.                                                   
> io.netty.util.concurrent.BlockingOperationException:
> DefaultChannelPromise@786d2cd8(uncancellable)                         
> 
>         at io.netty.util.concurrent.DefaultPromise.checkDeadLock(DefaultPromise.java:394)
> 
>         at io.netty.channel.DefaultChannelPromise.checkDeadLock(DefaultChannelPromise.java:157)
> 
>         at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:230)
> ....

> Exception in thread "Thread-32" java.io.IOException: Stream closed    
> 
>         at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170)
> 
>         at java.io.BufferedInputStream.read1(BufferedInputStream.java:283)       
> 
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:345)

> 20/05/09 01:43:48 WARN ContextLauncher: Child process exited with code
20/05/09 01:43:48 ERROR SparkProcApp: job was killed by user  

标签： scalaapache-sparkrpclivyspark-shell