r - R 和 Hive 之间的连接(在 Spark 上)
问题描述
我正在尝试连接 R 和 Hive (Spark)。在我的桌面(Windows 10,R-3.4.2)上它运行良好,但在 R-server(Linux,R - 3.4.4)上我收到错误:
library(rJava)
library(RJDBC)
driver <- JDBC("org.apache.hive.jdbc.HiveDriver", "~/Drivers/Spark/hive-jdbc-1.2.1-spark2-amzn-0.jar",identifier.quote="`")
url <- "jdbc:hive2://<MyIP>:10001"
conn <- dbConnect(driver, url)
Error in .jcall(drv@jdrv,"Ljava/sql/Connection;", "connect", as.character(url)[1], : java.lang.NoClassDefFoundError: org/apache/http/client/CookieStore
怎么了?
解决方案
我找到了解决方案:
library(rJava)
library(RJDBC)
options(java.parameters = '-Xmx256m')
hadoop_jar_dirs <- c('//home//ubuntu//spark-jdbc//')
clpath <- c()
for (d in hadoop_jar_dirs) {
clpath <- c(clpath, list.files(d, pattern = 'jar', full.names = TRUE))
}
.jinit(classpath = clpath)
.jaddClassPath(clpath)
hive_jdbc_jar <- 'hive-jdbc-1.2.1-spark2-amzn-0.jar'
hive_driver <- 'org.apache.hive.jdbc.HiveDriver'
hive_url <- 'jdbc:hive2://<MyIP>:10001'
drv <- JDBC(hive_driver, hive_jdbc_jar)
conn <- dbConnect(drv, hive_url)
show_databases <- dbGetQuery(conn, "show databases")
show_databases
推荐阅读
- nestjs - When using nested subscribers app will crash (NestJs+TypeOrm+Mysql)
- javascript - Plotly react. The annotation disappears after changing border color
- cluster-computing - Tile38 - Sharding across multiple/disparate instantiations
- excel - Looping through Macro to sort through column of data
- typescript - Isolate cause of Typescript max recursion error with getAwaitedTypeWorker => getAwaitedType => mapType
- .net-core - .Net Core The name 'Host' does not exist
- jquery - 防止用户注册不同的电子邮件域
- reactjs - How to fetch data after a specific Interval with useState, useEffect and useCallback in React with proper dependencies
- php - HTML - 输入字段跳出 html 表格
- python - Will per feature LayerNormalization be exported in Keras Sequential model after training?