apache-spark - (Windows 10) 例外:Java 网关进程在发送其端口号之前退出
问题描述
明白这是一个经常被问到的问题,但我已经设置了我的环境和东西。(附图)。运行 Windows 10 需要 Pyspark 作为课程模块。请帮忙...
运行后报错:
import pyspark
from pyspark.sql import SparkSession
spark = SparkSession.builder.config("spark-master","local").getOrCreate()
这是回溯:
Exception Traceback (most recent call last)
<ipython-input-3-12e393ff7f76> in <module>
1 import pyspark
2 from pyspark.sql import SparkSession
----> 3 spark = SparkSession.builder.config("spark-master","local").getOrCreate()
C:\opt\spark\spark-2.4.6-bin-hadoop2.7\python\pyspark\sql\session.py in getOrCreate(self)
171 for key, value in self._options.items():
172 sparkConf.set(key, value)
--> 173 sc = SparkContext.getOrCreate(sparkConf)
174 # This SparkContext may be an existing one.
175 for key, value in self._options.items():
C:\opt\spark\spark-2.4.6-bin-hadoop2.7\python\pyspark\context.py in getOrCreate(cls, conf)
365 with SparkContext._lock:
366 if SparkContext._active_spark_context is None:
--> 367 SparkContext(conf=conf or SparkConf())
368 return SparkContext._active_spark_context
369
C:\opt\spark\spark-2.4.6-bin-hadoop2.7\python\pyspark\context.py in __init__(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, gateway, jsc, profiler_cls)
131 " note this option will be removed in Spark 3.0")
132
--> 133 SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
134 try:
135 self._do_init(master, appName, sparkHome, pyFiles, environment, batchSize, serializer,
C:\opt\spark\spark-2.4.6-bin-hadoop2.7\python\pyspark\context.py in _ensure_initialized(cls, instance, gateway, conf)
314 with SparkContext._lock:
315 if not SparkContext._gateway:
--> 316 SparkContext._gateway = gateway or launch_gateway(conf)
317 SparkContext._jvm = SparkContext._gateway.jvm
318
C:\opt\spark\spark-2.4.6-bin-hadoop2.7\python\pyspark\java_gateway.py in launch_gateway(conf)
44 :return: a JVM gateway
45 """
---> 46 return _launch_gateway(conf)
47
48
C:\opt\spark\spark-2.4.6-bin-hadoop2.7\python\pyspark\java_gateway.py in _launch_gateway(conf, insecure)
106
107 if not os.path.isfile(conn_info_file):
--> 108 raise Exception("Java gateway process exited before sending its port number")
109
110 with open(conn_info_file, "rb") as info:
Exception: Java gateway process exited before sending its port number
<img src="https://i.stack.imgur.com/nBulL.png" alt="环境设置" />
解决方案
推荐阅读
- python - LSTM 分类器的准确率低
- firebase - Firebase 分析每小时粒度报告
- sharepoint - 使用 CSOM 将 AD 组添加到 Sharepoint Online 组
- javascript - 为什么传递道具无限循环会使我的应用程序崩溃并且甚至不更新道具?
- elasticsearch - 快速引导大量分层数据的策略以及在任何记录发生更改时更新 Elasticsearch 中单个分层 json 文档的方法
- realm - Realm Cocoa:潜在的线程/通知问题(或者我从文档中遗漏了一些东西)
- c# - 尝试从 android 设备 C# Xamarin 建立蓝牙连接时出错
- python - 代码中断迭代 - 仅打印第一项
- redirect - 仅重定向到裸域
- sql - 创建查询以从 B 和 C 为 A 中的每个项目返回 val