时间戳

首页 > 解决方案 > PySpark 未启动 - Windows 10

pyspark - PySpark 未启动 - Windows 10

问题描述

我正在尝试在 Windows 10 pro 机器上为 Python 设置 Spark。但是，执行以下步骤后：

当我尝试通过命令提示符启动 pyspark 时，会输出以下文本，此后没有任何反应 - 也没有错误？

(base) C:\Spark\bin>pyspark
Python 3.7.6 (default, Jan  8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32 Type "help", "copyright", "credits" or "license" for more information. 
20/08/03 07:49:58 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

最终 1 + 小时后打印此错误：

Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\socket.py", line 589, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

在处理上述异常的过程中，又出现了一个异常：

Traceback (most recent call last):
  File "C:\Spark\python\pyspark\shell.py", line 41, in <module>
    spark = SparkSession._create_shell_session()
  File "C:\Spark\python\pyspark\sql\session.py", line 573, in _create_shell_session
    return SparkSession.builder\
  File "C:\Spark\python\pyspark\sql\session.py", line 173, in getOrCreate
    sc = SparkContext.getOrCreate(sparkConf)
  File "C:\Spark\python\pyspark\context.py", line 367, in getOrCreate
    SparkContext(conf=conf or SparkConf())
  File "C:\Spark\python\pyspark\context.py", line 136, in __init__
    conf, jsc, profiler_cls)
  File "C:\Spark\python\pyspark\context.py", line 198, in _do_init
    self._jsc = jsc or self._initialize_context(self._conf._jconf)
  File "C:\Spark\python\pyspark\context.py", line 306, in _initialize_context
    return self._jvm.JavaSparkContext(jconf)
  File "C:\Spark\python\lib\py4j-0.10.7-src.zip\py4j\java_gateway.py", line 1523, in __call__
  File "C:\Spark\python\lib\py4j-0.10.7-src.zip\py4j\java_gateway.py", line 985, in send_command
  File "C:\Spark\python\lib\py4j-0.10.7-src.zip\py4j\java_gateway.py", line 1152, in send_command
  File "C:\Program Files\Python37\lib\socket.py", line 589, in readinto
    return self._sock.recv_into(b)

标签： pyspark

解决方案

推荐阅读