windows - Pycharm error while running Pyspark code on windows

Question

WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). 20/05/21 12:55:53 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.

标签： windowspysparkpycharm

Answer 1

这只是一个警告，您仍然可以运行 pyspark 代码。从 pycharm 你必须使用 spark-submit 来执行你的代码。当你想使用 REPL 时使用 pyspark。