python - Hadoop 的 Python 3 不支持子列表参数

Question

我在为 (lambda (x, (y, z)) 解决此问题时遇到问题：[(dest, z/len(y))

Python 3.xPylance 中不支持错误消息子列表参数

仅位置参数分隔符不允许作为第一个参数

  print "Run 30 Iterations"
for i in range(1, 30):
    print "Number of Iterations"
    print i
    JoinRDD = AdjList3.join(PageRankValues)
    print "join results"
    print JoinRDD.collect()
    contributions = JoinRDD.flatMap(lambda (x, (y, z)) : [(dest, z/len(y)) for dest in y])  # 4. Replace the lambda function with yours
    print "contributions"
    print contributions.collect()
    accumulations = contributions.reduceByKey(lambda x, y : x + y)  # 5. Replace the lambda function with yours
    print "accumulations"
    print accumulations.collect()
    PageRankValues = accumulations.mapValues(lambda v : 0.85 * v + 0.15 / float(nNumOfNodes))  # 6. Replace the lambda function with yours
    print "PageRankValues"
    print PageRankValues.collect() 

标签： pythonapache-sparkhadooppyspark

Answer 1

Python 3 不支持(x, (y,z))where(y,z)将是 sublist 参数。

您可以将此行重写为

contributions = JoinRDD.flatMap(lambda x, yz : [(dest, yz[1]/len(yz[0])) for dest in yz[0]]) 

让我知道这是否适合您。