时间戳

首页 > 解决方案 > Dask Distributed.core - 错误 - “元组”对象不支持项目分配

python - Dask Distributed.core - 错误 - “元组”对象不支持项目分配

问题描述

我在我的项目中使用 Dask 和 cython，我在向客户端注册后调用 cython 代码并将获得的结果从 cython 代码收集到我的 dask-python 代码。当我使用 processes=True 创建一个集群时，它工作正常。但是，一旦我写出 processes=False，它就会给我以下错误：

 distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/core.py", line 555, in handle_stream
    msgs = await comm.read()
  File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 549, in nested_deserialize
    return replace_inner(x)
  File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 541, in replace_inner
    x[k] = replace_inner(v)
  File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 534, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 388, in deserialize
    return loads(header, frames)
  File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py", line 83, in pickle_loads
    buffers[i] = buf.cast(mv.format)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment

我正在使用以下代码片段：

from dask.distributed import Client, LocalCluster,wait
cluster=LocalCluster(processes=False)
client=Client(cluster)
client.register_worker_callbacks(init_pyx)  ## init_pyx is a function by which worker register to cython code
df = dd.read_csv(path_to_csv_file)
processed_df=df.map_partitions(lambda part: handle_partition(part),meta=meta)  #handle partition is function which uses cython code to preprocess each partition of dataframe, meta is metadata related to preproceesed part
client.close()

另外，早些时候我遇到了与内存视图相关的问题，但是当我重新安装 dask 时，内存视图错误消失了。现在出现了一个与“TypeError: 'tuple' object does not support item assignment”相关的新错误。

我也是Dask的新手，如果有人知道如何解决它，请在这方面帮助我。提前致谢。

标签： pythondaskdask-distributeddask-delayeddask-dataframe

解决方案

推荐阅读