时间戳

首页 > 解决方案 > Python 的多处理器和最近的 Win 10 更新之间是否存在内存管理或分页文件大小的已知错误？

python - Python 的多处理器和最近的 Win 10 更新之间是否存在内存管理或分页文件大小的已知错误？

问题描述

我有一个使用多处理器库的 Python 脚本，它运行良好。然后第二天我决定再次运行它，但它给出了内存错误，并且页面文件太小错误。我能想到的唯一改变的是我的硬盘空间和对 Windows 的更新。我认为这可能与 Windows 更新冲突有关，因此我卸载了 Python 3.8.3 并升级到 3.9 并更新了所有依赖包以及 C++ 可分发包。我尝试的一件事是将工人人数减少了另外 6 名工人。它有效，但显然速度较慢。我通常与总共 -4 名工人一起工作，这没有造成任何问题。我什至尝试在所有其他应用程序关闭和网络禁用的情况下运行，但问题仍然存在于 -4 核。虽然适用于 -10 个内核。似乎发生了一些事情，需要脚本使用更多内存，这解释了为什么减少流程实例的数量是有效的。但这是导致这种情况的错误吗？

任何帮助，将不胜感激。谢谢！

  Traceback (most recent call last):
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\__init__.py", line 30, in <module>
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main
    from pandas._libs import hashtable as _hashtable, lib as _lib, tslib as _tslib
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\_libs\__init__.py", line 13, in <module>
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 125, in _main
    from pandas._libs.interval import Interval
  File "pandas\_libs\interval.pyx", line 1, in init pandas._libs.interval
    prepare(preparation_data)
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare
    prepare(preparation_data)
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
  File "pandas\_libs\hashtable.pyx", line 1, in init pandas._libs.hashtable
    main_content = runpy.run_path(main_path,
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 268, in run_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 268, in run_path
  File "pandas\_libs\missing.pyx", line 1, in init pandas._libs.missing
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 97, in _run_module_code
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\_libs\tslibs\__init__.py", line 30, in <module>
    exec(code, run_globals)
    exec(code, run_globals)
  File "C:\Users\david\Documents\GitHub\nimbleheart\Modules\STRATTEST_MULTITRIAL_MASTER.py", line 18, in <module>
  File "C:\Users\david\Documents\GitHub\nimbleheart\Modules\STRATTEST_MULTITRIAL_MASTER.py", line 18, in <module>
    from .conversion import OutOfBoundsTimedelta, localize_pydatetime
  File "pandas\_libs\tslibs\conversion.pyx", line 58, in init pandas._libs.tslibs.conversion
    import pandas as pd
    import pandas as pd
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\__init__.py", line 52, in <module>
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\__init__.py", line 52, in <module>
  File "pandas\_libs\tslibs\parsing.pyx", line 1, in init pandas._libs.tslibs.parsing
    from pandas.core.api import (
    from pandas.core.api import (
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\api.py", line 15, in <module>
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\api.py", line 29, in <module>
    from pandas.core.arrays import Categorical
  File "pandas\_libs\tslibs\offsets.pyx", line 1, in init pandas._libs.tslibs.offsets
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\arrays\__init__.py", line 10, in <module>
    from pandas.core.groupby import Grouper, NamedAgg
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\groupby\__init__.py", line 1, in <module>
    from pandas.core.arrays.interval import IntervalArray
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\arrays\interval.py", line 43, in <module>
    from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\groupby\generic.py", line 57, in <module>
  File "pandas\_libs\tslibs\timestamps.pyx", line 47, in init pandas._libs.tslibs.timestamps
    from pandas.core.indexes.base import ensure_index
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\indexes\base.py", line 31, in <module>
    from pandas.core.aggregation import (
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\aggregation.py", line 27, in <module>
    from pandas.core.dtypes import concat as _concat
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\dtypes\concat.py", line 25, in <module>
    from pandas.core.series import FrameOrSeriesUnion, Series
ImportError: DLL load failed while importing fields: The paging file is too small for this operation to complete.
  File "C:\Users\david\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\series.py", line 68, in <module>

The above exception was the di

标签： pythonpython-3.xpandaswindowsmultiprocessing

解决方案

没有我知道的。Windows 上的 Panda 1.0.3（我上次使用它时）存在一些问题，这些问题取决于 DLL 位置/回退等。我会尝试为 Visual Studio 2015、2017 和 2019 安装最新的 Microsoft Visual C++ Redistributable并查看如果可行（您已经这样做了，只需重复我采取的步骤）。可能是 Windows 更新删除/更新了一些东西......

如果您已经安装了最新版本，您也可以尝试将 concrt140.dll 和 msvcp140.dll 从 C:\Windows\System32 复制到 pandas_libs\windows 文件夹。

最后的手段是将 Pandas 和/或 Python 恢复为已知的工作副本（对我来说是 1.0.1 和 python 3.7）

推荐阅读