python - 在 Windows 上使用 MKL 编译 NumPy

问题描述

我一直在尝试使用 MSVC 编译器和英特尔 MKL 在 Windows 10 上从源代码编译 NumPy。我正在使用 Microsoft Visual Studio 2019 (16.8.4) 和 Intel MKL 2017.8.275 运行 Windows 10.0.18363。

我设法使用最新的 Python 和 NumPy 以最小的设置重现了这个问题。

下载最新的 Python (3.9.1) 和最新的 NumPy (1.20.1) 源代码。
打开 VS 命令提示符，解压 Python 源代码，构建PCbuild\build.bat
运行mklvars.bat intel64以设置正确的环境变量。

将 Intel 编译器（ifort 需要）添加到 PATH：

set PATH=C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64;%PATH%

创建一个虚拟环境，从 Python 构建中复制一些文件并激活虚拟环境：

copy Python\PCbuild\amd64\python39.dll venv\Scripts
copy Python\PC\pyconfig.h venv\Include

从源代码构建 NumPy 并安装：pip install . -v
尝试导入 NumPy：python -c "import numpy"

错误信息

Traceback (most recent call last):
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 22, in <module>
    from . import multiarray
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\multiarray.py", line 12, in <module>
    from . import overrides
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\__init__.py", line 145, in <module>
    from . import core
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "C:\cygwin\home\user\numpy_clean_env\venv\Scripts\python.exe"
  * The NumPy version is: "1.20.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.

其他信息

MKL 库在编译期间被拾取，因为它返回：

FOUND:
        libraries = ['mkl_rt']
        library_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib\\intel64']
        define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
        include_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\include', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib']

我尝试使用 Dependencies（较新版本的 Dependency Walker）分析 DLL 解析，但似乎 MKL DLL 加载正常。有一些 DLL 显示为未正确加载，但据我了解，这是由 Windows API 集的检查软件限制引起的，而不是由该 DLL 的实际问题引起的，所以我认为系统设置正确。以下是NOT_FOUND在分析时导致依赖关系的唯一 DLL 列表_multiarray_umath.pyd：

EMCLIENT.dll (NOT_FOUND) :
HvsiFileTrust.dll (NOT_FOUND) :
UpdateAPI.dll (NOT_FOUND) :
api-ms-win-core-comm-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-ansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-stringansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-winrt-string-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-0.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-1.dll (NOT_FOUND) :
api-ms-win-coreui-secruntime-l1-1-0.dll (NOT_FOUND) :
ext-ms-mf-pal-l2-1-0.dll (NOT_FOUND) :
ext-ms-onecore-appmodel-emclient-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-defaultdiscovery-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-orientation-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-shellchromeapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-deployment-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-usercontext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-viewscalefactor-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-audiocore-pal-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-com-suspendresiliency-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-core-winrt-remote-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dwmapi-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dxcore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-networking-wlanstorage-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-2.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-3.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-4.dll (NOT_FOUND) :
ext-ms-win-ras-tapi32-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ro-typeresolution-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-input-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-private-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-capauthz-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-chambers-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-knownfolderext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-shell32-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-shell-shlwapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-tabbedtitlebar-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellcom-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellfolders-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ui-viewmanagement-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-uiacore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-wer-xbox-l1-1-0.dll (NOT_FOUND) :
ext-ms-win32-subsystem-query-l1-1-0.dll (NOT_FOUND) :
ext-ms-windowscore-deviceinfo-l1-1-0.dll (NOT_FOUND) :

我还尝试按照本指南进行操作，并且可以确认os.path.exists('_multiarray_umath.pyd')返回True，但ctypes.WinDLL('_multiarray_umath.pyd')无法加载 DLL（或其依赖项之一）。

工作示例

与上述步骤相同，但没有执行mklvars.bat，因此没有链接到英特尔 MKL。

过去，此设置用于 Python 3.6、VS2015 和类似版本的英特尔 MKL。我可能遗漏了一些东西，任何建议都非常受欢迎。

标签： pythonnumpyvisual-c++dllintel-mkl

python - 在 Windows 上使用 MKL 编译 NumPy - DLL 加载失败

问题描述

错误信息

其他信息

工作示例

解决方案

推荐阅读