时间戳

首页 > 解决方案 > 使用已编译的 pyc 模块的 setup.py 的 ModuleNotFoundError

python - 使用已编译的 pyc 模块的 setup.py 的 ModuleNotFoundError

问题描述

我通常可以导入已编译的.pyc模块以及.py，但是当尝试使用 打包一个简单的项目时setup.py，我得到了已编译模块的ModuleNotFoundError异常。.pyc因为这仅在使用时才会发生setup.py，否则工作正常，我不知道我是否应该做一些事情来setup.py完成这项工作。

项目结构目前是这样的：

proj
├── FAILING.pyc
├── __init__.py
├── aux/
│   ├── __init__.py
│   └── aux.c
└── main.py

和setup.py：

from setuptools import setup, Extension, find_packages

DISTNAME = 'proj'

INSTALL_REQUIRES = [
        'cython>=0.29.13',
        'numpy>=1.16.4'
]
PYTHON_REQUIRES = '>=3.6'

ENTRY_POINTS = {
    'console_scripts': ['proj = proj.main:main']
}

def setup_extensions(metadata):
    ext_modules = [Extension('proj.aux.aux', sources=['proj/aux/aux.c'])]
    metadata['ext_modules'] = ext_modules

def setup_package():
    metadata = dict(
        name=DISTNAME,
        version='0.1',
        package_dir={'': '.'},
        packages=find_packages(),
        entry_points=ENTRY_POINTS,
        python_requires=PYTHON_REQUIRES,
        install_requires=INSTALL_REQUIRES,
        zip_safe=False,
    )
    setup_extensions(metadata)
    setup(**metadata)


if __name__ == '__main__':
    setup_package()

main.py: _

#!/usr/bin/env python3

import proj.aux.aux as aux
import proj.FAILING

def main():
    print('Hello World')

如果我只是尝试FAILING.pyc在 repl 上导入，一切都会按预期工作：

>>> import FAILING
>>>

但是，如果我先运行python3 setup.py intall然后调用，则会proj收到以下错误：

$ proj
Traceback (most recent call last):
  File "/path/to/bin/proj", line 11, in <module>
    load_entry_point('proj==0.1', 'console_scripts', 'proj')()
  File "/path/to/lib/python3.8/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/path/to/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2852, in load_entry_point
    return ep.load()
  File "/path/to/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load
    return self.resolve()
  File "/path/to/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/path/to/lib/python3.8/site-packages/proj-0.1-py3.8-macosx-10.14-x86_64.egg/proj/main.py", line 4, in <module>
    import proj.FAILING
ModuleNotFoundError: No module named 'proj.FAILING'

我也在 virtualenv 环境中运行它，尽管我猜这与错误无关。

我做错了什么，或者我需要改变什么才能完成这项工作？

标签： pythonpython-3.xpython-importsetup.pypyc

解决方案

这是一个小演示，它构建了一个包含 .pyc 文件的源代码分发和轮子

请注意，我已经从您的示例中删除了大部分内容，因为 cython 的内容与您的问题无关

set -euxo pipefail

rm -rf dist testpkg setup.py

cat > setup.py <<EOF
from setuptools import setup

setup(
    name='foo',
    version='1',
    packages=['testpkg'],
    package_data={'testpkg': ['*.pyc']},
)
EOF

mkdir testpkg
touch testpkg/__init__.py
echo 'print("hello hello world")' > testpkg/mod.py

python3 -m compileall -b testpkg/mod.py
rm testpkg/mod.py

python3 setup.py sdist bdist_wheel
tar --list -f dist/*.tar.gz
unzip -l dist/*.whl

关于 setup.py 有几点需要注意：

  • 我包含在包含文件packages的包中.pyc——我本可以使用它setuptools.find_packages，但这更简单
  • 该.pyc文件包含为package_data-- 默认情况下，pyc文件未打包，因为它们通常是剩余的构建工件
  • -b我需要使用以下标志将 pyc 编译到旧位置python3 -m compileall
  • 即使是“已编译”的 pyc 文件也不会混淆实际代码，它可以使用dis例如恢复 - 当您在这里谈论“已编译”时，它只是意味着它已被转换为（仍然相对较高级别的）python 字节码

从源代码分发或轮子，您可以安装包。

例如运行脚本：

$ bash t.sh
+ rm -rf dist testpkg setup.py
+ cat
+ mkdir testpkg
+ touch testpkg/__init__.py
+ echo 'print("hello hello world")'
+ python3 -m compileall -b testpkg/mod.py
Compiling 'testpkg/mod.py'...
+ rm testpkg/mod.py
+ python3 setup.py sdist bdist_wheel
running sdist
running egg_info
writing foo.egg-info/PKG-INFO
writing dependency_links to foo.egg-info/dependency_links.txt
writing top-level names to foo.egg-info/top_level.txt
reading manifest file 'foo.egg-info/SOURCES.txt'
writing manifest file 'foo.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md

running check
warning: check: missing required meta-data: url

warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

creating foo-1
creating foo-1/foo.egg-info
creating foo-1/testpkg
copying files to foo-1...
copying setup.py -> foo-1
copying foo.egg-info/PKG-INFO -> foo-1/foo.egg-info
copying foo.egg-info/SOURCES.txt -> foo-1/foo.egg-info
copying foo.egg-info/dependency_links.txt -> foo-1/foo.egg-info
copying foo.egg-info/top_level.txt -> foo-1/foo.egg-info
copying testpkg/__init__.py -> foo-1/testpkg
copying testpkg/mod.pyc -> foo-1/testpkg
Writing foo-1/setup.cfg
creating dist
Creating tar archive
removing 'foo-1' (and everything under it)
running bdist_wheel
running build
running build_py
copying testpkg/__init__.py -> build/lib/testpkg
copying testpkg/mod.pyc -> build/lib/testpkg
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/testpkg
copying build/lib/testpkg/__init__.py -> build/bdist.linux-x86_64/wheel/testpkg
copying build/lib/testpkg/mod.pyc -> build/bdist.linux-x86_64/wheel/testpkg
running install_egg_info
Copying foo.egg-info to build/bdist.linux-x86_64/wheel/foo-1-py3.8.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/foo-1.dist-info/WHEEL
creating 'dist/foo-1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'testpkg/__init__.py'
adding 'testpkg/mod.pyc'
adding 'foo-1.dist-info/METADATA'
adding 'foo-1.dist-info/WHEEL'
adding 'foo-1.dist-info/top_level.txt'
adding 'foo-1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
+ tar --list -f dist/foo-1.tar.gz
foo-1/
foo-1/PKG-INFO
foo-1/foo.egg-info/
foo-1/foo.egg-info/PKG-INFO
foo-1/foo.egg-info/SOURCES.txt
foo-1/foo.egg-info/dependency_links.txt
foo-1/foo.egg-info/top_level.txt
foo-1/setup.cfg
foo-1/setup.py
foo-1/testpkg/
foo-1/testpkg/__init__.py
foo-1/testpkg/mod.pyc
+ unzip -l dist/foo-1-py3-none-any.whl
Archive:  dist/foo-1-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2021-02-17 22:27   testpkg/__init__.py
      136  2021-02-17 22:27   testpkg/mod.pyc
      163  2021-02-17 22:28   foo-1.dist-info/METADATA
       92  2021-02-17 22:28   foo-1.dist-info/WHEEL
        8  2021-02-17 22:27   foo-1.dist-info/top_level.txt
      408  2021-02-17 22:28   foo-1.dist-info/RECORD
---------                     -------
      807                     6 files

之后，我可以安装这个包并使用它：

$ mkdir t
$ cd t
$ virtualenv venv
...
$ . venv/bin/activate
$ pip install ../dist/foo-1-py3-none-any.whl
...
$ python3 -c 'import testpkg.mod'
hello hello world

推荐阅读