python-3.x - 如何修复 gitlab ci 中长时间运行的命令“python setup.py test”？

Question

我正在尝试使用 gitlab-ci/cd 自动测试我的项目。因此我想使用“python setup.py test”命令。但它在 gitlab 管道中运行了大约半小时。

在我的本地机器上，准备和测试在几秒钟内运行成功。但是在gitlab runner上，安装需求需要半个多小时。我试图修改我的 .gitlab-ci.yml 和 setup.py 文件来解决这个问题。例如，我在 setup.py 文件中添加了 install_requires 行。根据长时间停的地方，应该和pandas需求有关。

这是我当前的 .gitlab-ci.yml：

image: python:3.6

stages:
  - test
  - release

before_script:
  - apk add git openssl-dev build-base libffi-dev libxml2-dev libxslt-dev python-dev
  - pip install -U setuptools

test:
  stage: test
  script:
    - echo "Running Tests"
    - python setup.py test
    - echo "Tests finished successfully"
  tags:
    - xyz
  variables:
    GIT_STRATEGY: clone
#...

这是我的 setup.py：

import sys
from setuptools import setup


def setup_package():
    needs_sphinx = {'build_sphinx', 'upload_docs'}.intersection(sys.argv)
    sphinx = ['sphinx'] if needs_sphinx else []
    setup(setup_requires=['six', 'pyscaffold>=2.5a0,<2.6a0'] + sphinx,
          use_pyscaffold=True,
          install_requires=['requests == 2.18', 'pandas == 0.23', #...]
    )


if __name__ == "__main__":
    setup_package()

[...显示后面有更多代码，这可能是不相关的。]

gitlab-ci中耗时较长的部分代码很多，所以我尝试贴出重要的几行：

$ python setup.py test
running test
Searching for requests
Reading https://pypi.org/simple/requests/
Downloading https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl#sha256=7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b
Best match: requests 2.21.0
Processing requests-2.21.0-py2.py3-none-any.whl
Installing requests-2.21.0-py2.py3-none-any.whl to /builds/analytics/data_utils/.eggs
writing requirements to /builds/analytics/data_utils/.eggs/requests-2.21.0-py3.6.egg/EGG-INFO/requires.txt

Installed /builds/analytics/data_utils/.eggs/requests-2.21.0-py3.6.egg
Searching for pandas
Reading https://pypi.org/simple/pandas/
Downloading https://files.pythonhosted.org/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz#sha256=4f919f409c433577a501e023943e582c57355d50a724c589e78bc1d551a535a2
Best match: pandas 0.24.2
Processing pandas-0.24.2.tar.gz
Writing /tmp/easy_install-au2bizhf/pandas-0.24.2/setup.cfg
Running pandas-0.24.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-au2bizhf/pandas-0.24.2/egg-dist-tmp-76fg254v
/bin/sh: svnversion: not found
non-existing path in 'numpy/distutils': 'site.cfg'
Could not locate executable gfortran
Could not locate executable f95
Could not locate executable ifort
Could not locate executable ifc
Could not locate executable lf95
Could not locate executable pgfortran
Could not locate executable f90
Could not locate executable f77
Could not locate executable fort
Could not locate executable efort
Could not locate executable efc
Could not locate executable g77
Could not locate executable g95
Could not locate executable pathf95
Could not locate executable nagfor
don't know how to compile Fortran code on platform 'posix'
_configtest.c:1:5: warning: conflicting types for built-in function 'exp' [-Wbuiltin-declaration-mismatch]
 int exp (void);
     ^~~

一百多个警告

File: build/src.linux-x86_64-3.6/numpy/core/include/numpy/config.h
#define HAVE_ENDIAN_H 1
#define SIZEOF_PY_INTPTR_T 8
#define SIZEOF_OFF_T 8
#define SIZEOF_PY_LONG_LONG 8
#define MATHLIB 
...

In file included from /tmp/easy_install-au2bizhf/pandas-0.24.2/.eggs/numpy-1.16.2-py3.6-linux-x86_64.egg/numpy/core/include/numpy/ndarraytypes.h:1822,
                 from /tmp/easy_install-au2bizhf/pandas-0.24.2/.eggs/numpy-1.16.2-py3.6-linux-x86_64.egg/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /tmp/easy_install-au2bizhf/pandas-0.24.2/.eggs/numpy-1.16.2-py3.6-linux-x86_64.egg/numpy/core/include/numpy/arrayobject.h:4,
                 from pandas/_libs/src/ujson/python/JSONtoObj.c:44:
/tmp/easy_install-au2bizhf/pandas-0.24.2/.eggs/numpy-1.16.2-py3.6-linux-x86_64.egg/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it with " \
  ^~~~~~~
In file included from pandas/_libs/tslibs/src/datetime/np_datetime.c:24:
/usr/local/include/python3.6m/datetime.h:200:25: warning: 'PyDateTimeAPI' defined but not used [-Wunused-variable]
 static PyDateTime_CAPI *PyDateTimeAPI = NULL;
                         ^~~~~~~~~~~~~
In file included from pandas/_libs/tslibs/src/datetime/np_datetime.h:25,
                 from pandas/_libs/tslibs/src/datetime/np_datetime_strings.c:37:
/usr/local/include/python3.6m/datetime.h:200:25: warning: 'PyDateTimeAPI' defined but not used [-Wunused-variable]
 static PyDateTime_CAPI *PyDateTimeAPI = NULL;
                         ^~~~~~~~~~~~~
UPDATING build/lib.linux-x86_64-3.6/pandas/_version.py
set build/lib.linux-x86_64-3.6/pandas/_version.py to '0.24.2'
creating /builds/analytics/data_utils/.eggs/pandas-0.24.2-py3.6-linux-x86_64.egg
Extracting pandas-0.24.2-py3.6-linux-x86_64.egg to /builds/analytics/data_utils/.eggs

Installed /builds/analytics/data_utils/.eggs/pandas-0.24.2-py3.6-linux-x86_64.egg

最后，工作通过了，但需要半个多小时。对于如何加快处理这项工作的任何帮助，我将不胜感激。

标签： python-3.xpython-sphinxgitlab-ci