gitlab-ci/cd を使用してプロジェクトのテストを自動化しようとしています。したがって、「python setup.py test」コマンドを使用したいと思います。しかし、gitlab パイプラインでは約 30 分実行されています。
私のローカル マシンでは、準備とテストが数秒で正常に実行されています。しかし、gitlab ランナーでは、要件のインストールに 30 分以上かかります。問題を修正するために、.gitlab-ci.yml と setup.py ファイルを変更しようとしました。たとえば、setup.py ファイルに install_requires 行を追加しました。長時間停止する場所によると、パンダの要件と関係があるはずです。
ここに私の現在の .gitlab-ci.yml があります:
image: python:3.6
stages:
- test
- release
before_script:
- apk add git openssl-dev build-base libffi-dev libxml2-dev libxslt-dev python-dev
- pip install -U setuptools
test:
stage: test
script:
- echo "Running Tests"
- python setup.py test
- echo "Tests finished successfully"
tags:
- xyz
variables:
GIT_STRATEGY: clone
#...
これが私のsetup.pyです：
import sys
from setuptools import setup
def setup_package():
needs_sphinx = {'build_sphinx', 'upload_docs'}.intersection(sys.argv)
sphinx = ['sphinx'] if needs_sphinx else []
setup(setup_requires=['six', 'pyscaffold>=2.5a0,<2.6a0'] + sphinx,
use_pyscaffold=True,
install_requires=['requests == 2.18', 'pandas == 0.23', #...]
)
if __name__ == "__main__":
setup_package()
[...さらにコードが続くことを表示しますが、これはおそらく関連性がありません。]
gitlab-ci で時間がかかる部分はコードが多いので、重要な行を載せてみます。
$ python setup.py test
running test
Searching for requests
Reading https://pypi.org/simple/requests/
Downloading https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl#sha256=7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b
Best match: requests 2.21.0
Processing requests-2.21.0-py2.py3-none-any.whl
Installing requests-2.21.0-py2.py3-none-any.whl to /builds/analytics/data_utils/.eggs
writing requirements to /builds/analytics/data_utils/.eggs/requests-2.21.0-py3.6.egg/EGG-INFO/requires.txt
Installed /builds/analytics/data_utils/.eggs/requests-2.21.0-py3.6.egg
Searching for pandas
Reading https://pypi.org/simple/pandas/
Downloading https://files.pythonhosted.org/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz#sha256=4f919f409c433577a501e023943e582c57355d50a724c589e78bc1d551a535a2
Best match: pandas 0.24.2
Processing pandas-0.24.2.tar.gz
Writing /tmp/easy_install-au2bizhf/pandas-0.24.2/setup.cfg
Running pandas-0.24.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-au2bizhf/pandas-0.24.2/egg-dist-tmp-76fg254v
/bin/sh: svnversion: not found
non-existing path in 'numpy/distutils': 'site.cfg'
Could not locate executable gfortran
Could not locate executable f95
Could not locate executable ifort
Could not locate executable ifc
Could not locate executable lf95
Could not locate executable pgfortran
Could not locate executable f90
Could not locate executable f77
Could not locate executable fort
Could not locate executable efort
Could not locate executable efc
Could not locate executable g77
Could not locate executable g95
Could not locate executable pathf95
Could not locate executable nagfor
don't know how to compile Fortran code on platform 'posix'
_configtest.c:1:5: warning: conflicting types for built-in function 'exp' [-Wbuiltin-declaration-mismatch]
int exp (void);
^~~
さらに何百もの警告
File: build/src.linux-x86_64-3.6/numpy/core/include/numpy/config.h
#define HAVE_ENDIAN_H 1
#define SIZEOF_PY_INTPTR_T 8
#define SIZEOF_OFF_T 8
#define SIZEOF_PY_LONG_LONG 8
#define MATHLIB
...
In file included from /tmp/easy_install-au2bizhf/pandas-0.24.2/.eggs/numpy-1.16.2-py3.6-linux-x86_64.egg/numpy/core/include/numpy/ndarraytypes.h:1822,
from /tmp/easy_install-au2bizhf/pandas-0.24.2/.eggs/numpy-1.16.2-py3.6-linux-x86_64.egg/numpy/core/include/numpy/ndarrayobject.h:12,
from /tmp/easy_install-au2bizhf/pandas-0.24.2/.eggs/numpy-1.16.2-py3.6-linux-x86_64.egg/numpy/core/include/numpy/arrayobject.h:4,
from pandas/_libs/src/ujson/python/JSONtoObj.c:44:
/tmp/easy_install-au2bizhf/pandas-0.24.2/.eggs/numpy-1.16.2-py3.6-linux-x86_64.egg/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it with " \
^~~~~~~
In file included from pandas/_libs/tslibs/src/datetime/np_datetime.c:24:
/usr/local/include/python3.6m/datetime.h:200:25: warning: 'PyDateTimeAPI' defined but not used [-Wunused-variable]
static PyDateTime_CAPI *PyDateTimeAPI = NULL;
^~~~~~~~~~~~~
In file included from pandas/_libs/tslibs/src/datetime/np_datetime.h:25,
from pandas/_libs/tslibs/src/datetime/np_datetime_strings.c:37:
/usr/local/include/python3.6m/datetime.h:200:25: warning: 'PyDateTimeAPI' defined but not used [-Wunused-variable]
static PyDateTime_CAPI *PyDateTimeAPI = NULL;
^~~~~~~~~~~~~
UPDATING build/lib.linux-x86_64-3.6/pandas/_version.py
set build/lib.linux-x86_64-3.6/pandas/_version.py to '0.24.2'
creating /builds/analytics/data_utils/.eggs/pandas-0.24.2-py3.6-linux-x86_64.egg
Extracting pandas-0.24.2-py3.6-linux-x86_64.egg to /builds/analytics/data_utils/.eggs
Installed /builds/analytics/data_utils/.eggs/pandas-0.24.2-py3.6-linux-x86_64.egg
最後にジョブはパスしますが、30 分以上かかります。このジョブの処理を高速化する方法について何か助けていただければ幸いです。