Specify Minimum Python Version as 3.8 in package #1732

oliverholworthy · 2022-12-14T10:38:15Z

Specifes the minimum Python Version as 3.8.
Add specific python versions to the classifiers configuratiuon to indicate the intended Python version compatibility of the package.

indicating the intended Python version compatibility of the library

nvidia-merlin-bot · 2022-12-14T10:56:36Z

Click to view CI Results

GitHub pull request #1732 of commit f80bc65baf840b4d2427f7cfd90610caa6324134, no merge conflicts.
Running as SYSTEM
Setting status of f80bc65baf840b4d2427f7cfd90610caa6324134 to PENDING with url http://merlin-infra1.nvidia.com:8080/job/nvtabular_tests/4966/ and message: 'Build started for merge commit.'
Using context: Jenkins Unit Test Run
Building on the built-in node in workspace /var/jenkins_home/jobs/nvtabular_tests/workspace
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1732/*:refs/remotes/origin/pr/1732/* # timeout=10
 > git rev-parse f80bc65baf840b4d2427f7cfd90610caa6324134^{commit} # timeout=10
Checking out Revision f80bc65baf840b4d2427f7cfd90610caa6324134 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f80bc65baf840b4d2427f7cfd90610caa6324134 # timeout=10
Commit message: "Add Programming Language classifiers to setup.py configuration"
 > git rev-list --no-walk 71de9869c0d9f922c67133270f0f9368aa0d694c # timeout=10
[workspace] $ /bin/bash /tmp/jenkins15497272407053043530.sh
GLOB sdist-make: /var/jenkins_home/workspace/nvtabular_tests/nvtabular/setup.py
test-gpu recreate: /var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu
test-gpu installdeps: pytest, pytest-cov
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu inst: /var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/.tmp/package/1/nvtabular-1.6.0+21.gf80bc65b.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,alembic==1.8.1,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,autopage==0.5.1,awscli==1.27.29,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==22.6.0,bleach==5.0.1,boto3==1.24.75,botocore==1.29.29,Brotli==1.0.9,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,clang==5.0,click==8.1.3,cliff==4.1.0,cloudpickle==2.2.0,cmaes==0.9.0,cmake==3.24.1.1,cmd2==2.4.2,colorama==0.4.4,colorlog==6.7.0,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,horovod==0.26.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-cache==0.4.3,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.8.0,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Mako==1.2.4,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-dataloader==0.0.3,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.5.0+4.g15074ad,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,-e git+https://github.com/NVIDIA-Merlin/NVTabular.git@f80bc65baf840b4d2427f7cfd90610caa6324134#egg=nvtabular,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,optuna==3.0.4,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pbr==5.11.0,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,plotly==5.11.0,pluggy==1.0.0,prettytable==3.5.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyperclip==1.8.2,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-xdist==3.1.0,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.8.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.45,stack-data==0.5.0,starlette==0.20.4,stevedore==4.1.1,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.9.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='854143823'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-qw8fyuya
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-qw8fyuya
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit c2ef3f06ce2caf864d84bc6e92799c4d1392f74a
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (4.64.1)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (2022.3.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (1.3.5)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (1.2.5)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (7.0.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (0.55.1)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (2022.3.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (21.3)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (3.19.5)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (2022.5.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (1.10.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (0.4.3)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (0.12.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (5.4.1)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (1.0.4)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (2.4.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (5.8.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (1.7.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (3.1.2)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (6.1)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (2.0.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (8.1.3)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+15.gc2ef3f0) (65.5.1)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+15.gc2ef3f0) (0.38.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+15.gc2ef3f0) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.9.0+15.gc2ef3f0) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.9.0+15.gc2ef3f0-py3-none-any.whl size=119064 sha256=1e3fbcc76f33357f4b3768e34f064ef7fcf4e96e4d8466f3aa5ed46d1f517f69
  Stored in directory: /tmp/pip-ephem-wheel-cache-y6cm2dh3/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.9.0+15.gc2ef3f0
test-gpu run-test: commands[1] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/dataloader.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/dataloader.git
  Cloning https://github.com/NVIDIA-Merlin/dataloader.git to /tmp/pip-req-build-4kaapiie
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/dataloader.git /tmp/pip-req-build-4kaapiie
  Resolved https://github.com/NVIDIA-Merlin/dataloader.git to commit 42dd30158f279fdb57f051211fe1a36860f48228
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-core>=0.8.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-dataloader==0.0.2+25.g42dd301) (0.9.0+15.gc2ef3f0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (4.64.1)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2022.3.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.3.5)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.2.5)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (7.0.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.55.1)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2022.3.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (21.3)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (3.19.5)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2022.5.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.10.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.4.3)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.12.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (5.4.1)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.0.4)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.4.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (5.8.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.7.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (3.1.2)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (6.1)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.0.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (8.1.3)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (65.5.1)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.38.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (4.0.0)
Building wheels for collected packages: merlin-dataloader
  Building wheel for merlin-dataloader (pyproject.toml): started
  Building wheel for merlin-dataloader (pyproject.toml): finished with status 'done'
  Created wheel for merlin-dataloader: filename=merlin_dataloader-0.0.2+25.g42dd301-py3-none-any.whl size=40768 sha256=039e6a2aac8ace2c3b8ac1423411d00ee7d6b838ca9678d5063f4f3bf4c0659a
  Stored in directory: /tmp/pip-ephem-wheel-cache-_r5avida/wheels/de/f5/d9/251909f4627d2920fb15548f5ffd6daf1bf24c3c56bb4977b1
Successfully built merlin-dataloader
Installing collected packages: merlin-dataloader
  Attempting uninstall: merlin-dataloader
    Found existing installation: merlin-dataloader 0.0.3
    Uninstalling merlin-dataloader-0.0.3:
      Successfully uninstalled merlin-dataloader-0.0.3
Successfully installed merlin-dataloader-0.0.2+25.g42dd301
test-gpu run-test: commands[2] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/models.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/models.git
  Cloning https://github.com/NVIDIA-Merlin/models.git to /tmp/pip-req-build-x9es64c0
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/models.git /tmp/pip-req-build-x9es64c0
  Resolved https://github.com/NVIDIA-Merlin/models.git to commit df84c81f99e98ba46b72e86ee3b28e46e741128a
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-dataloader>=0.0.2 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-models==0.9.0+65.gdf84c81f) (0.0.2+25.g42dd301)
Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-models==0.9.0+65.gdf84c81f) (0.9.0+15.gc2ef3f0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (4.64.1)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2022.3.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.3.5)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.2.5)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (7.0.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.55.1)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2022.3.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (21.3)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (3.19.5)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2022.5.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.10.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.4.3)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.12.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (5.4.1)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.0.4)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.4.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (5.8.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.7.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (3.1.2)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (6.1)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.0.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (8.1.3)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (65.5.1)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.38.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (4.0.0)
Building wheels for collected packages: merlin-models
  Building wheel for merlin-models (pyproject.toml): started
  Building wheel for merlin-models (pyproject.toml): finished with status 'done'
  Created wheel for merlin-models: filename=merlin_models-0.9.0+65.gdf84c81f-py3-none-any.whl size=354723 sha256=5ab59feb1eba37dfd11fe134eef71dcb931a9bc11fa6e3a32c2d20dba730fce7
  Stored in directory: /tmp/pip-ephem-wheel-cache-bvp7g11u/wheels/5a/43/99/d50fe2c33b4f4686db73207ce3865e0d6be6609ffb03abade5
Successfully built merlin-models
Installing collected packages: merlin-models
  Attempting uninstall: merlin-models
    Found existing installation: merlin-models 0.7.0+11.g280956aa4
    Not uninstalling merlin-models at /usr/local/lib/python3.8/dist-packages, outside environment /var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu
    Can't uninstall 'merlin-models'. No files were found to uninstall.
Successfully installed merlin-models-0.9.0+65.gdf84c81f
test-gpu run-test: commands[3] | python -m pytest --cov-report term --cov merlin -rxs tests/unit
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
cachedir: .tox/test-gpu/.pytest_cache
rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml
plugins: anyio-3.5.0, cov-4.0.0, xdist-3.1.0
collected 1435 items / 1 skipped
tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

....                                                                     [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [  9%]

tests/unit/test_triton_inference.py ................................     [ 12%]

tests/unit/examples/test_01-Getting-started.py .                         [ 12%]

tests/unit/examples/test_02-Advanced-NVTabular-workflow.py .             [ 12%]

tests/unit/examples/test_03-Running-on-multiple-GPUs-or-on-CPU.py .      [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 17%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 17%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

.....................................................                    [ 28%]

tests/unit/ops/test_categorify.py ...................................... [ 31%]

........................................................................ [ 36%]

.....................................................                    [ 40%]

tests/unit/ops/test_column_similarity.py ........................        [ 41%]

tests/unit/ops/test_drop_low_cardinality.py ..                           [ 42%]

tests/unit/ops/test_fill.py ............................................ [ 45%]

........                                                                 [ 45%]

tests/unit/ops/test_groupyby.py .......................                  [ 47%]

tests/unit/ops/test_hash_bucket.py .........................             [ 49%]

tests/unit/ops/test_join.py ............................................ [ 52%]

........................................................................ [ 57%]

..................................                                       [ 59%]

tests/unit/ops/test_lambda.py ..........                                 [ 60%]

tests/unit/ops/test_normalize.py ....................................... [ 62%]

..                                                                       [ 63%]

tests/unit/ops/test_ops.py ............................................. [ 66%]

....................                                                     [ 67%]

tests/unit/ops/test_ops_schema.py ...................................... [ 70%]

........................................................................ [ 75%]

........................................................................ [ 80%]

........................................................................ [ 85%]

.......................................                                  [ 88%]

tests/unit/ops/test_reduce_dtype_size.py ..                              [ 88%]

tests/unit/ops/test_target_encode.py .....................               [ 89%]

tests/unit/ops/test_value_count.py ...                                   [ 89%]

tests/unit/workflow/test_cpu_workflow.py ......                          [ 90%]

tests/unit/workflow/test_workflow.py ................................... [ 92%]

..........................................................               [ 96%]

tests/unit/workflow/test_workflow_chaining.py ...                        [ 96%]

tests/unit/workflow/test_workflow_node.py ...........                    [ 97%]

tests/unit/workflow/test_workflow_ops.py ...                             [ 97%]

tests/unit/workflow/test_workflow_schemas.py ........................... [ 99%]

...                                                                      [100%]
=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33

/usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)
.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: 34 warnings

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)
tests/unit/test_dask_nvt.py: 6 warnings

tests/unit/workflow/test_workflow.py: 78 warnings

/var/jenkins_home/.local/lib/python3.8/site-packages/dask/base.py:1282: UserWarning: Running on a single-machine scheduler when a distributed client is active might lead to unexpected results.

warnings.warn(
tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu/lib/python3.8/site-packages/merlin/core/utils.py:431: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(
tests/unit/ops/test_fill.py::test_fill_missing[True-True-parquet]

tests/unit/ops/test_fill.py::test_fill_missing[True-False-parquet]

tests/unit/ops/test_ops.py::test_filter[parquet-0.1-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/pandas/core/indexing.py:1732: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

self._setitem_single_block(indexer, value, name)
tests/unit/ops/test_ops_schema.py: 12 warnings

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(
tests/unit/ops/test_ops_schema.py: 12 warnings

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                Stmts   Miss  Cover
merlin/transforms/init.py           1      1     0%

merlin/transforms/ops/init.py       1      1     0%
TOTAL                                   2      2     0%
=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/io/tests/test_s3.py:14: could not import 'moto': No module named 'moto'

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

========== 1434 passed, 2 skipped, 147 warnings in 1004.20s (0:16:44) ==========

/usr/local/lib/python3.8/dist-packages/coverage/control.py:801: CoverageWarning: No data was collected. (no-data-collected)

self._warn("No data was collected.", slug="no-data-collected")

___________________________________ summary ____________________________________

test-gpu: commands succeeded

congratulations :)

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://github.com/gitapi/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[workspace] $ /bin/bash /tmp/jenkins5986595671577996736.sh

github-actions · 2022-12-14T11:23:29Z

Documentation preview

https://nvidia-merlin.github.io/NVTabular/review/pr-1732

0.6.0 addded the merlin.dag.executors which is used in NVTabular

nvidia-merlin-bot · 2022-12-14T11:54:23Z

Click to view CI Results

GitHub pull request #1732 of commit d1a8198fdd3539510463d04a653390711a110665, no merge conflicts.
Running as SYSTEM
Setting status of d1a8198fdd3539510463d04a653390711a110665 to PENDING with url http://merlin-infra1.nvidia.com:8080/job/nvtabular_tests/4968/ and message: 'Build started for merge commit.'
Using context: Jenkins Unit Test Run
Building on the built-in node in workspace /var/jenkins_home/jobs/nvtabular_tests/workspace
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1732/*:refs/remotes/origin/pr/1732/* # timeout=10
 > git rev-parse d1a8198fdd3539510463d04a653390711a110665^{commit} # timeout=10
Checking out Revision d1a8198fdd3539510463d04a653390711a110665 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f d1a8198fdd3539510463d04a653390711a110665 # timeout=10
Commit message: "Update minimum version of merlin-core from 0.2.0 to 0.6.0"
 > git rev-list --no-walk ed0dfc845a9422b47c73095dfa8e48b3b13d21a1 # timeout=10
[workspace] $ /bin/bash /tmp/jenkins16619325182843187286.sh
GLOB sdist-make: /var/jenkins_home/workspace/nvtabular_tests/nvtabular/setup.py
test-gpu recreate: /var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu
test-gpu installdeps: pytest, pytest-cov
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu inst: /var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/.tmp/package/1/nvtabular-1.6.0+22.gd1a8198f.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,alembic==1.8.1,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,autopage==0.5.1,awscli==1.27.29,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==22.6.0,bleach==5.0.1,boto3==1.24.75,botocore==1.29.29,Brotli==1.0.9,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,clang==5.0,click==8.1.3,cliff==4.1.0,cloudpickle==2.2.0,cmaes==0.9.0,cmake==3.24.1.1,cmd2==2.4.2,colorama==0.4.4,colorlog==6.7.0,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,horovod==0.26.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-cache==0.4.3,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.8.0,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Mako==1.2.4,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-dataloader==0.0.3,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.5.0+4.g15074ad,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,-e git+https://github.com/NVIDIA-Merlin/NVTabular.git@d1a8198fdd3539510463d04a653390711a110665#egg=nvtabular,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,optuna==3.0.4,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pbr==5.11.0,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,plotly==5.11.0,pluggy==1.0.0,prettytable==3.5.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyperclip==1.8.2,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-xdist==3.1.0,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.8.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.45,stack-data==0.5.0,starlette==0.20.4,stevedore==4.1.1,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.9.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='3641660728'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-pno7jqlt
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-pno7jqlt
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit c2ef3f06ce2caf864d84bc6e92799c4d1392f74a
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (7.0.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (1.10.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (0.55.1)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (2022.3.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (4.64.1)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (2022.5.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (1.2.5)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (1.3.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (3.19.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+15.gc2ef3f0) (21.3)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+15.gc2ef3f0) (2022.3.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (0.4.3)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (5.4.1)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (0.12.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (2.2.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (1.2.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (5.8.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (3.1.2)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (2.0.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (2.4.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (6.1)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (8.1.3)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (1.0.4)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (1.7.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+15.gc2ef3f0) (0.38.1)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+15.gc2ef3f0) (65.5.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+15.gc2ef3f0) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.9.0+15.gc2ef3f0) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+15.gc2ef3f0) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.9.0+15.gc2ef3f0) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.9.0+15.gc2ef3f0) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.9.0+15.gc2ef3f0-py3-none-any.whl size=119064 sha256=f40fc462bf333bf2d5bdd100cb444a18adddc57b4873e5cb2c4eb42cc62d8404
  Stored in directory: /tmp/pip-ephem-wheel-cache-g4hvnq6j/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.9.0
    Uninstalling merlin-core-0.9.0:
      Successfully uninstalled merlin-core-0.9.0
Successfully installed merlin-core-0.9.0+15.gc2ef3f0
test-gpu run-test: commands[1] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/dataloader.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/dataloader.git
  Cloning https://github.com/NVIDIA-Merlin/dataloader.git to /tmp/pip-req-build-4xs7ptkc
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/dataloader.git /tmp/pip-req-build-4xs7ptkc
  Resolved https://github.com/NVIDIA-Merlin/dataloader.git to commit 42dd30158f279fdb57f051211fe1a36860f48228
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-core>=0.8.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-dataloader==0.0.2+25.g42dd301) (0.9.0+15.gc2ef3f0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (7.0.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.10.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.55.1)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2022.3.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (4.64.1)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2022.5.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.2.5)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.3.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (3.19.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (21.3)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2022.3.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.4.3)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (5.4.1)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.12.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.2.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.2.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (5.8.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (3.1.2)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.0.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.4.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (6.1)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (8.1.3)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.0.4)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.7.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.38.1)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (65.5.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.8.0->merlin-dataloader==0.0.2+25.g42dd301) (4.0.0)
Building wheels for collected packages: merlin-dataloader
  Building wheel for merlin-dataloader (pyproject.toml): started
  Building wheel for merlin-dataloader (pyproject.toml): finished with status 'done'
  Created wheel for merlin-dataloader: filename=merlin_dataloader-0.0.2+25.g42dd301-py3-none-any.whl size=40768 sha256=6e9a19ab0e5b2a96abd15ee4c2aa5226ddeee94678694ec4db407fcc9f473e92
  Stored in directory: /tmp/pip-ephem-wheel-cache-nh0g3ka9/wheels/de/f5/d9/251909f4627d2920fb15548f5ffd6daf1bf24c3c56bb4977b1
Successfully built merlin-dataloader
Installing collected packages: merlin-dataloader
  Attempting uninstall: merlin-dataloader
    Found existing installation: merlin-dataloader 0.0.3
    Uninstalling merlin-dataloader-0.0.3:
      Successfully uninstalled merlin-dataloader-0.0.3
Successfully installed merlin-dataloader-0.0.2+25.g42dd301
test-gpu run-test: commands[2] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/models.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/models.git
  Cloning https://github.com/NVIDIA-Merlin/models.git to /tmp/pip-req-build-_8sxozg2
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/models.git /tmp/pip-req-build-_8sxozg2
  Resolved https://github.com/NVIDIA-Merlin/models.git to commit df84c81f99e98ba46b72e86ee3b28e46e741128a
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-dataloader>=0.0.2 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-models==0.9.0+65.gdf84c81f) (0.0.2+25.g42dd301)
Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-models==0.9.0+65.gdf84c81f) (0.9.0+15.gc2ef3f0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (7.0.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.10.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.55.1)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2022.3.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (4.64.1)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2022.5.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.2.5)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.3.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (3.19.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (21.3)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2022.3.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.4.3)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (5.4.1)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.12.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.2.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.2.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (5.8.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (3.1.2)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.0.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.4.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (6.1)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (8.1.3)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.0.4)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.7.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.38.1)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (65.5.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-models==0.9.0+65.gdf84c81f) (4.0.0)
Building wheels for collected packages: merlin-models
  Building wheel for merlin-models (pyproject.toml): started
  Building wheel for merlin-models (pyproject.toml): finished with status 'done'
  Created wheel for merlin-models: filename=merlin_models-0.9.0+65.gdf84c81f-py3-none-any.whl size=354723 sha256=bb554cdf4cf2f585b25b075a034d941629b1afa574ec0229c125a2a955196aa9
  Stored in directory: /tmp/pip-ephem-wheel-cache-5nxfz8ye/wheels/5a/43/99/d50fe2c33b4f4686db73207ce3865e0d6be6609ffb03abade5
Successfully built merlin-models
Installing collected packages: merlin-models
  Attempting uninstall: merlin-models
    Found existing installation: merlin-models 0.7.0+11.g280956aa4
    Not uninstalling merlin-models at /usr/local/lib/python3.8/dist-packages, outside environment /var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu
    Can't uninstall 'merlin-models'. No files were found to uninstall.
Successfully installed merlin-models-0.9.0+65.gdf84c81f
test-gpu run-test: commands[3] | python -m pytest --cov-report term --cov merlin -rxs tests/unit
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
cachedir: .tox/test-gpu/.pytest_cache
rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml
plugins: anyio-3.5.0, cov-4.0.0, xdist-3.1.0
collected 1435 items / 1 skipped
tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

....                                                                     [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [  9%]

tests/unit/test_triton_inference.py FFFFFFFFF....................FFF     [ 12%]

tests/unit/examples/test_01-Getting-started.py .                         [ 12%]

tests/unit/examples/test_02-Advanced-NVTabular-workflow.py .             [ 12%]

tests/unit/examples/test_03-Running-on-multiple-GPUs-or-on-CPU.py F      [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 17%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 17%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

.....................................................                    [ 28%]

tests/unit/ops/test_categorify.py ...................................... [ 31%]

........................................................................ [ 36%]

.....................................................                    [ 40%]

tests/unit/ops/test_column_similarity.py ........................        [ 41%]

tests/unit/ops/test_drop_low_cardinality.py ..                           [ 42%]

tests/unit/ops/test_fill.py ............................................ [ 45%]

........                                                                 [ 45%]

tests/unit/ops/test_groupyby.py .......................                  [ 47%]

tests/unit/ops/test_hash_bucket.py .........................             [ 49%]

tests/unit/ops/test_join.py ............................................ [ 52%]

........................................................................ [ 57%]

..................................                                       [ 59%]

tests/unit/ops/test_lambda.py ..........                                 [ 60%]

tests/unit/ops/test_normalize.py ....................................... [ 62%]

..                                                                       [ 63%]

tests/unit/ops/test_ops.py ............................................. [ 66%]

....................                                                     [ 67%]

tests/unit/ops/test_ops_schema.py ...................................... [ 70%]

........................................................................ [ 75%]

........................................................................ [ 80%]

........................................................................ [ 85%]

.......................................                                  [ 88%]

tests/unit/ops/test_reduce_dtype_size.py ..                              [ 88%]

tests/unit/ops/test_target_encode.py .....................               [ 89%]

tests/unit/ops/test_value_count.py ...                                   [ 89%]

tests/unit/workflow/test_cpu_workflow.py ......                          [ 90%]

tests/unit/workflow/test_workflow.py ................................... [ 92%]

..........................................................               [ 96%]

tests/unit/workflow/test_workflow_chaining.py ...                        [ 96%]

tests/unit/workflow/test_workflow_node.py ...........                    [ 97%]

tests/unit/workflow/test_workflow_ops.py ...                             [ 97%]

tests/unit/workflow/test_workflow_schemas.py ........................... [ 99%]

...                                                                      [100%]
=================================== FAILURES ===================================

_____________________________ test_error_handling ______________________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_error_handling0')
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
def test_error_handling(tmpdir):
    df = make_df({"x": np.arange(10), "y": np.arange(10)})

    def custom_transform(col):
        if len(col) == 2:
            raise ValueError("Lets cause some problems")
        return col

    features = ["x", "y"] >> ops.FillMissing() >> ops.Normalize() >> custom_transform
    workflow = nvt.Workflow(features)
    workflow.fit(nvt.Dataset(df))

    model_name = "test_error_handling"
    triton.generate_nvtabular_model(
        workflow, model_name, tmpdir + f"/{model_name}", backend=BACKEND
    )

    with run_triton_server(tmpdir) as client:
        inputs = triton.convert_df_to_triton_input(["x", "y"], df[:2])
        with pytest.raises(tritonclient.utils.InferenceServerException) as exception_info:
            client.infer(model_name, inputs)


      assert "ValueError: Lets cause some problems" in str(exception_info.value)


E           assert 'ValueError: Lets cause some problems' in "[StatusCode.NOT_FOUND] Request for unknown model: 'test_error_handling' is not found"

E            +  where "[StatusCode.NOT_FOUND] Request for unknown model: 'test_error_handling' is not found" = str(InferenceServerException())

E            +    where InferenceServerException() = <ExceptionInfo InferenceServerException() tblen=3>.value
tests/unit/test_triton_inference.py:148: AssertionError

----------------------------- Captured stderr call -----------------------------

I1214 11:41:00.175080 25298 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7fa456000000' with size 268435456

I1214 11:41:00.175823 25298 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864

I1214 11:41:00.178950 25298 model_lifecycle.cc:459] loading: test_error_handling:1

I1214 11:41:05.229592 25298 python_be.cc:1767] TRITONBACKEND_ModelInstanceInitialize: test_error_handling (GPU device 0)

________________ test_tritonserver_inference_string[tensorflow] ________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_tritonserver_inference_st0')

output_model = 'tensorflow'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def test_tritonserver_inference_string(tmpdir, output_model):
    df = make_df({"user": ["aaaa", "bbbb", "cccc", "aaaa", "bbbb", "aaaa"]})
    features = ["user"] >> ops.Categorify()
    workflow = nvt.Workflow(features)


  _verify_workflow_on_tritonserver(


        tmpdir,
        workflow,
        df,
        "test_inference_string",
        output_model,
    )

tests/unit/test_triton_inference.py:158:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...","file_line":1069,"grpc_message":"Request for unknown model: 'test_inference_string' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'test_inference_string' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

----------------------------- Captured stderr call -----------------------------

1214 11:41:06.268191 25434 pb_stub.cc:1016] Non-graceful termination detected.

_________________ test_tritonserver_inference_string[pytorch] __________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_tritonserver_inference_st1')

output_model = 'pytorch'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def test_tritonserver_inference_string(tmpdir, output_model):
    df = make_df({"user": ["aaaa", "bbbb", "cccc", "aaaa", "bbbb", "aaaa"]})
    features = ["user"] >> ops.Categorify()
    workflow = nvt.Workflow(features)


  _verify_workflow_on_tritonserver(


        tmpdir,
        workflow,
        df,
        "test_inference_string",
        output_model,
    )

tests/unit/test_triton_inference.py:158:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...","file_line":1069,"grpc_message":"Request for unknown model: 'test_inference_string' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'test_inference_string' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

________________________ test_large_strings[tensorflow] ________________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_large_strings_tensorflow_0')

output_model = 'tensorflow'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def test_large_strings(tmpdir, output_model):
    strings = ["a" * (2**exp) for exp in range(1, 17)]
    df = make_df({"description": strings})
    features = ["description"] >> ops.Categorify()
    workflow = nvt.Workflow(features)
    workflow.fit(nvt.Dataset(df))


  _verify_workflow_on_tritonserver(


        tmpdir,
        workflow,
        df,
        "test_large_string",
        output_model,
    )

tests/unit/test_triton_inference.py:176:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...l.cc","file_line":1069,"grpc_message":"Request for unknown model: 'test_large_string' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'test_large_string' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

_________________________ test_large_strings[pytorch] __________________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_large_strings_pytorch_0')

output_model = 'pytorch'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def test_large_strings(tmpdir, output_model):
    strings = ["a" * (2**exp) for exp in range(1, 17)]
    df = make_df({"description": strings})
    features = ["description"] >> ops.Categorify()
    workflow = nvt.Workflow(features)
    workflow.fit(nvt.Dataset(df))


  _verify_workflow_on_tritonserver(


        tmpdir,
        workflow,
        df,
        "test_large_string",
        output_model,
    )

tests/unit/test_triton_inference.py:176:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...l.cc","file_line":1069,"grpc_message":"Request for unknown model: 'test_large_string' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'test_large_string' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

____________________ testconcatenate_dataframe[tensorflow] _____________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/testconcatenate_dataframe_tens0')

output_model = 'tensorflow'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def testconcatenate_dataframe(tmpdir, output_model):
    # we were seeing an issue in the rossmann workflow where we dropped certain columns,
    # https://github.com/NVIDIA/NVTabular/issues/961
    df = make_df(
        {
            "cat": ["aaaa", "bbbb", "cccc", "aaaa", "bbbb", "aaaa"],
            "cont": [0.0, 1.0, 2.0, 3.0, 4.0, 5],
        }
    )
    # this bug only happened with a dataframe representation: force this by using a lambda
    cats = ["cat"] >> ops.LambdaOp(lambda col: hash_series(col) % 1000)
    conts = ["cont"] >> ops.Normalize() >> ops.FillMissing() >> ops.LogOp()

    workflow = nvt.Workflow(cats + conts)


  _verify_workflow_on_tritonserver(


        tmpdir,
        workflow,
        df,
        "test_concatenate_dataframe",
        output_model,
        cats=["cat"],
        conts=["cont"],
    )

tests/unit/test_triton_inference.py:202:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...le_line":1069,"grpc_message":"Request for unknown model: 'test_concatenate_dataframe' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'test_concatenate_dataframe' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

______________________ testconcatenate_dataframe[pytorch] ______________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/testconcatenate_dataframe_pyto0')

output_model = 'pytorch'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def testconcatenate_dataframe(tmpdir, output_model):
    # we were seeing an issue in the rossmann workflow where we dropped certain columns,
    # https://github.com/NVIDIA/NVTabular/issues/961
    df = make_df(
        {
            "cat": ["aaaa", "bbbb", "cccc", "aaaa", "bbbb", "aaaa"],
            "cont": [0.0, 1.0, 2.0, 3.0, 4.0, 5],
        }
    )
    # this bug only happened with a dataframe representation: force this by using a lambda
    cats = ["cat"] >> ops.LambdaOp(lambda col: hash_series(col) % 1000)
    conts = ["cont"] >> ops.Normalize() >> ops.FillMissing() >> ops.LogOp()

    workflow = nvt.Workflow(cats + conts)


  _verify_workflow_on_tritonserver(


        tmpdir,
        workflow,
        df,
        "test_concatenate_dataframe",
        output_model,
        cats=["cat"],
        conts=["cont"],
    )

tests/unit/test_triton_inference.py:202:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...le_line":1069,"grpc_message":"Request for unknown model: 'test_concatenate_dataframe' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'test_concatenate_dataframe' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

_______________________ test_numeric_dtypes[tensorflow] ________________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_numeric_dtypes_tensorflow0')

output_model = 'tensorflow'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def test_numeric_dtypes(tmpdir, output_model):
    def make_dtypes(prefix, widths, info_type):
        return [(f"{prefix}{width}", info_type(f"{prefix}{width}")) for width in widths]

    int_dtypes = make_dtypes("int", [32, 64], np.iinfo)
    uint_dtypes = make_dtypes("uint", [32, 64], np.iinfo)
    float_dtypes = make_dtypes("float", [32, 64], np.finfo)

    def check_dtypes(col):
        assert str(col.dtype) == col.name
        return col

    # simple transform to make sure we can round-trip the min/max values for each dtype,
    # through triton, with the 'transform' here just checking that the dtypes are correct
    dtypes = int_dtypes + uint_dtypes + float_dtypes
    df = make_df(
        {dtype: np.array([limits.max, 0, limits.min], dtype=dtype) for dtype, limits in dtypes}
    )

    features = nvt.ColumnSelector(df.columns) >> ops.LambdaOp(check_dtypes)
    workflow = nvt.Workflow(features)
    dataset = nvt.Dataset(df)
    workflow.fit(dataset)

    if output_model == "pytorch":
        for dtype, _ in dtypes:
            workflow.output_dtypes[dtype] = np.dtype(dtype)


  _verify_workflow_on_tritonserver(


        tmpdir,
        workflow,
        df,
        "test_numeric_dtypes",
        output_model,
        cats=[dtype for dtype, _ in int_dtypes + uint_dtypes],
        conts=[dtype for dtype, _ in float_dtypes],
    )

tests/unit/test_triton_inference.py:243:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...cc","file_line":1069,"grpc_message":"Request for unknown model: 'test_numeric_dtypes' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'test_numeric_dtypes' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

_________________________ test_numeric_dtypes[pytorch] _________________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_numeric_dtypes_pytorch_0')

output_model = 'pytorch'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def test_numeric_dtypes(tmpdir, output_model):
    def make_dtypes(prefix, widths, info_type):
        return [(f"{prefix}{width}", info_type(f"{prefix}{width}")) for width in widths]

    int_dtypes = make_dtypes("int", [32, 64], np.iinfo)
    uint_dtypes = make_dtypes("uint", [32, 64], np.iinfo)
    float_dtypes = make_dtypes("float", [32, 64], np.finfo)

    def check_dtypes(col):
        assert str(col.dtype) == col.name
        return col

    # simple transform to make sure we can round-trip the min/max values for each dtype,
    # through triton, with the 'transform' here just checking that the dtypes are correct
    dtypes = int_dtypes + uint_dtypes + float_dtypes
    df = make_df(
        {dtype: np.array([limits.max, 0, limits.min], dtype=dtype) for dtype, limits in dtypes}
    )

    features = nvt.ColumnSelector(df.columns) >> ops.LambdaOp(check_dtypes)
    workflow = nvt.Workflow(features)
    dataset = nvt.Dataset(df)
    workflow.fit(dataset)

    if output_model == "pytorch":
        for dtype, _ in dtypes:
            workflow.output_dtypes[dtype] = np.dtype(dtype)


  _verify_workflow_on_tritonserver(


        tmpdir,
        workflow,
        df,
        "test_numeric_dtypes",
        output_model,
        cats=[dtype for dtype, _ in int_dtypes + uint_dtypes],
        conts=[dtype for dtype, _ in float_dtypes],
    )

tests/unit/test_triton_inference.py:243:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...cc","file_line":1069,"grpc_message":"Request for unknown model: 'test_numeric_dtypes' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'test_numeric_dtypes' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

________________________ test_groupby_model[tensorflow] ________________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_groupby_model_tensorflow_0')

output_model = 'tensorflow'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def test_groupby_model(tmpdir, output_model):
    size = 20
    df = make_df(
        {
            "id": np.random.choice([0, 1], size=size),
            "ts": np.linspace(0.0, 10.0, num=size),
            "x": np.arange(size),
            "y": np.linspace(0.0, 10.0, num=size),
        }
    )

    groupby_features = ColumnSelector(["id", "ts", "x", "y"]) >> ops.Groupby(
        groupby_cols=["id"],
        sort_cols=["ts"],
        aggs={
            "x": ["sum"],
            "y": ["first"],
        },
        name_sep="-",
    )
    workflow = nvt.Workflow(groupby_features)


  _verify_workflow_on_tritonserver(


        tmpdir, workflow, df, "groupby", output_model, cats=["id", "y-first"], conts=["x-sum"]
    )

tests/unit/test_triton_inference.py:379:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...urface/call.cc","file_line":1069,"grpc_message":"Request for unknown model: 'groupby' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'groupby' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

_________________________ test_groupby_model[pytorch] __________________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_groupby_model_pytorch_0')

output_model = 'pytorch'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow", "pytorch"])
def test_groupby_model(tmpdir, output_model):
    size = 20
    df = make_df(
        {
            "id": np.random.choice([0, 1], size=size),
            "ts": np.linspace(0.0, 10.0, num=size),
            "x": np.arange(size),
            "y": np.linspace(0.0, 10.0, num=size),
        }
    )

    groupby_features = ColumnSelector(["id", "ts", "x", "y"]) >> ops.Groupby(
        groupby_cols=["id"],
        sort_cols=["ts"],
        aggs={
            "x": ["sum"],
            "y": ["first"],
        },
        name_sep="-",
    )
    workflow = nvt.Workflow(groupby_features)


  _verify_workflow_on_tritonserver(


        tmpdir, workflow, df, "groupby", output_model, cats=["id", "y-first"], conts=["x-sum"]
    )

tests/unit/test_triton_inference.py:379:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...urface/call.cc","file_line":1069,"grpc_message":"Request for unknown model: 'groupby' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'groupby' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

______________________ test_seq_etl_tf_model[tensorflow] _______________________
tmpdir = local('/tmp/pytest-of-jenkins/pytest-4/test_seq_etl_tf_model_tensorfl0')

output_model = 'tensorflow'
@pytest.mark.skipif(TRITON_SERVER_PATH is None, reason="Requires tritonserver on the path")
@pytest.mark.parametrize("output_model", ["tensorflow"])
def test_seq_etl_tf_model(tmpdir, output_model):
    size = 100
    max_length = 10
    df = make_df(
        {
            "id": np.random.choice([0, 1], size=size),
            "item_id": np.random.randint(1, 10, size),
            "ts": np.linspace(0.0, 10.0, num=size).astype(np.float32),
            "y": np.linspace(0.0, 10.0, num=size).astype(np.float32),
        }
    )

    groupby_features = ColumnSelector(["id", "item_id", "ts", "y"]) >> ops.Groupby(
        groupby_cols=["id"],
        sort_cols=["ts"],
        aggs={
            "item_id": ["list"],
            "y": ["list"],
        },
        name_sep="-",
    )
    feats_list = groupby_features["item_id-list", "y-list"]
    feats_trim = feats_list >> ops.ListSlice(0, max_length, pad=True)
    selected_features = groupby_features["id"] + feats_trim

    workflow = nvt.Workflow(selected_features)

    sparse_max = {"item_id-list": max_length, "y-list": max_length}


  _verify_workflow_on_tritonserver(


        tmpdir,
        workflow,
        df,
        "groupby",
        output_model,
        sparse_max,
        cats=["id", "item_id-list"],
        conts=["y-list"],
    )

tests/unit/test_triton_inference.py:415:

tests/unit/test_triton_inference.py:112: in _verify_workflow_on_tritonserver

response = client.infer(model_name, inputs, outputs=outputs)

/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)

rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.NOT_FOUND

details = "Request for unknown model: ...urface/call.cc","file_line":1069,"grpc_message":"Request for unknown model: 'groupby' is not found","grpc_status":5}"


def raise_error_grpc(rpc_error):


  raise get_error_grpc(rpc_error) from None


E       tritonclient.utils.InferenceServerException: [StatusCode.NOT_FOUND] Request for unknown model: 'groupby' is not found
/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

_______________________________ test_example_03 ________________________________
def test_example_03():
    with testbook(
        REPO_ROOT / "examples" / "03-Running-on-multiple-GPUs-or-on-CPU.ipynb",
        execute=False,
        timeout=180,
    ) as tb:
        tb.inject(
            """
            import os
            from unittest.mock import patch
            from merlin.datasets.synthetic import generate_data
            mock_train, mock_valid = generate_data(
                input="movielens-1m",
                num_rows=1000,
                set_sizes=(0.8, 0.2)
            )
            input_path = os.environ.get(
                "INPUT_DATA_DIR",
                os.path.expanduser("~/merlin-framework/movielens/")
            )
            from pathlib import Path
            Path(f'{input_path}ml-1m').mkdir(parents=True, exist_ok=True)
            mock_train.compute().to_parquet(f'{input_path}ml-1m/train.parquet')
            mock_train.compute().to_parquet(f'{input_path}ml-1m/valid.parquet')

            p1 = patch(
                "merlin.datasets.entertainment.get_movielens",
                return_value=[mock_train, mock_valid]
            )
            p1.start()

            """
        )


      tb.execute()


tests/unit/examples/test_03-Running-on-multiple-GPUs-or-on-CPU.py:59:

../../../.local/lib/python3.8/site-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/local/lib/python3.8/dist-packages/nest_asyncio.py:89: in run_until_complete

return f.result()

/usr/lib/python3.8/asyncio/futures.py:178: in result

raise self._exception

/usr/lib/python3.8/asyncio/tasks.py:280: in __step

result = coro.send(None)

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)

self = <testbook.client.TestbookNotebookClient object at 0x7f956af9f070>

cell = {'cell_type': 'code', 'execution_count': 6, 'id': 'e02409ee', 'metadata': {'execution': {'iopub.status.busy': '2022-12...rkdir,\n        dashboard_address=":" + dashboard_port,\n        rmm_pool_size=(device_pool_size // 256) * 256\n    )'}

cell_index = 11

exec_reply = {'buffers': [], 'content': {'ename': 'MemoryError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'a1b577b4-9d15-435...e, 'engine': 'a1b577b4-9d15-4352-9775-46cff1112c30', 'started': '2022-12-14T11:43:02.996618Z', 'status': 'error'}, ...}
async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:


      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)


E E           ------------------
 E           if cluster is None:
 E E E E E E E E E               )
 E           ------------------
 E
 E E E E           �[1;32m E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E
 E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E
 E E           �[1;32m E           �[1;32m E           �[1;32m E           �[0;32m   (...)�[0m
 E           �[1;32m E           �[1;32m     40�[0m ):
 E           �[1;32m E           �[1;32m E           �[1;32m E           �[0;32m   (...)�[0m
 E           �[1;32m E           �[1;32m E E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E           �[1;32m E
 E E
 E E
 E E
 E E

nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
 cluster = LocalCUDACluster(
 protocol=protocol,
 n_workers=len(visible_devices.split(",")),
 CUDA_VISIBLE_DEVICES=visible_devices,
 device_memory_limit=device_limit,
 local_directory=dask_workdir,
 dashboard_address=":" + dashboard_port,
 rmm_pool_size=(device_pool_size // 256) * 256
 �[0;31m---------------------------------------------------------------------------�[0m
 �[0;31mMemoryError�[0m                               Traceback (most recent call last)
 Cell �[0;32mIn [6], line 2�[0m
 1�[0m �[38;5;28;01mif�[39;00m cluster �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m:
 �[0;32m----> 2�[0m     cluster �[38;5;241m=�[39m �[43mLocalCUDACluster�[49m�[43m(�[49m
 3�[0m �[43m        �[49m�[43mprotocol�[49m�[38;5;241;43m=�[39;49m�[43mprotocol�[49m�[43m,�[49m
 4�[0m �[43m        �[49m�[43mn_workers�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43mlen�[39;49m�[43m(�[49m�[43mvisible_devices�[49m�[38;5;241;43m.�[39;49m�[43msplit�[49m�[43m(�[49m�[38;5;124;43m"�[39;49m�[38;5;124;43m,�[39;49m�[38;5;124;43m"�[39;49m�[43m)�[49m�[43m)�[49m�[43m,�[49m
 5�[0m �[43m        �[49m�[43mCUDA_VISIBLE_DEVICES�[49m�[38;5;241;43m=�[39;49m�[43mvisible_devices�[49m�[43m,�[49m
 6�[0m �[43m        �[49m�[43mdevice_memory_limit�[49m�[38;5;241;43m=�[39;49m�[43mdevice_limit�[49m�[43m,�[49m
 7�[0m �[43m        �[49m�[43mlocal_directory�[49m�[38;5;241;43m=�[39;49m�[43mdask_workdir�[49m�[43m,�[49m
 8�[0m �[43m        �[49m�[43mdashboard_address�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m"�[39;49m�[38;5;124;43m:�[39;49m�[38;5;124;43m"�[39;49m�[43m �[49m�[38;5;241;43m+�[39;49m�[43m �[49m�[43mdashboard_port�[49m�[43m,�[49m
 9�[0m �[43m        �[49m�[43mrmm_pool_size�[49m�[38;5;241;43m=�[39;49m�[43m(�[49m�[43mdevice_pool_size�[49m�[43m �[49m�[38;5;241;43m/�[39;49m�[38;5;241;43m/�[39;49m�[43m �[49m�[38;5;241;43m256�[39;49m�[43m)�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[43m �[49m�[38;5;241;43m256�[39;49m
 10�[0m �[43m    �[49m�[43m)�[49m
 File �[0;32m/usr/local/lib/python3.8/dist-packages/dask_cuda/local_cuda_cluster.py:367�[0m, in �[0;36mLocalCUDACluster.__init__�[0;34m(self, CUDA_VISIBLE_DEVICES, n_workers, threads_per_worker, memory_limit, device_memory_limit, data, local_directory, shared_filesystem, protocol, enable_tcp_over_ucx, enable_infiniband, enable_nvlink, enable_rdmacm, rmm_pool_size, rmm_maximum_pool_size, rmm_managed_memory, rmm_async, rmm_log_directory, rmm_track_allocations, jit_unspill, log_spilling, worker_class, pre_import, **kwargs)�[0m
 365�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mcuda_visible_devices �[38;5;241m=�[39m CUDA_VISIBLE_DEVICES
 366�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mscale(n_workers)
 �[0;32m--> 367�[0m �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43msync�[49m�[43m(�[49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43m_correct_state�[49m�[43m)�[49m
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/utils.py:309�[0m, in �[0;36mSyncMethodMixin.sync�[0;34m(self, func, asynchronous, callback_timeout, args, **kwargs)�[0m
 307�[0m     �[38;5;28;01mreturn�[39;00m future
 308�[0m �[38;5;28;01melse�[39;00m:
 �[0;32m--> 309�[0m     �[38;5;28;01mreturn�[39;00m �[43msync�[49m�[43m(�[49m
 310�[0m �[43m        �[49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mloop�[49m�[43m,�[49m�[43m �[49m�[43mfunc�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[43margs�[49m�[43m,�[49m�[43m �[49m�[43mcallback_timeout�[49m�[38;5;241;43m=�[39;49m�[43mcallback_timeout�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m�[39;49m�[38;5;241;43m*�[39;49m�[43mkwargs�[49m
 311�[0m �[43m    �[49m�[43m)�[49m
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/utils.py:376�[0m, in �[0;36msync�[0;34m(loop, func, callback_timeout, *args, **kwargs)�[0m
 374�[0m �[38;5;28;01mif�[39;00m error:
 375�[0m     typ, exc, tb �[38;5;241m=�[39m error
 �[0;32m--> 376�[0m     �[38;5;28;01mraise�[39;00m exc�[38;5;241m.�[39mwith_traceback(tb)
 377�[0m �[38;5;28;01melse�[39;00m:
 378�[0m     �[38;5;28;01mreturn�[39;00m result
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/utils.py:349�[0m, in �[0;36msync..f�[0;34m()�[0m
 347�[0m         future �[38;5;241m=�[39m asyncio�[38;5;241m.�[39mwait_for(future, callback_timeout)
 348�[0m     future �[38;5;241m=�[39m asyncio�[38;5;241m.�[39mensure_future(future)
 �[0;32m--> 349�[0m     result �[38;5;241m=�[39m �[38;5;28;01myield�[39;00m future
 350�[0m �[38;5;28;01mexcept�[39;00m �[38;5;167;01mException�[39;00m:
 351�[0m     error �[38;5;241m=�[39m sys�[38;5;241m.�[39mexc_info()
 File �[0;32m~/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg/tornado/gen.py:762�[0m, in �[0;36mRunner.run�[0;34m(self)�[0m
 759�[0m exc_info �[38;5;241m=�[39m �[38;5;28;01mNone�[39;00m
 761�[0m �[38;5;28;01mtry�[39;00m:
 �[0;32m--> 762�[0m     value �[38;5;241m=�[39m �[43mfuture�[49m�[38;5;241;43m.�[39;49m�[43mresult�[49m�[43m(�[49m�[43m)�[49m
 763�[0m �[38;5;28;01mexcept�[39;00m �[38;5;167;01mException�[39;00m:
 764�[0m     exc_info �[38;5;241m=�[39m sys�[38;5;241m.�[39mexc_info()
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/deploy/spec.py:352�[0m, in �[0;36mSpecCluster._correct_state_internal�[0;34m(self)�[0m
 350�[0m     �[38;5;28;01mfor�[39;00m w �[38;5;129;01min�[39;00m workers:
 351�[0m         w�[38;5;241m.�[39m_cluster �[38;5;241m=�[39m weakref�[38;5;241m.�[39mref(�[38;5;28mself�[39m)
 �[0;32m--> 352�[0m         �[38;5;28;01mawait�[39;00m w  �[38;5;66;03m# for tornado gen.coroutine support�[39;00m
 353�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mworkers�[38;5;241m.�[39mupdate(�[38;5;28mdict�[39m(�[38;5;28mzip�[39m(to_open, workers)))
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/core.py:299�[0m, in �[0;36mServer.await.._�[0;34m()�[0m
 293�[0m             �[38;5;28;01mraise�[39;00m �[38;5;167;01mTimeoutError�[39;00m(
 294�[0m                 �[38;5;124m"�[39m�[38;5;132;01m{}�[39;00m�[38;5;124m failed to start in �[39m�[38;5;132;01m{}�[39;00m�[38;5;124m seconds�[39m�[38;5;124m"�[39m�[38;5;241m.�[39mformat(
 295�[0m                     �[38;5;28mtype�[39m(�[38;5;28mself�[39m)�[38;5;241m.�[39m�[38;5;18m__name__�[39m, timeout
 296�[0m                 )
 297�[0m             )
 298�[0m     �[38;5;28;01melse�[39;00m:
 �[0;32m--> 299�[0m         �[38;5;28;01mawait�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mstart()
 300�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mstatus �[38;5;241m=�[39m Status�[38;5;241m.�[39mrunning
 301�[0m �[38;5;28;01mreturn�[39;00m �[38;5;28mself�[39m
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/nanny.py:347�[0m, in �[0;36mNanny.start�[0;34m(self)�[0m
 344�[0m     �[38;5;28;01mawait�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mplugin_add(plugin�[38;5;241m=�[39mplugin, name�[38;5;241m=�[39mname)
 346�[0m logger�[38;5;241m.�[39minfo(�[38;5;124m"�[39m�[38;5;124m        Start Nanny at: �[39m�[38;5;132;01m%r�[39;00m�[38;5;124m"�[39m, �[38;5;28mself�[39m�[38;5;241m.�[39maddress)
 �[0;32m--> 347�[0m response �[38;5;241m=�[39m �[38;5;28;01mawait�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39minstantiate()
 348�[0m �[38;5;28;01mif�[39;00m response �[38;5;241m==�[39m Status�[38;5;241m.�[39mrunning:
 349�[0m     �[38;5;28;01massert�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mworker_address
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/nanny.py:430�[0m, in �[0;36mNanny.instantiate�[0;34m(self)�[0m
 428�[0m �[38;5;28;01melse�[39;00m:
 429�[0m     �[38;5;28;01mtry�[39;00m:
 �[0;32m--> 430�[0m         result �[38;5;241m=�[39m �[38;5;28;01mawait�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mprocess�[38;5;241m.�[39mstart()
 431�[0m     �[38;5;28;01mexcept�[39;00m �[38;5;167;01mException�[39;00m:
 432�[0m         �[38;5;28;01mawait�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mclose()
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/nanny.py:685�[0m, in �[0;36mWorkerProcess.start�[0;34m(self)�[0m
 683�[0m     �[38;5;28;01mreturn�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mstatus
 684�[0m �[38;5;28;01mtry�[39;00m:
 �[0;32m--> 685�[0m     msg �[38;5;241m=�[39m �[38;5;28;01mawait�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39m_wait_until_connected(uid)
 686�[0m �[38;5;28;01mexcept�[39;00m �[38;5;167;01mException�[39;00m:
 687�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39mstatus �[38;5;241m=�[39m Status�[38;5;241m.�[39mfailed
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/nanny.py:803�[0m, in �[0;36mWorkerProcess._wait_until_connected�[0;34m(self, uid)�[0m
 799�[0m �[38;5;28;01mif�[39;00m �[38;5;124m"�[39m�[38;5;124mexception�[39m�[38;5;124m"�[39m �[38;5;129;01min�[39;00m msg:
 800�[0m     logger�[38;5;241m.�[39merror(
 801�[0m         �[38;5;124m"�[39m�[38;5;124mFailed while trying to start worker process: �[39m�[38;5;132;01m%s�[39;00m�[38;5;124m"�[39m, msg[�[38;5;124m"�[39m�[38;5;124mexception�[39m�[38;5;124m"�[39m]
 802�[0m     )
 �[0;32m--> 803�[0m     �[38;5;28;01mraise�[39;00m msg[�[38;5;124m"�[39m�[38;5;124mexception�[39m�[38;5;124m"�[39m]
 804�[0m �[38;5;28;01melse�[39;00m:
 805�[0m     �[38;5;28;01mreturn�[39;00m msg
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/nanny.py:869�[0m, in �[0;36mrun�[0;34m()�[0m
 865�[0m �[38;5;124;03m"""�[39;00m
 866�[0m �[38;5;124;03mTry to start worker and inform parent of outcome.�[39;00m
 867�[0m �[38;5;124;03m"""�[39;00m
 868�[0m �[38;5;28;01mtry�[39;00m:
 �[0;32m--> 869�[0m     �[38;5;28;01mawait�[39;00m worker
 870�[0m �[38;5;28;01mexcept�[39;00m �[38;5;167;01mException�[39;00m �[38;5;28;01mas�[39;00m e:
 871�[0m     logger�[38;5;241m.�[39mexception(�[38;5;124m"�[39m�[38;5;124mFailed to start worker�[39m�[38;5;124m"�[39m)
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/core.py:299�[0m, in �[0;36m_�[0;34m()�[0m
 293�[0m             �[38;5;28;01mraise�[39;00m �[38;5;167;01mTimeoutError�[39;00m(
 294�[0m                 �[38;5;124m"�[39m�[38;5;132;01m{}�[39;00m�[38;5;124m failed to start in �[39m�[38;5;132;01m{}�[39;00m�[38;5;124m seconds�[39m�[38;5;124m"�[39m�[38;5;241m.�[39mformat(
 295�[0m                     �[38;5;28mtype�[39m(�[38;5;28mself�[39m)�[38;5;241m.�[39m�[38;5;18m__name__�[39m, timeout
 296�[0m                 )
 297�[0m             )
 298�[0m     �[38;5;28;01melse�[39;00m:
 �[0;32m--> 299�[0m         �[38;5;28;01mawait�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mstart()
 300�[0m         �[38;5;28mself�[39m�[38;5;241m.�[39mstatus �[38;5;241m=�[39m Status�[38;5;241m.�[39mrunning
 301�[0m �[38;5;28;01mreturn�[39;00m �[38;5;28mself�[39m
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/worker.py:1372�[0m, in �[0;36mstart�[0;34m()�[0m
 1370�[0m         �[38;5;28;01mfor�[39;00m exc �[38;5;129;01min�[39;00m plugins_exceptions:
 1371�[0m             logger�[38;5;241m.�[39merror(�[38;5;28mrepr�[39m(exc))
 �[0;32m-> 1372�[0m     �[38;5;28;01mraise�[39;00m plugins_exceptions[�[38;5;241m0�[39m]
 1374�[0m �[38;5;28mself�[39m�[38;5;241m.�[39m_pending_plugins �[38;5;241m=�[39m ()
 1376�[0m �[38;5;28;01mawait�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39m_register_with_scheduler()
 File �[0;32m~/.local/lib/python3.8/site-packages/distributed/worker.py:3248�[0m, in �[0;36mplugin_add�[0;34m()�[0m
 3246�[0m �[38;5;28;01mif�[39;00m �[38;5;28mhasattr�[39m(plugin, �[38;5;124m"�[39m�[38;5;124msetup�[39m�[38;5;124m"�[39m):
 3247�[0m     �[38;5;28;01mtry�[39;00m:
 �[0;32m-> 3248�[0m         result �[38;5;241m=�[39m plugin�[38;5;241m.�[39msetup(worker�[38;5;241m=�[39m�[38;5;28mself�[39m)
 3249�[0m         �[38;5;28;01mif�[39;00m isawaitable(result):
 3250�[0m             result �[38;5;241m=�[39m �[38;5;28;01mawait�[39;00m result
 File �[0;32m/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py:78�[0m, in �[0;36msetup�[0;34m()�[0m
 74�[0m     �[38;5;28;01mimport�[39;00m �[38;5;21;01mrmm�[39;00m
 76�[0m     pool_allocator �[38;5;241m=�[39m �[38;5;28;01mFalse�[39;00m �[38;5;28;01mif�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39minitial_pool_size �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m �[38;5;28;01melse�[39;00m �[38;5;28;01mTrue�[39;00m
 �[0;32m---> 78�[0m     rmm�[38;5;241m.�[39mreinitialize(
 79�[0m         pool_allocator�[38;5;241m=�[39mpool_allocator,
 80�[0m         managed_memory�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mmanaged_memory,
 81�[0m         initial_pool_size�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39minitial_pool_size,
 82�[0m         maximum_pool_size�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mmaximum_pool_size,
 83�[0m         logging�[38;5;241m=�[39m�[38;5;28mself�[39m�[38;5;241m.�[39mlogging,
 84�[0m         log_file_name�[38;5;241m=�[39mget_rmm_log_file_name(
 85�[0m             worker, �[38;5;28mself�[39m�[38;5;241m.�[39mlogging, �[38;5;28mself�[39m�[38;5;241m.�[39mlog_directory
 86�[0m         ),
 87�[0m     )
 88�[0m �[38;5;28;01mif�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mrmm_track_allocations:
 89�[0m     �[38;5;28;01mimport�[39;00m �[38;5;21;01mrmm�[39;00m
 File �[0;32m/usr/local/lib/python3.8/dist-packages/rmm/rmm.py:85�[0m, in �[0;36mreinitialize�[0;34m()�[0m
 32�[0m �[38;5;28;01mdef�[39;00m �[38;5;21mreinitialize�[39m(
 33�[0m     pool_allocator�[38;5;241m=�[39m�[38;5;28;01mFalse�[39;00m,
 34�[0m     managed_memory�[38;5;241m=�[39m�[38;5;28;01mFalse�[39;00m,
 39�[0m     log_file_name�[38;5;241m=�[39m�[38;5;28;01mNone�[39;00m,
 41�[0m     �[38;5;124;03m"""�[39;00m
 42�[0m �[38;5;124;03m    Finalizes and then initializes RMM using the options passed. Using memory�[39;00m
 43�[0m �[38;5;124;03m    from a previous initialization of RMM is undefined behavior and should be�[39;00m
 83�[0m �[38;5;124;03m    corresponding to each device.�[39;00m
 84�[0m �[38;5;124;03m    """�[39;00m
 �[0;32m---> 85�[0m     rmm�[38;5;241m.�[39mmr�[38;5;241m.�[39m_initialize(
 86�[0m         pool_allocator�[38;5;241m=�[39mpool_allocator,
 87�[0m         managed_memory�[38;5;241m=�[39mmanaged_memory,
 88�[0m         initial_pool_size�[38;5;241m=�[39minitial_pool_size,
 89�[0m         maximum_pool_size�[38;5;241m=�[39mmaximum_pool_size,
 90�[0m         devices�[38;5;241m=�[39mdevices,
 91�[0m         logging�[38;5;241m=�[39mlogging,
 92�[0m         log_file_name�[38;5;241m=�[39mlog_file_name,
 93�[0m     )
 File �[0;32mmemory_resource.pyx:823�[0m, in �[0;36mrmm._lib.memory_resource._initialize�[0;34m()�[0m
 File �[0;32mmemory_resource.pyx:883�[0m, in �[0;36mrmm._lib.memory_resource._initialize�[0;34m()�[0m
 File �[0;32mmemory_resource.pyx:342�[0m, in �[0;36mrmm._lib.memory_resource.PoolMemoryResource.__cinit__�[0;34m()�[0m
 �[0;31mMemoryError�[0m: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded
 MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded
../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

----------------------------- Captured stderr call -----------------------------

2022-12-14 11:43:05,918 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:05,920 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:05,938 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:05,996 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:06,462 - distributed.utils - ERROR - std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/utils.py", line 693, in log_errors

yield

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:06,463 - distributed.nanny - ERROR - Failed to start worker

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/nanny.py", line 869, in run

await worker

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/core.py", line 299, in _

await self.start()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 1372, in start

raise plugins_exceptions[0]

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:09,164 - distributed.diskutils - INFO - Found stale lock file and directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/test_dask/workdir/dask-worker-space/worker-573731q8', purging

2022-12-14 11:43:09,165 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:09,442 - distributed.utils - ERROR - std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/utils.py", line 693, in log_errors

yield

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:09,443 - distributed.nanny - ERROR - Failed to start worker

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/nanny.py", line 869, in run

await worker

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/core.py", line 299, in _

await self.start()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 1372, in start

raise plugins_exceptions[0]

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:12,571 - distributed.diskutils - INFO - Found stale lock file and directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/test_dask/workdir/dask-worker-space/worker-9snau6g9', purging

2022-12-14 11:43:12,572 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:12,603 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:12,638 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:12,682 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:13,263 - distributed.utils - ERROR - std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/utils.py", line 693, in log_errors

yield

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:13,265 - distributed.nanny - ERROR - Failed to start worker

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/nanny.py", line 869, in run

await worker

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/core.py", line 299, in _

await self.start()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 1372, in start

raise plugins_exceptions[0]

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:13,292 - distributed.utils - ERROR - std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/utils.py", line 693, in log_errors

yield

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:13,293 - distributed.nanny - ERROR - Failed to start worker

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/nanny.py", line 869, in run

await worker

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/core.py", line 299, in _

await self.start()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 1372, in start

raise plugins_exceptions[0]

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:13,450 - distributed.utils - ERROR - std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/utils.py", line 693, in log_errors

yield

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:13,451 - distributed.nanny - ERROR - Failed to start worker

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/nanny.py", line 869, in run

await worker

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/core.py", line 299, in _

await self.start()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 1372, in start

raise plugins_exceptions[0]

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:16,932 - distributed.diskutils - INFO - Found stale lock file and directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/test_dask/workdir/dask-worker-space/worker-kouohdwq', purging

2022-12-14 11:43:16,933 - distributed.diskutils - INFO - Found stale lock file and directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/test_dask/workdir/dask-worker-space/worker-esia89a9', purging

2022-12-14 11:43:16,933 - distributed.diskutils - INFO - Found stale lock file and directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/test_dask/workdir/dask-worker-space/worker-87lbo6pc', purging

2022-12-14 11:43:16,934 - distributed.preloading - INFO - Import preload module: dask_cuda.initialize

2022-12-14 11:43:17,287 - distributed.utils - ERROR - std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/utils.py", line 693, in log_errors

yield

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

2022-12-14 11:43:17,288 - distributed.nanny - ERROR - Failed to start worker

Traceback (most recent call last):

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/nanny.py", line 869, in run

await worker

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/core.py", line 299, in _

await self.start()

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 1372, in start

raise plugins_exceptions[0]

File "/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/worker.py", line 3248, in plugin_add

result = plugin.setup(worker=self)

File "/usr/local/lib/python3.8/dist-packages/dask_cuda/utils.py", line 78, in setup

rmm.reinitialize(

File "/usr/local/lib/python3.8/dist-packages/rmm/rmm.py", line 85, in reinitialize

rmm.mr._initialize(

File "memory_resource.pyx", line 823, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 883, in rmm._lib.memory_resource._initialize

File "memory_resource.pyx", line 342, in rmm._lib.memory_resource.PoolMemoryResource.cinit

MemoryError: std::bad_alloc: out_of_memory: RMM failure at:/opt/rapids/rmm/include/rmm/mr/device/pool_memory_resource.hpp:192: Maximum pool size exceeded

/usr/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 51 leaked semaphore objects to clean up at shutdown

warnings.warn('resource_tracker: There appear to be %d '

=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33

/usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)
.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: 34 warnings

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)
tests/unit/test_dask_nvt.py: 6 warnings

tests/unit/workflow/test_workflow.py: 78 warnings

/var/jenkins_home/.local/lib/python3.8/site-packages/dask/base.py:1282: UserWarning: Running on a single-machine scheduler when a distributed client is active might lead to unexpected results.

warnings.warn(
tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu/lib/python3.8/site-packages/merlin/core/utils.py:431: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(
tests/unit/ops/test_fill.py::test_fill_missing[True-True-parquet]

tests/unit/ops/test_fill.py::test_fill_missing[True-False-parquet]

tests/unit/ops/test_ops.py::test_filter[parquet-0.1-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/pandas/core/indexing.py:1732: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

self._setitem_single_block(indexer, value, name)
tests/unit/ops/test_ops_schema.py: 12 warnings

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(
tests/unit/ops/test_ops_schema.py: 12 warnings

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                Stmts   Miss  Cover
merlin/transforms/init.py           1      1     0%

merlin/transforms/ops/init.py       1      1     0%
TOTAL                                   2      2     0%
=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/io/tests/test_s3.py:14: could not import 'moto': No module named 'moto'

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

===== 13 failed, 1421 passed, 2 skipped, 147 warnings in 924.70s (0:15:24) =====

/usr/local/lib/python3.8/dist-packages/coverage/control.py:801: CoverageWarning: No data was collected. (no-data-collected)

self._warn("No data was collected.", slug="no-data-collected")

/usr/local/lib/python3.8/dist-packages/coverage/data.py:130: CoverageWarning: Data file '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.coverage.10.20.17.231.27728.807725' doesn't seem to be a coverage data file: cannot unpack non-iterable NoneType object

data._warn(str(exc))

/usr/local/lib/python3.8/dist-packages/coverage/data.py:130: CoverageWarning: Data file '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.coverage.10.20.17.231.27959.410508' doesn't seem to be a coverage data file: cannot unpack non-iterable NoneType object

data._warn(str(exc))

/usr/local/lib/python3.8/dist-packages/coverage/data.py:130: CoverageWarning: Data file '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/.coverage.10.20.17.231.27736.725710' doesn't seem to be a coverage data file: cannot unpack non-iterable NoneType object

data._warn(str(exc))

ERROR: InvocationError for command /var/jenkins_home/workspace/nvtabular_tests/nvtabular/.tox/test-gpu/bin/python -m pytest --cov-report term --cov merlin -rxs tests/unit (exited with code 1)

___________________________________ summary ____________________________________

ERROR:   test-gpu: commands failed

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://github.com/gitapi/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[workspace] $ /bin/bash /tmp/jenkins13767705993352413709.sh

This version bump complicates the picture of getting this change merged, since there are unrelated issues with dependency version compatibility. We should also bump the Core version, but let's do that in a separate PR.

oliverholworthy added 2 commits December 14, 2022 10:32

Add minimum Python version as 3.8 to setup.py configuration

5bd52f4

Add Programming Language classifiers to setup.py configuration

f80bc65

indicating the intended Python version compatibility of the library

oliverholworthy added the ci label Dec 14, 2022

oliverholworthy added this to the Merlin 22.12 milestone Dec 14, 2022

oliverholworthy self-assigned this Dec 14, 2022

Update minimum version of merlin-core from 0.2.0 to 0.6.0

d1a8198

0.6.0 addded the merlin.dag.executors which is used in NVTabular

karlhigley added 3 commits January 13, 2023 11:10

Merge branch 'main' into python-requires-3.8

7ee62cf

Merge branch 'main' into python-requires-3.8

b406740

Revert Merlin Core version bump

8e58802

This version bump complicates the picture of getting this change merged, since there are unrelated issues with dependency version compatibility. We should also bump the Core version, but let's do that in a separate PR.

karlhigley approved these changes Jan 30, 2023

View reviewed changes

karlhigley and others added 3 commits January 30, 2023 15:19

Merge branch 'main' into python-requires-3.8

d1b0081

Merge branch 'main' into python-requires-3.8

17451b5

Merge branch 'main' into python-requires-3.8

993c303

karlhigley merged commit d226e6a into NVIDIA-Merlin:main Feb 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Specify Minimum Python Version as 3.8 in package #1732

Specify Minimum Python Version as 3.8 in package #1732

oliverholworthy commented Dec 14, 2022

nvidia-merlin-bot commented Dec 14, 2022

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Cover

merlin/transforms/init.py 1 1 0%
merlin/transforms/ops/init.py 1 1 0%

github-actions bot commented Dec 14, 2022

nvidia-merlin-bot commented Dec 14, 2022

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Cover

merlin/transforms/init.py 1 1 0%
merlin/transforms/ops/init.py 1 1 0%

Specify Minimum Python Version as 3.8 in package #1732

Specify Minimum Python Version as 3.8 in package #1732

Conversation

oliverholworthy commented Dec 14, 2022

nvidia-merlin-bot commented Dec 14, 2022

---------- coverage: platform linux, python 3.8.10-final-0 ----------- Name Stmts Miss Cover

merlin/transforms/init.py 1 1 0% merlin/transforms/ops/init.py 1 1 0%

github-actions bot commented Dec 14, 2022

Documentation preview

nvidia-merlin-bot commented Dec 14, 2022

---------- coverage: platform linux, python 3.8.10-final-0 ----------- Name Stmts Miss Cover

merlin/transforms/init.py 1 1 0% merlin/transforms/ops/init.py 1 1 0%

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Cover

merlin/transforms/init.py 1 1 0%
merlin/transforms/ops/init.py 1 1 0%

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Cover

merlin/transforms/init.py 1 1 0%
merlin/transforms/ops/init.py 1 1 0%