-
Notifications
You must be signed in to change notification settings - Fork 114
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds integration tests to Merlin Models #438
Conversation
Documentation preview |
Click to view CI ResultsGitHub pull request #438 of commit c2a14c46544f9f91730dddb0324e50b01b92d222, no merge conflicts. Running as SYSTEM Setting status of c2a14c46544f9f91730dddb0324e50b01b92d222 to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/232/console and message: 'Pending' Using context: Jenkins Building on master in workspace /var/jenkins_home/workspace/merlin_merlin using credential systems-login > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10 Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin > git --version # timeout=10 using GIT_ASKPASS to set credentials login for merlin-systems > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/438/*:refs/remotes/origin/pr/438/* # timeout=10 > git rev-parse c2a14c46544f9f91730dddb0324e50b01b92d222^{commit} # timeout=10 Checking out Revision c2a14c46544f9f91730dddb0324e50b01b92d222 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f c2a14c46544f9f91730dddb0324e50b01b92d222 # timeout=10 Commit message: "Added integration tests to Merlin Models" > git rev-list --no-walk 43d5ab866d6e360ba7de67d50dd6e76d99473049 # timeout=10 [merlin_merlin] $ /bin/bash /tmp/jenkins15760467702778833485.sh ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0 collected 2 items |
ci/test_container.sh
Outdated
@@ -101,6 +101,10 @@ echo "#####################" | |||
echo "Run integration tests for NVTabular" | |||
/nvtabular/ci/test_integration.sh $container $devices --report 1 | |||
|
|||
# Test Transformers4Rec | |||
echo "Run integration tests for Merlin Models" | |||
/models/ci/test_integration.sh $container $devices |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is WB set to stream metrics by default? if it is, can we invert it, so that it is set not to report by default and we have to turn on reporting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes @jperez999 . For integration tests W&B is set to stream metrics by default. What could be a reason for disabling that?
@jperez999 The PR introducing the retrieval integration tests into Merlin Models was already merged, so we can now merge this PR to enable running those integration tests nightly on Blossom |
c2a14c4
to
39677fa
Compare
Click to view CI ResultsGitHub pull request #438 of commit 39677fa4eb9b65b20522c7e816b998eb2e3c31d3, no merge conflicts. Running as SYSTEM Setting status of 39677fa4eb9b65b20522c7e816b998eb2e3c31d3 to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/258/console and message: 'Pending' Using context: Jenkins Building on master in workspace /var/jenkins_home/workspace/merlin_merlin using credential systems-login > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10 Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin > git --version # timeout=10 using GIT_ASKPASS to set credentials login for merlin-systems > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/438/*:refs/remotes/origin/pr/438/* # timeout=10 > git rev-parse 39677fa4eb9b65b20522c7e816b998eb2e3c31d3^{commit} # timeout=10 Checking out Revision 39677fa4eb9b65b20522c7e816b998eb2e3c31d3 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 39677fa4eb9b65b20522c7e816b998eb2e3c31d3 # timeout=10 Commit message: "Enabling calling Merlin Models integration tests after rebasing" > git rev-list --no-walk e636ce63a75841e6054c975ebdde60e104ad51f8 # timeout=10 [merlin_merlin] $ /bin/bash /tmp/jenkins2365010800849903378.sh ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0 collected 2 items |
Click to view CI ResultsGitHub pull request #438 of commit dd36e3afd92da6d92cdab47fa4fbce43161d1c4b, no merge conflicts. Running as SYSTEM Setting status of dd36e3afd92da6d92cdab47fa4fbce43161d1c4b to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/265/console and message: 'Pending' Using context: Jenkins Building on master in workspace /var/jenkins_home/workspace/merlin_merlin using credential systems-login > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10 Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin > git --version # timeout=10 using GIT_ASKPASS to set credentials login for merlin-systems > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/438/*:refs/remotes/origin/pr/438/* # timeout=10 > git rev-parse dd36e3afd92da6d92cdab47fa4fbce43161d1c4b^{commit} # timeout=10 Checking out Revision dd36e3afd92da6d92cdab47fa4fbce43161d1c4b (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f dd36e3afd92da6d92cdab47fa4fbce43161d1c4b # timeout=10 Commit message: "Merge branch 'main' into mm_integration_tests" > git rev-list --no-walk c34be4a49f5be131cd8b77577d3cfa050047d738 # timeout=10 [merlin_merlin] $ /bin/bash /tmp/jenkins11989105243338007253.sh ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0 collected 2 items |
rerun tests |
Click to view CI ResultsGitHub pull request #438 of commit dd36e3afd92da6d92cdab47fa4fbce43161d1c4b, no merge conflicts. Running as SYSTEM Setting status of dd36e3afd92da6d92cdab47fa4fbce43161d1c4b to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/266/console and message: 'Pending' Using context: Jenkins Building on master in workspace /var/jenkins_home/workspace/merlin_merlin using credential systems-login > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10 Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin > git --version # timeout=10 using GIT_ASKPASS to set credentials login for merlin-systems > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/438/*:refs/remotes/origin/pr/438/* # timeout=10 > git rev-parse dd36e3afd92da6d92cdab47fa4fbce43161d1c4b^{commit} # timeout=10 Checking out Revision dd36e3afd92da6d92cdab47fa4fbce43161d1c4b (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f dd36e3afd92da6d92cdab47fa4fbce43161d1c4b # timeout=10 Commit message: "Merge branch 'main' into mm_integration_tests" > git rev-list --no-walk dd36e3afd92da6d92cdab47fa4fbce43161d1c4b # timeout=10 [merlin_merlin] $ /bin/bash /tmp/jenkins2016635391869878287.sh ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0 collected 2 items |
rerun tests |
Click to view CI ResultsGitHub pull request #438 of commit dd36e3afd92da6d92cdab47fa4fbce43161d1c4b, no merge conflicts. Running as SYSTEM Setting status of dd36e3afd92da6d92cdab47fa4fbce43161d1c4b to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/268/console and message: 'Pending' Using context: Jenkins Building on master in workspace /var/jenkins_home/workspace/merlin_merlin using credential systems-login > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10 Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin > git --version # timeout=10 using GIT_ASKPASS to set credentials login for merlin-systems > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/438/*:refs/remotes/origin/pr/438/* # timeout=10 > git rev-parse dd36e3afd92da6d92cdab47fa4fbce43161d1c4b^{commit} # timeout=10 Checking out Revision dd36e3afd92da6d92cdab47fa4fbce43161d1c4b (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f dd36e3afd92da6d92cdab47fa4fbce43161d1c4b # timeout=10 Commit message: "Merge branch 'main' into mm_integration_tests" > git rev-list --no-walk 3bbcca6fd9a613df6b23c9934340d056e04c13d1 # timeout=10 [merlin_merlin] $ /bin/bash /tmp/jenkins14588817199850278494.sh ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0 collected 2 items |
rerun tests |
Click to view CI ResultsGitHub pull request #438 of commit dd36e3afd92da6d92cdab47fa4fbce43161d1c4b, no merge conflicts. Running as SYSTEM Setting status of dd36e3afd92da6d92cdab47fa4fbce43161d1c4b to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/269/console and message: 'Pending' Using context: Jenkins Building on master in workspace /var/jenkins_home/workspace/merlin_merlin using credential systems-login > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10 Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin > git --version # timeout=10 using GIT_ASKPASS to set credentials login for merlin-systems > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/438/*:refs/remotes/origin/pr/438/* # timeout=10 > git rev-parse dd36e3afd92da6d92cdab47fa4fbce43161d1c4b^{commit} # timeout=10 Checking out Revision dd36e3afd92da6d92cdab47fa4fbce43161d1c4b (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f dd36e3afd92da6d92cdab47fa4fbce43161d1c4b # timeout=10 Commit message: "Merge branch 'main' into mm_integration_tests" > git rev-list --no-walk dd36e3afd92da6d92cdab47fa4fbce43161d1c4b # timeout=10 [merlin_merlin] $ /bin/bash /tmp/jenkins2714096223939441451.sh ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0 collected 2 items |
This PR runs integration tests for Merlin Models: