Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Daily tests #1870

Merged
merged 74 commits into from
Jan 10, 2023
Merged

Daily tests #1870

merged 74 commits into from
Jan 10, 2023

Conversation

mvpatel2000
Copy link
Contributor

@mvpatel2000 mvpatel2000 commented Jan 9, 2023

What does this PR do?

Migrates daily tests to GHA. Now, tests marked with @pytest.mark.daily will run every time a PR is merged into dev/main and every night at 2:30am (github recommends picking a randomish time as opposed to midnight to avoid load spikes).

Along for the ride, a few tweaks to notebook tests to speed them up and avoid OOMs in the smaller GHA machines.

To verify tests passed, I opened a PR in my fork and changed workflow to run on pull requests as well. mvpatel2000#26 Commit change workflows has all tests passing before I set it to only run on merges (push) and scheduled cron

Note: This is still only CPU tests. GPU tests will be added later

Note: We currently run all pytest.mark.remote tests only in the daily tests because hitting real servers is expensive. We maintain the remote marker in this PR though since this decision might be changed later.

What issue(s) does this change relate to?

CO-1519

@mvpatel2000 mvpatel2000 requested a review from a team as a code owner January 9, 2023 16:23
@mvpatel2000 mvpatel2000 changed the title Mvpatel2000/nightly Daily tests Jan 9, 2023
Copy link
Contributor

@dakinggg dakinggg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, with one question. Should the daily be running remote tests too? I think yes, and it looks like you have it set up for that with all of the secrets youre passing.

Copy link
Contributor

@bandish-shah bandish-shah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mvpatel2000 mvpatel2000 merged commit ba1a939 into mosaicml:dev Jan 10, 2023
@mvpatel2000 mvpatel2000 deleted the mvpatel2000/nightly branch January 10, 2023 05:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants