Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add environment checks #233

Merged
merged 4 commits into from
Jul 7, 2023
Merged

Add environment checks #233

merged 4 commits into from
Jul 7, 2023

Conversation

KSGulin
Copy link
Contributor

@KSGulin KSGulin commented Jun 23, 2023

This PR adds environment checks that will help detect a broken sparsify environment before a run is fully launched. There are three pathways by which environment checks are launched:

  • The CLI command sparsify.check_environment will run all health checks
  • Running an experiment with sparsify.run training-aware or sparsify.run sparse-transfer will run health checks relevant to these two pathways
  • Running an experiment with sparsify.run one-shot will run the health checks relevant to one-shot

Currently implemented health checks
GPU device check - this is a non-failing check, but it will raise a loud error
ORT health check - this is a failing check which will point the user to the ORT-GPU dependency matrix

Test Plan

  • sparsify.check_environment
  • CUDA_VISIBLE_DEVICES="" sparsify.check_environment
  • sparsify.run training-aware --use-case image-classification --model mobilenet-v1 --data /network/datasets/imagenet --optim-level 0
  • sparsify.run one-shot --model model.onnx --data coco_calibration --use-case cv-detection --optim-level 0.5
  • Test on system with broken ORT-GPU environment

@KSGulin KSGulin self-assigned this Jun 23, 2023
bfineran
bfineran previously approved these changes Jun 26, 2023
Base automatically changed from sparsify.alpha to main June 30, 2023 19:21
@bfineran bfineran dismissed their stale review June 30, 2023 19:21

The base branch was changed.

@KSGulin KSGulin merged commit cd5a938 into main Jul 7, 2023
@KSGulin KSGulin deleted the health_checks branch July 7, 2023 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants