Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor CLI commands w/ and w/o project #910

Merged

Conversation

vinnamkim
Copy link
Contributor

@vinnamkim vinnamkim commented Apr 3, 2023

Summary

  • Ticket no. 107276
  • Refactor CLI commands w/ and w/o project

To-be (w/o project)

(datumaro) vinnamki@vinnamki:~/datumaro$ datum -h
usage: datum [-h] [--version] [--loglevel LOGLEVEL]

Dataset Framework

optional arguments:
 -h, --help           show this help message and exit
 --version            show program's version number and exit
 --loglevel LOGLEVEL  Logging level (options: debug, info, warning, error, critical; default: info)

Contexts:
 model                Actions with models
 project              Actions with projects
 source               Actions with data sources
 util                 Auxillary tools and utilities

Basic Commands:
 convert              Convert dataset between formats
 detect-format        Detect the format of a dataset
 download             Download a publicly available dataset
 explain              Run Explainable AI algorithm for model
 filter               Filter dataset items
 generate             Generate synthetic dataset
 merge                Merge datasets
 patch                Update dataset from another one
 search               Search similar datasetitems of query

Run 'datum COMMAND --help' for more information on a command.:

To-be (w/ project)

(datumaro) vinnamki@vinnamki:~/datumaro$ datum project -h
usage: datum project [-h]

Manipulate projects.

By default, the project to be operated on is searched for in the current directory. An additional '-p' argument can be passed to specify project location.

optional arguments:
  -h, --help  show this help message and exit

Project modification:
  add                  Add dataset
  create               Create empty project
  import               Import dataset
  remove               Remove dataset
Project versioning:  
  checkout             Switch to another branch or revision
  commit               Commit changes in tracked files
  log                  List history
  status               Display current status
  pinfo                Print project info
Dataset operations:  
  diff                 Compare datasets
  export               Export dataset in some format
  stats                Compute dataset statistics
  transform            Modify dataset items
  validate             Validate dataset
  dinfo                Print dataset info

Run 'datum project COMMAND --help' for more information on a command.:

As-is

(datumaro) vinnamki@vinnamki:~/datumaro$ datum -h
usage: datum [-h] [--version] [--loglevel LOGLEVEL]

Dataset Framework

optional arguments:
  -h, --help           show this help message and exit
  --version            show program's version number and exit
  --loglevel LOGLEVEL  Logging level (options: debug, info, warning, error, critical; default: info)

Contexts:
  model                Actions with models
  project              Actions with projects
  source               Actions with data sources
  util                 Auxillary tools and utilities

Commands:
  Project modification:
  add                  Add dataset
  create               Create empty project
  import               Import dataset
  remove               Remove dataset
                       
  Project versioning:  
  checkout             Switch to another branch or revision
  commit               Commit changes in tracked files
  log                  List history
  status               Display current status
                       
  Dataset operations:  
  convert              Convert dataset between formats
  describe-downloads   Print information about downloadable datasets
  detect-format        Detect the format of a dataset
  diff                 Compare datasets
  download             Download a publicly available dataset
  explain              Run Explainable AI algorithm for model
  export               Export dataset in some format
  filter               Filter dataset items
  generate             Generate synthetic dataset
  info                 Print dataset info
  merge                Merge datasets
  patch                Update dataset from another one
  stats                Compute dataset statistics
  transform            Modify dataset items
  validate             Validate dataset
  search               Search similar datasetitems of query

Run 'datum COMMAND --help' for more information on a command.:

How to test

This change will be covered by the modified integration/cli tests.

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added the description of my changes into CHANGELOG.​
  • I have updated the documentation accordingly

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

 - "datum describe-downloads" -> "datum download describe"
 - "datum download" -> "datum download get"
 - Refactor datum download cli test to use pytest framework

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim marked this pull request as ready for review April 3, 2023 11:17
@vinnamkim vinnamkim requested review from a team as code owners April 3, 2023 11:17
@vinnamkim vinnamkim added this to the 1.2.0 milestone Apr 3, 2023
@codecov-commenter
Copy link

codecov-commenter commented Apr 3, 2023

Codecov Report

Patch coverage: 31.14% and project coverage change: -0.01 ⚠️

Comparison is base (3e0cbf0) 78.88% compared to head (f9ad058) 78.87%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #910      +/-   ##
===========================================
- Coverage    78.88%   78.87%   -0.01%     
===========================================
  Files          210      215       +5     
  Lines        25065    25120      +55     
  Branches      5032     5039       +7     
===========================================
+ Hits         19773    19814      +41     
- Misses        4159     4173      +14     
  Partials      1133     1133              
Flag Coverage Δ
macos-11_Python-3.8 77.86% <31.14%> (-0.01%) ⬇️
ubuntu-20.04_Python-3.8 78.86% <31.14%> (-0.01%) ⬇️
windows-2019_Python-3.8 78.81% <31.14%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
datumaro/cli/__main__.py 0.00% <0.00%> (ø)
datumaro/cli/util/diff.py 20.00% <ø> (ø)
datumaro/util/telemetry_utils.py 45.83% <0.00%> (ø)
datumaro/cli/contexts/project/__init__.py 39.34% <11.76%> (+14.95%) ⬆️
datumaro/cli/util/__init__.py 26.22% <12.50%> (-2.08%) ⬇️
...ro/cli/commands/require_project/versioning/info.py 19.56% <19.56%> (ø)
...i/commands/require_project/modification/import_.py 20.89% <20.89%> (ø)
...o/cli/commands/require_project/modification/add.py 21.66% <21.66%> (ø)
...mands/require_project/dataset_operations/export.py 23.94% <23.94%> (ø)
...mmands/require_project/dataset_operations/stats.py 31.70% <31.70%> (ø)
... and 20 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@vinnamkim vinnamkim mentioned this pull request Apr 4, 2023
6 tasks
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim marked this pull request as draft April 5, 2023 05:10
 - datumaro/cli/commands <- do not require a project
 - datumaro/cli/commands/require_project <- require a project

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim marked this pull request as ready for review April 5, 2023 06:19
@vinnamkim
Copy link
Contributor Author

It's ready. Please review this PR.

@vinnamkim vinnamkim merged commit edfd106 into openvinotoolkit:develop Apr 6, 2023
@vinnamkim vinnamkim deleted the refactor/cli-w-and-wo-project branch April 6, 2023 07:56
vinnamkim added a commit that referenced this pull request Apr 19, 2023
 - Ticket no. 108466
- While working on #949,
I realized that several commands grouped by `dataset operations` in
`datum project` are actually project free commands (previously changed
in #910).
 - So, I decided to move them outside of `datum project` again.
- Aligning in terms of documentation to these changes will be made in
the next PR.

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim mentioned this pull request Apr 19, 2023
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants