Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download Kaggle datasets #1487

Merged

Conversation

itrushkin
Copy link
Contributor

Summary

CVS-137601
Adds the downloading logic for Kaggle datasets.
258 pre-defined configurations are implemented.

Kaggle API has a request limit, so fetching the metadata for each dataset is still unresolved.

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added the description of my changes into CHANGELOG.​
  • I have updated the documentation accordingly

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT

Signed-off-by: Ilya Trushkin <ilya.trushkin@intel.com>
@itrushkin itrushkin requested review from a team as code owners May 2, 2024 13:21
@itrushkin itrushkin requested review from jihyeonyi and removed request for a team May 2, 2024 13:21
Signed-off-by: Ilya Trushkin <ilya.trushkin@intel.com>
Signed-off-by: Ilya Trushkin <ilya.trushkin@intel.com>
Signed-off-by: Ilya Trushkin <ilya.trushkin@intel.com>
Signed-off-by: Ilya Trushkin <ilya.trushkin@intel.com>
Copy link

codecov bot commented May 2, 2024

Codecov Report

Attention: Patch coverage is 34.89362% with 153 lines in your changes are missing coverage. Please review.

Project coverage is 80.62%. Comparing base (44cc56a) to head (2ee4699).
Report is 50 commits behind head on develop.

Files Patch % Lines
src/datumaro/cli/commands/downloaders/tfds.py 29.89% 68 Missing ⚠️
src/datumaro/cli/commands/downloaders/kaggle.py 32.85% 47 Missing ⚠️
src/datumaro/plugins/data_formats/kaggle/base.py 29.03% 20 Missing and 2 partials ⚠️
src/datumaro/cli/commands/download.py 33.33% 10 Missing ⚠️
src/datumaro/plugins/data_formats/imagenet.py 33.33% 4 Missing ⚠️
src/datumaro/components/project.py 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1487      +/-   ##
===========================================
- Coverage    80.85%   80.62%   -0.24%     
===========================================
  Files          271      276       +5     
  Lines        30689    31320     +631     
  Branches      6197     6303     +106     
===========================================
+ Hits         24815    25251     +436     
- Misses        4489     4656     +167     
- Partials      1385     1413      +28     
Flag Coverage Δ
ubuntu-20.04_Python-3.10 80.60% <34.89%> (-0.24%) ⬇️
windows-2022_Python-3.10 80.59% <34.89%> (-0.24%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@wonjuleee wonjuleee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but could you add document for download? This could be done in the separate PR.

@yunchu yunchu added this to the 1.7.0 milestone May 7, 2024
Copy link
Contributor

@jihyeonyi jihyeonyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but could you add test code more later?
It seems that we need to cover more lines in test code.

@wonjuleee wonjuleee merged commit 6e89b4f into openvinotoolkit:develop May 7, 2024
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants