[FEATURE-BRANCH] ImageField: add support to new fields of type image #5279
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #5279 +/- ##
===========================================
- Coverage 91.20% 91.12% -0.09%
===========================================
Files 139 139
Lines 5745 5816 +71
===========================================
+ Hits 5240 5300 +60
- Misses 505 516 +11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
…ine (#5355) # Description <!-- Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change. --> This PR adds support to indexing datasets and records with image fields. Note: A datasets reindex is required with this change. **Type of change** <!-- Please delete options that are not relevant. Remember to title the PR according to the type of change --> - New feature (non-breaking change which adds functionality) **How Has This Been Tested** <!-- Please add some reference about how your feature has been tested. --> **Checklist** <!-- Please go over the list and make sure you've taken everything into account --> - I added relevant documentation - I followed the style guidelines of this project - I did a self-review of my code - I made corresponding changes to the documentation - I confirm My changes generate no new warnings - I have added tests that prove my fix is effective or that my feature works - I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)
frascuchon
reviewed
Aug 2, 2024
...tests/unit/api/handlers/v1/datasets/records/records_bulk/test_create_dataset_records_bulk.py
Show resolved
Hide resolved
# Description <!-- Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change. --> This PR adds support to work with image fields using the SDK. ## TODO (as separate PRs) - [ ] Add documentation section - [ ] Verify behavior using from/to_hub methods - [ ] Add tooling to read data URLs from image files/folders (based on logic defined in `image_to_html`) - [ ] Add some media type validation (supported image types) - [ ] [Optional] Add tooling to re-scale images. The lower the image sizes, the better the UI work. We can define a transparent re-scaling process to support large images (currently the limit size in the backend is 5MB for data URL) **Type of change** <!-- Please delete options that are not relevant. Remember to title the PR according to the type of change --> - New feature (non-breaking change which adds functionality) **How Has This Been Tested** <!-- Please add some reference about how your feature has been tested. --> **Checklist** <!-- Please go over the list and make sure you've taken everything into account --> - I added relevant documentation - I followed the style guidelines of this project - I did a self-review of my code - I made corresponding changes to the documentation - I confirm My changes generate no new warnings - I have added tests that prove my fix is effective or that my feature works - I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/) --------- Co-authored-by: José Francisco Calvo <jose@argilla.io>
burtenshaw
changed the title
...tests/unit/api/handlers/v1/datasets/records/records_bulk/test_create_dataset_records_bulk.py
Show resolved
Hide resolved
Co-authored-by: David Berenstein <david.m.berenstein@gmail.com>
…illa-io/argilla into feat/add-image-field-support
|
Docs for this PR have been deployed hidden from versioning: https://docs.argilla.io/docs_5405-docs-tutorial-on-the-usage-of-image-fields |
# Description <!-- Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change. --> This PR changes the mapper to identify fields of all types (not just text). Closes #5406 **Type of change** <!-- Please delete options that are not relevant. Remember to title the PR according to the type of change --> - Bug fix (non-breaking change which fixes an issue) - New feature (non-breaking change which adds functionality) - Breaking change (fix or feature that would cause existing functionality to not work as expected) - Refactor (change restructuring the codebase without changing functionality) - Improvement (change adding some improvement to an existing functionality) - Documentation update **How Has This Been Tested** <!-- Please add some reference about how your feature has been tested. --> **Checklist** <!-- Please go over the list and make sure you've taken everything into account --> - I added relevant documentation - I followed the style guidelines of this project - I did a self-review of my code - I made corresponding changes to the documentation - I confirm My changes generate no new warnings - I have added tests that prove my fix is effective or that my feature works - I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)
# Description <!-- Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change. --> Closes #<issue_number> **Type of change** <!-- Please delete options that are not relevant. Remember to title the PR according to the type of change --> - Bug fix (non-breaking change which fixes an issue) - New feature (non-breaking change which adds functionality) - Breaking change (fix or feature that would cause existing functionality to not work as expected) - Refactor (change restructuring the codebase without changing functionality) - Improvement (change adding some improvement to an existing functionality) - Documentation update **How Has This Been Tested** <!-- Please add some reference about how your feature has been tested. --> **Checklist** <!-- Please go over the list and make sure you've taken everything into account --> - I added relevant documentation - I followed the style guidelines of this project - I did a self-review of my code - I made corresponding changes to the documentation - I confirm My changes generate no new warnings - I have added tests that prove my fix is effective or that my feature works - I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)
jfcalvo
changed the title
frascuchon
reviewed
Aug 26, 2024
Co-authored-by: Paco Aranda <frascuchon@gmail.com>
frascuchon
reviewed
Aug 26, 2024
| ) | ||
| ``` | ||
|
|
||
| 1. The image can be referenced as either a remote url or a data uri. |
There was a problem hiding this comment.
Adding an example using data URI would be great @sdiazlor
There was a problem hiding this comment.
@burtenshaw maybe we can include here the image_to_data_uri (or whatever) util function usage once you have implemented it
frascuchon
reviewed
Aug 27, 2024
| fields={"image": "https://example.com/image.jpg"}, | ||
| ), | ||
| rg.Record( | ||
| fields={"image": "data:image/png;base64,iV..."}, # (1) |
There was a problem hiding this comment.
same here about the imate-to-data-uri function
…illa-io/argilla into feat/add-image-field-support
# Description <!-- Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change. --> This PR introduces a utility into the SDK so that it automatically casts PIL objects to base64 based on the datasets `Features` **Type of change** <!-- Please delete options that are not relevant. Remember to title the PR according to the type of change --> - Bug fix (non-breaking change which fixes an issue) - New feature (non-breaking change which adds functionality) - Breaking change (fix or feature that would cause existing functionality to not work as expected) - Refactor (change restructuring the codebase without changing functionality) - Improvement (change adding some improvement to an existing functionality) - Documentation update **How Has This Been Tested** <!-- Please add some reference about how your feature has been tested. --> **Checklist** <!-- Please go over the list and make sure you've taken everything into account --> - I added relevant documentation - I followed the style guidelines of this project - I did a self-review of my code - I made corresponding changes to the documentation - I confirm My changes generate no new warnings - I have added tests that prove my fix is effective or that my feature works - I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
# Description <!-- Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change. --> Closes #5405 **Type of change** <!-- Please delete options that are not relevant. Remember to title the PR according to the type of change --> - Documentation update **How Has This Been Tested** <!-- Please add some reference about how your feature has been tested. --> **Checklist** <!-- Please go over the list and make sure you've taken everything into account --> - I added relevant documentation - I followed the style guidelines of this project - I did a self-review of my code - I made corresponding changes to the documentation - I confirm My changes generate no new warnings - I have added tests that prove my fix is effective or that my feature works - I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/) --------- Co-authored-by: burtenshaw <ben@argilla.io> Co-authored-by: Paco Aranda <frascuchon@gmail.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is the first one related with the addition of a new field of type
imageincluding the following changes:imagefield type supporting URLs and Data URLs.imagefield for records.imagefields and create records specifying images.This new type will have the following characteristics:
imageis supported (without additional settings).imageare validated to have correct Web URLs or data URLs like:https://argilla.io/image.jpeg:HTTPSschema is present2038characters.http://argilla.io/image.jpeg:HTTPschema is present2038characters.data:image/webp;base64,UklGRhgCAABXRUJQVlA4WA:dataschema is present5million characters.Closes #5276
Type of change
How Has This Been Tested
Checklist