Lazy list for array field #8229
Conversation
Handle list modifications and throw away used up string.
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThe recent changes enhance the Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
1. Support pickling 2. Handle cases where LazyList is extended with/added to another LazyList 3. Fix cases when unparsed LazyList was being compared result 4. Fix cases when deep copying resulted in duplicate values
![coderabbitai[bot]](https://github.com/avatars/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- cvat/apps/engine/models.py (3 hunks)
- cvat/apps/engine/tests/test_lazy_list.py (1 hunks)
Additional comments not posted (28)
cvat/apps/engine/tests/test_lazy_list.py (28)
16-19: LGTM!The
test_skipped_valuesfunction correctly tests the handling of skipped values in theLazyList.
21-24: LGTM!The
test_lenfunction correctly tests the length of theLazyList.
26-31: LGTM!The
test_reprfunction correctly tests the string representation of theLazyList.
33-37: LGTM!The
test_deepcopyfunction correctly tests the deep copy functionality of theLazyList.
39-41: LGTM!The
test_getitemfunction correctly tests the item access functionality of theLazyList.
43-49: LGTM!The
test_iterfunction correctly tests the iterator functionality of theLazyList.
51-53: LGTM!The
test_appendfunction correctly tests the append functionality of theLazyList.
55-57: LGTM!The
test_copyfunction correctly tests the copy functionality of theLazyList.
59-61: LGTM!The
test_insertfunction correctly tests the insert functionality of theLazyList.
63-66: LGTM!The
test_popfunction correctly tests the pop functionality of theLazyList.
68-70: LGTM!The
test_removefunction correctly tests the remove functionality of theLazyList.
72-74: LGTM!The
test_reversefunction correctly tests the reverse functionality of theLazyList.
76-79: LGTM!The
test_sortfunction correctly tests the sort functionality of theLazyList.
81-83: LGTM!The
test_clearfunction correctly tests the clear functionality of theLazyList.
85-86: LGTM!The
test_indexfunction correctly tests the index functionality of theLazyList.
88-89: LGTM!The
test_countfunction correctly tests the count functionality of theLazyList.
91-93: LGTM!The
test_setitemfunction correctly tests the set item functionality of theLazyList.
95-97: LGTM!The
test_delitemfunction correctly tests the delete item functionality of theLazyList.
99-100: LGTM!The
test_containsfunction correctly tests the contains functionality of theLazyList.
102-103: LGTM!The
test_reversedfunction correctly tests the reversed functionality of theLazyList.
105-106: LGTM!The
test_mulfunction correctly tests the multiplication functionality of theLazyList.
108-109: LGTM!The
test_rmulfunction correctly tests the reverse multiplication functionality of theLazyList.
111-113: LGTM!The
test_imulfunction correctly tests the in-place multiplication functionality of theLazyList.
115-117: LGTM!The
test_extendfunction correctly tests the extend functionality of theLazyList.
119-121: LGTM!The
test_addfunction correctly tests the addition functionality of theLazyList.
123-124: LGTM!The
test_eqfunction correctly tests the equality functionality of theLazyList.
126-128: LGTM!The
test_iaddfunction correctly tests the in-place addition functionality of theLazyList.
130-162: LGTM!The test functions correctly test the respective functionalities of the
LazyList.
<!-- Raise an issue to propose your change (https://github.com/cvat-ai/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context Decided to split changes in this PR: #8223 1. Annotations import (#8226) 2. Array fields optimization (#8229) 3. Logging function optimization (this PR) ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved data integrity by ensuring annotations change handling occurs every time an item is deleted. - **New Features** - Enhanced data filtering logic for annotations, allowing for more efficient processing without unnecessary deep copies. - Introduced a new function for streamlined filtering of track data, simplifying data management. - **Refactor** - Consolidated data filtering into reusable functions for better maintainability and clarity. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Raise an issue to propose your change (https://github.com/cvat-ai/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context Decided to split changes in this PR: #8223 1. Annotations import (this PR) 2. Array fields optimization (#8229) 3. Logging function optimization (#8228) ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Improved data type consistency for shape points during the annotation loading process, ensuring all coordinates are treated as numeric values. - Enhanced clarity and reliability in loading various shape types (box, ellipse, cuboid, mask) by converting relevant attributes to floats. - **Bug Fixes** - Removed redundant database initialization calls during annotation import, potentially improving the efficiency of the import process. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #8229 +/- ##
===========================================
+ Coverage 83.30% 83.40% +0.09%
===========================================
Files 388 389 +1
Lines 41352 41526 +174
Branches 3856 3856
===========================================
+ Hits 34450 34636 +186
+ Misses 6902 6890 -12
|
|
<!-- Raise an issue to propose your change (https://github.com/cvat-ai/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context Following [v2.16.2](https://github.com/cvat-ai/cvat/releases/tag/v2.16.2) release, we noticed an issue with annotations export with CVAT format and masks. Upon investigation, we found that the problem stemmed from changes introduced in this PR: #8229. The issue was caused by the improper handling of slices with negative positions in LazyList, particularly affecting the computation of resulting data. This behaviour was triggered by the following line: https://github.com/cvat-ai/cvat/blob/cfce7be5a9422da2c367296f56d70181185f503e/cvat/apps/dataset_manager/formats/cvat.py#L797 <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [x] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [x] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Enhanced handling of negative indices in list slicing, improving robustness and performance. - Added new test methods to validate slicing operations, ensuring correct behavior for various scenarios. - **Bug Fixes** - Adjusted existing tests to account for empty values in the list, improving accuracy of expected outputs. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Motivation and context
Decided to split changes in this PR: #8223
How has this been tested?
Checklist
developbranch(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.
Summary by CodeRabbit
New Features
LazyListclass for efficient, on-demand parsing of list elements from strings.Tests
LazyListclass, validating core functionalities and ensuring robustness.