Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python] Append mode with resizing #3148

Merged
merged 4 commits into from
Oct 18, 2024
Merged

[python] Append mode with resizing #3148

merged 4 commits into from
Oct 18, 2024

Conversation

johnkerl
Copy link
Member

@johnkerl johnkerl commented Oct 8, 2024

Issue and/or context: As tracked on issue #2407 / [sc-51048].

Note that the intended Python and R API changes are all agreed on and finalized as described in #2407.

Changes:

Support experipment-level upgrade_shape and resize.

Notes for Reviewer:

@johnkerl johnkerl changed the base branch from main to kerl/py-resizer-connects October 8, 2024 16:31
Copy link

codecov bot commented Oct 8, 2024

Codecov Report

Attention: Patch coverage is 86.95652% with 3 lines in your changes missing coverage. Please review.

Project coverage is 83.29%. Comparing base (22aa800) to head (251542b).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3148      +/-   ##
==========================================
+ Coverage   83.18%   83.29%   +0.10%     
==========================================
  Files          51       51              
  Lines        5483     5495      +12     
==========================================
+ Hits         4561     4577      +16     
+ Misses        922      918       -4     
Flag Coverage Δ
python 83.29% <86.95%> (+0.10%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
python_api 83.29% <86.95%> (+0.10%) ⬆️
libtiledbsoma ∅ <ø> (∅)

Base automatically changed from kerl/py-resizer-connects to main October 8, 2024 18:19
@johnkerl johnkerl force-pushed the kerl/py-exp-resize branch 3 times, most recently from 73934a7 to 8938131 Compare October 9, 2024 00:16
@johnkerl johnkerl mentioned this pull request Oct 9, 2024
@johnkerl johnkerl changed the base branch from main to kerl/py-can-upgrade-shape October 9, 2024 00:25
@johnkerl johnkerl force-pushed the kerl/py-exp-resize branch 12 times, most recently from 6f1fe0e to 57b7263 Compare October 9, 2024 03:46
@johnkerl johnkerl force-pushed the kerl/py-exp-resize branch 2 times, most recently from f269ee9 to adb5dc7 Compare October 9, 2024 03:59
@johnkerl johnkerl changed the base branch from kerl/py-can-upgrade-shape to kerl/registration-shape-acceessors October 9, 2024 04:01
@johnkerl johnkerl force-pushed the kerl/py-exp-resize branch 2 times, most recently from d4cd94e to bf17979 Compare October 9, 2024 04:27
Base automatically changed from kerl/registration-shape-acceessors to main October 9, 2024 13:51
@johnkerl johnkerl changed the title [python] Experiment-level shaping [WIP] [python] Append mode with resizing [WIP] Oct 9, 2024
@johnkerl johnkerl changed the base branch from main to kerl/py-exp-shaping2 October 9, 2024 20:56
Base automatically changed from kerl/py-exp-shaping2 to main October 10, 2024 03:43
@johnkerl johnkerl force-pushed the kerl/py-exp-resize branch 2 times, most recently from 08ac2d7 to 8d2a773 Compare October 15, 2024 13:53
@johnkerl johnkerl force-pushed the kerl/py-exp-resize branch 2 times, most recently from a122eac to 1c3b958 Compare October 16, 2024 23:07
@johnkerl johnkerl changed the title [python] Append mode with resizing [WIP] [python] Append mode with resizing Oct 16, 2024
@johnkerl johnkerl marked this pull request as ready for review October 16, 2024 23:43
Copy link
Collaborator

@jp-dark jp-dark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good, but I'm not super familiar with the ingestor code so you may want to want to wait for @nguyenv to review as well.

One question: Is that resize code required before ingesting with the new shape feature? If so, what error does the user get if they fail to resize first?

@johnkerl
Copy link
Member Author

One question: Is that resize code required before ingesting with the new shape feature? If so, what error does the user get if they fail to resize first?

@jp-dark they get a message like this:

tiledbsoma._exception.SOMAError: TileDB internal: ['UnorderedWriter::dowork]  (FragmentMetadata: Cells are written outside of the defined current domain.)

which we can perhaps catch and rethrow with a friendlier error message ...

@johnkerl johnkerl merged commit 7336955 into main Oct 18, 2024
@johnkerl johnkerl deleted the kerl/py-exp-resize branch October 18, 2024 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants