Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python] Append-mode enum bit-width mods #2455

Merged
merged 6 commits into from
May 6, 2024
Merged

[python] Append-mode enum bit-width mods #2455

merged 6 commits into from
May 6, 2024

Conversation

johnkerl
Copy link
Member

@johnkerl johnkerl commented Apr 19, 2024

Issue and/or context: Presented in detail on #2456.

Changes: As on #2456.

Notes for Reviewer: See also updates below.

@johnkerl
Copy link
Member Author

johnkerl commented May 2, 2024

Update: @nguyenv 's recent cppification has fixed the case when a larger-bit-width enum exists on disk, with a smaller-bit-width append -- this now works correctly.

The only remaining issue is: when a smaller bit width is done for the initial ingest, and then we overflow.

  • For thetiledbsoma API: this is intentional: if a user supplies say int8 index type in their user-provided Arrow schema and user-provided data exceed that capacity, then we must fail -- and we do
  • For the tiledbsoma.io API: users do not pass in Arrow schema -- they simply pass in AnnData objects and they are entitled to expect appendability. The necessary step here is for tiledbsoma.io to broaden our inference for index types to int32 -- regardless of the initial input data -- since any enumerated column on any dataset could be appended to.

Copy link

codecov bot commented May 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.19%. Comparing base (94bfd1c) to head (83fd5c4).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2455      +/-   ##
==========================================
+ Coverage   89.87%   90.19%   +0.31%     
==========================================
  Files          37       37              
  Lines        3941     3988      +47     
==========================================
+ Hits         3542     3597      +55     
+ Misses        399      391       -8     
Flag Coverage Δ
python 90.19% <100.00%> (+0.31%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
python_api 90.19% <100.00%> (+0.31%) ⬆️
libtiledbsoma ∅ <ø> (∅)

@johnkerl johnkerl changed the title [python] Append-mode enum bit-width mods [WIP] [python] Append-mode enum bit-width mods May 3, 2024
@johnkerl johnkerl marked this pull request as ready for review May 3, 2024 20:03
@johnkerl johnkerl requested a review from nguyenv May 6, 2024 15:19
@johnkerl
Copy link
Member Author

johnkerl commented May 6, 2024

@nguyenv 83fd5c4

@johnkerl johnkerl merged commit a5d40d1 into main May 6, 2024
12 checks passed
@johnkerl johnkerl deleted the kerl/enum-append branch May 6, 2024 15:51
github-actions bot pushed a commit that referenced this pull request May 6, 2024
* [python] Append-mode enum bit-width mods [WIP]

* lint

* Fix remaining issue

* unit-test updates

* complete the unit-test mod
johnkerl added a commit that referenced this pull request May 6, 2024
* [python] Append-mode enum bit-width mods [WIP]

* lint

* Fix remaining issue

* unit-test updates

* complete the unit-test mod

Co-authored-by: John Kerl <kerl.john.r@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants