Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make PhysicalKey a facade above s3://, file:// and (TODO) sharepoint:// #4064

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

fiskus
Copy link
Member

@fiskus fiskus commented Jul 31, 2024

Description

TODO:

  • Pass location to PhysicalKey instead of bucket, path, version_id
  • location is a S3ObjectLocation (s3://) or LocalLocation (file://)
  • Tests succeed
  • location can be SharePointLocation (sharepoint://)
  • Tests succeed

  • Unit tests
  • Automated tests (e.g. Preflight)
  • Confirm that this change meets security best practices and does not violate the security model
  • Documentation
    • Python: Run build.py for new docstrings
    • JavaScript: basic explanation and screenshot of new features
    • Markdown somewhere in docs/**/*.md that explains the feature to end users (said .md files should be linked from SUMMARY.md so they appear on https://docs.quiltdata.com)
    • Markdown docs for developers
  • Changelog entry (skip if change is not significant to end users, e.g. docs only)

Copy link

codecov bot commented Jul 31, 2024

Codecov Report

Attention: Patch coverage is 92.25806% with 12 lines in your changes missing coverage. Please review.

Project coverage is 37.29%. Comparing base (68bfbc3) to head (676d737).
Report is 8 commits behind head on master.

Files Patch % Lines
api/python/quilt3/util.py 90.55% 12 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4064      +/-   ##
==========================================
- Coverage   38.47%   37.29%   -1.18%     
==========================================
  Files         717      711       -6     
  Lines       33023    32413     -610     
  Branches     4837     4837              
==========================================
- Hits        12704    12090     -614     
+ Misses      19695    19176     -519     
- Partials      624     1147     +523     
Flag Coverage Δ
api-python 90.66% <92.25%> (-0.09%) ⬇️
catalog 11.51% <ø> (ø)
lambda 86.10% <ø> (-1.86%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sir-sigurd
Copy link
Member

@fiskus
you may want to have a look on my previous attempt to split S3 and local physical keys
#1982

@drernie
Copy link
Member

drernie commented Sep 9, 2024

I wonder... should we instead just:

  1. Use the OpenDAL Python bindings for our physical keys
  2. Use the Rust SDK to add SharePoint support to OpenDAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants