Allow Importing Multiple Documents from a single GCS Path #214
Comments
holtskinner
added a commit
that referenced
this issue
Dec 12, 2023
… documents from a GCS directory. Fixes #214 - Note: `from_gcs()` takes in a GCS directory, but it only works for a single sharded document from a single input document source. - In a GA release, it would be a better practice to have `from_gcs()` take in any GCS directory and output a list of Wrapped Documents. But this would be a backwards-incompatible change now. - Not sure if it's possible/advisable to have two possible return types for `from_gcs()` and just have it return a list when there are multiple Wrapped Documents?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Inspired by https://stackoverflow.com/a/77609221/6216983
The
from_gcs()method can only create a single Wrapped Document from a single document output in GCS.It could be simpler for users if this method could output a list of Wrapped Documents from a GCS path similar to
from_batch_process_operation()andfrom_batch_process_metadata().However, this would be backwards incompatible, so an alternative is to create a new method for this functionality. (Not sure what the best name would be)
The text was updated successfully, but these errors were encountered: