-
Notifications
You must be signed in to change notification settings - Fork 281
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DRAFT] FEAT Update Data Type Serializer Functionality to Support MultiModal #276
base: main
Are you sure you want to change the base?
Conversation
else: | ||
raise ValueError(f"Data type {data_type} without prompt text not supported") | ||
|
||
|
||
class DiskStorageIO(StorageIO): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is good design, great work
container_client = self.azure_blob_service_client.get_container_client(container=self.container_name) | ||
matched_blobs = container_client.list_blobs(name_starts_with=path) | ||
# TODO: Does this logic really work? | ||
return len(matched_blobs) > 1 and any(blob.name == str(path) for blob in matched_blobs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would it be > 0?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before closing please verify the case where there is just one blob :)
self.azure_blob_service_client = azure_blob_service_client | ||
self.container_name = container_name | ||
|
||
def read(self, path: Path | str) -> bytes: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we make all these methods async? I think it's probably worth it to change to async only
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
work with Raja though, I won't block on it. I am okay with sync, just not my pref
Description
Enhance the
data_type_serializer
functionality to support multimodal capabilities using Azure SQL Database and Azure Storage Account deployed.CC: @rdheekonda @rlundeen2
Tests and Documentation