Skip to content

Commit

Permalink
default max file size to 16GB
Browse files Browse the repository at this point in the history
  • Loading branch information
bfineran committed Aug 29, 2023
1 parent 1018b13 commit ad52e32
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions src/sparsezoo/utils/onnx/external_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,9 @@ def load_model(model: Union[str, ModelProto]) -> ModelProto:


def split_external_data(
model_path: str, max_file_size: int, allow_large_tensors: bool = True
model_path: str,
max_file_size: int = 16e9,
allow_large_tensors: bool = True,
):
"""
Splits the external_data_path file into multiple files of size no larger than
Expand All @@ -184,7 +186,8 @@ def split_external_data(
:param model_path: path to ONNX model file who has external data writen
to a single file in the same directory
:param max_file_size: maximum file size in bytes of a single split out file
:param max_file_size: maximum file size in bytes of a single split out file.
defaults to 16000000000 (16e9 = 16GB)
:param allow_large_tensors: if False, will raise an exception if any model tensor
is larger than max_file_size. If True, will write the large tensor to a single
file regardless of max_file_size. Default True
Expand Down

0 comments on commit ad52e32

Please sign in to comment.