Skip to content

Commit

Permalink
just cast straight to string for ds hash
Browse files Browse the repository at this point in the history
  • Loading branch information
winglian committed Jan 22, 2024
1 parent 1c1a732 commit 30f49b6
Showing 1 changed file with 2 additions and 8 deletions.
10 changes: 2 additions & 8 deletions src/axolotl/utils/data.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,15 +116,9 @@ def load_tokenized_prepared_datasets(
(
str(cfg.sequence_len)
+ "@"
+ str(int(cfg.sample_packing))
+ str(cfg.sample_packing)
+ "@"
+ str(
int(
-1
if (cfg.eval_sample_packing is None)
else cfg.eval_sample_packing
)
)
+ str(cfg.eval_sample_packing)
+ "@"
+ str(int(cfg.group_by_length))
+ "@"
Expand Down

0 comments on commit 30f49b6

Please sign in to comment.