examples/mistral/config.yml errors on default configuration

[kmsydney]

Please check that this issue hasn't been reported before.

• I searched previous Bug Reports didn't find any similar reports.

Expected Behavior

For new users, running the example mistral/config.yaml should attempt to do something useful out-of-the-box without editing the config (from the README):

accelerate launch -m axolotl.cli.train examples/mistral/config.yml

Current behaviour

Fails to run with the following error:

  File "/mnt/task_runtime/axolotl/src/axolotl/cli/__init__.py", line 315, in load_datasets
    train_dataset, eval_dataset, total_num_steps, prompters = prepare_dataset(
                                                              ^^^^^^^^^^^^^^^^
  File "/mnt/task_runtime/axolotl/src/axolotl/utils/data.py", line 86, in prepare_dataset
    raise ValueError(
ValueError: eval dataset split is too small for sample_packing. You should set `eval_sample_packing: False`.

Steps to reproduce

accelerate launch -m axolotl.cli.train examples/mistral/config.yml

Config yaml

No response

Possible solution

Perhaps use a different default dataset that allows for eval_sample_packing to be True.

Which Operating Systems are you using?

• Linux
• macOS
• Windows

Python Version

3.11.7

axolotl branch-commit

main

Acknowledgements

• My issue title is concise, descriptive, and in title casing.
• I have searched the existing issues to make sure this bug has not been reported yet.
• I am using the latest version of axolotl.
• I have provided enough information for the maintainers to reproduce and diagnose the issue.

[NanoCode012]

Oh, would you be interested in making a PR to set that config?

[NanoCode012]

Thank you for the PR! Closed by #1003