swap the data collator for evals if not using sample packing

[winglian]

when sample packing is enabled, but eval sample packing is disabled, we need to handle the data collator separately.

[casper-hansen]

Does this resolve the error with using sample packing for training but not for evaluation? If so, could it be useful to add a test specifically for this scenario?

[winglian]

Does this resolve the error with using sample packing for training but not for evaluation? If so, could it be useful to add a test specifically for this scenario?

what error are you referring to? This should resolve an NCCL issue when there is a mismatch on multi-gpu

[casper-hansen]

Does this resolve the error with using sample packing for training but not for evaluation? If so, could it be useful to add a test specifically for this scenario?

what error are you referring to? This should resolve an NCCL issue when there is a mismatch on multi-gpu

I remember there was an error when the validation size was too small. Then I swapped to using eval without sample packing and there was some other issue where it would not work.

[winglian]

I'll come back and add e2e tests for this once we have multi-gpu runners for CI.