log eval dataset misconfiguration

[milocress]

Make Eval Dataset Misconfiguration Errors Visible through Mosaic Logger

Wraps eval dataset creation with a mosaic logger try/catch.

In Train context

mpt-125m-chinchilla-regression-6OSoWo (the log trace indicates that the error is caught and the context attribute is set)

[rank7]: │   623 │   │   if mosaicml_logger is not None:                                │
[rank7]: │   624 │   │   │   e.context = 'TrainContext'                                 │
[rank7]: │   625 │   │   │   mosaicml_logger.log_exception(e)                           │
[rank7]: │ ❱ 626 │   │   raise e 

In Eval context

mpt-125m-chinchilla-regression-6OSoWo (the log trace indicates that the error is caught and the context attribute is set)

[rank0]: │   656 │   │   │   if mosaicml_logger is not None:                            │
[rank0]: │   657 │   │   │   │   e.context = 'EvalContext'                              │
[rank0]: │   658 │   │   │   │   mosaicml_logger.log_exception(e)                       │
[rank0]: │ ❱ 659 │   │   │   raise e  

[dakinggg]

Can you add a run name for a manual test?

[jjanezhang]

Looking great, had a comment on renaming so we can use this for many things in the future.

Flagging that the dataset type won't be added in the convert_delta_to_json.py and convert_text_to_mds.py. I think we would have to parse the datapath to get the train split to wrap the error if you'd like to take a stab! Also happy to do it :) thanks so much Milo!

[milocress]

Flagging that the dataset type won't be added in the convert_delta_to_json.py and convert_text_to_mds.py. I think we would have to parse the datapath to get the train split to wrap the error if you'd like to take a stab! Also happy to do it :) thanks so much Milo!

I am happy to do this, but let's split it into a separate PR so this one stays small.

[jjanezhang]

small nit, otherwise lgtm