Fix Deepspeed loading

[winglian]

deepspeed wasn't loading properly to shard the model weights when loading them.

This PR fixes

• printing ascii art after loading the configuration and setting env vars so deepspeed gets loaded properly by accelerate
• adds an option to freeze/unfreeze certain parameters. This can allow for lower memory requirements when fine-tuning mixtral

example that freezes everything except layers 20-31

unfrozen_parameters:
  - lm_head.*
  - model.embed_tokens.*
  - model.layers.2[0-9].*
  - model.layers.29.*
  - model.layers.30.*
  - model.layers.31.*

[hamelsmu]

Why do you have to delete this key?

[winglian]

deepspeed maps the weights on its own and doesn't want device_map set

https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_utils.py#L2858-L2861

[hamelsmu]

@winglian this example is good but perhaps can you add this new unfrozen_parameters to the top level README ?

[hamelsmu]

I would also add a comment about how the regex works in the docs so people know?

[hamelsmu]

@winglian Really excited that you got this in so fast! Just have a suggestion about docs. I'm happy to add it too for you in a follow on PR

[winglian]

@winglian Really excited that you got this in so fast! Just have a suggestion about docs. I'm happy to add it too for you in a follow on PR

thanks for the review. I'll get this added to the docs later tonight or tomorrow.