Phi2 rewrite

[winglian]

• start over with latest phi2 modeling code from hf repo
• Gradient checkpointing for ParallelBlock by @abacaj via https://fxtwitter.com/abacaj/status/1743749460872634506
• update gradient checkpointing signature to match transformers
• Properly enable flash attention, thanks @vikhyat via https://fxtwitter.com/vikhyatk/status/1743876177037861291
• disable most of the upcasting to float32 in favor of bfloat16

[casper-hansen]

Looks good to me. I would remove the code you commented out. You can come back to the PR if you need to look up what changed.

Also, what are the possibilities of using sample packing with Phi2?

[winglian]

@casper-hansen will clean up the commented out code

As far as sample packing, it should be pretty straightforward. I started working on a fix for the previous implementation #877 but I may simply start over.

You had mentioned last year figuring out a way to manage sample packing across all the architectures in a more manageable way. I'm happy to take a stab at it if you have a prrof of concept or anything.

[casper-hansen]

You had mentioned last year figuring out a way to manage sample packing across all the architectures in a more manageable way. I'm happy to take a stab at it if you have a prrof of concept or anything.

I had a branch going but didn't get to test and further implement it as I got busy with other stuff. The concept is to have one implementation that can be managed more easily managed in one module.

https://github.com/OpenAccess-AI-Collective/axolotl/tree/refactor-flash-attention

[NanoCode012]

I would recommend maybe consolidate the class?

data_collator = BatchSamplerDataCollatorForSeq2Seq if training_args.sample_packing else DataCollatorForSeq2Seq

[winglian]

my IDE doesn't like that 😭

[winglian]

alright, looks good on a single 4090 https://api.wandb.ai/links/oaaic/51qvcv4z

[fakerybakery]

Hi, do I need to change any configuration options or just use the default ones w/ Phi 2?