More tests

[IlyasMoutawwakil]

Most of these features are covered in examples but not in the testing suite.

• Quantization schemes:
  • Onnxruntime @aliabdelkader
  • GPTQ @lopozz
  • AWQ @aliabdelkader
  • BnB @aliabdelkader
• Optimizations:
  • FAv2
  • torch.compile. @aliabdelkader

[aliabdelkader]

I am happy to take this issue if no one is working on it. It is my first issue, I might need some help.
It looks to me that we need to just add configuration files to make the current tests cover the mentioned use cases. Is that correct or did I miss something ?

[IlyasMoutawwakil]

@aliabdelkader yess I left these as first issues for anyone interested in contributing and having these settings tested as part of the CI. You just need to add a configuration and make sure that it's tested in the CI (might require adding some packages).

[aliabdelkader]

#take
ok great, I am working on it.

[lopozz]

Hi @aliabdelkader and @IlyasMoutawwakil I benchmarked some GPTQ models in the past week and I'd be up to write a test for this quantization scheme if you're not already on it. Let me know

[IlyasMoutawwakil]

thanks ! would love to review the quantization configs ! make sure u include backend.no_weights: true,false when possible to ur sweep configs as the no weights feature is probably one of the most important features of optimum-benchmark and making sure it's always compatible with quantization schemes seems interesting

[aliabdelkader]

@lopozz no i did not work on that scheme yet. So, I would not mind if you want to write the test for it.
@IlyasMoutawwakil sure, I will include the backend.no_weights: true,false sweep to the quantizatio test configs.

[lopozz]

Hi @IlyasMoutawwakil I was trying the same configuration file of #125 and I got this error

AttributeError: 'LlamaRotaryEmbedding' object has no attribute 'cos_cached'

I believe it is caused by the newest refactoring. The last commit I see is 97e0b9e of 4 days ago, so I believe I am working with the latest version. Should I open a new issue for this?

[IlyasMoutawwakil]

@lopozz I think it's a transformers issue unslothai/unsloth#168 (comment), try downgrading ur transformers version

[aliabdelkader]

Hi,

I just want to mention that I wont be able to work on the point related to FAv2 in case someone else wants to pick it up.