Bt benchmarks

[baskrahmer]

What does this PR do?

Refactors the benchmark scripts; add CPU support.
Next step is to make a script to run these benchmarks on all BetterTransformer architectures and generate the speedup tables :)

Still need to test on GPU

#1125 (review)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[regisss]

@IlyasMoutawwakil Do we already support BetterTransformer in Optimum Benchmark?

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[IlyasMoutawwakil]

@regisss Yep we do,

[baskrahmer]

@regisss @IlyasMoutawwakil ah I did not know of this repo but it seems useful! That leaves me questioning the purpose of this PR and generally the need for benchmark scripts in the Optimum library.

[regisss]

@regisss @IlyasMoutawwakil ah I did not know of this repo but it seems useful! That leaves me questioning the purpose of this PR and generally the need for benchmark scripts in the Optimum library.

Not sure exactly if these scripts are still maintained. Maybe @fxmarty knows more about this?

[fxmarty]

Yes, these scripts are there for reproducibility for the blog posts we published on pytorch medium/blog.

Probably we should use @IlyasMoutawwakil work to generate the benchmark tables for BT.