BetterTransformer support for VisionEncoderDecoder models like TrOCR

[donjuanpond]

Feature request

I would like for there to be support for Vision Encoder Decoder models, such as TrOCR, with the BetterTransformer tool.

Motivation

I have a TrOCR model I want to speed up inference for (reducing latency). I've tried every quantization option I could, and none of them are speeding up my inference. If BetterTransformers could support the VisionEncoderDecoder model type, that would make my life a whole lot easier!!

Your contribution

I'm not very knowledgeable about open source contribution, especially on such a big project with so many complicated parts, so I don't know if I could really help with this.