https://github.com/bnosac/golgotha

[jwijffels]

Just a note.
I've created this: https://github.com/bnosac/golgotha in order to easily use the BERT embeddings in some downstream predictive models and when I tried RBERT I couldn't get the multilingual model to work.
It was also a trial to see on speed of getting these embeddings and to see what these model outputs provide and how I could maybe develop this directly using libtorch bypassing python.

[jonathanbratt]

Thanks for the note!
I don't think we've tested the multilingual models specifically in RBERT.
What's the nature of the failure you encountered?
Were you able to get any of the English models to work (at least formally, even if it's not suited for the task)?

[jwijffels]

yes, english worked fine, can't remember the exact error on the multilingual model, i believe it was already at download_BERT_checkpoint

[jwijffels]

is there any interest from one of you 2 to work directly from libtorch instead of using python?

[jonathanbratt]

Possibly! We're working right now on using tensorflow hub (via tfhub package) to load the models, but are open to additional/other ways of proceeding. I haven't had much time lately to devote to this, but hope to soon.

[jonthegeek]

Update: Almost definitely! We're finally actively working on this again, and the path will definitely be along the lines of "get the TF2 version working, then investigate {torch}."

[jwijffels]

+1 for some torch developments regarding NLP. I was thinking about some named entity recognition models and started experimenting a bit myself at https://github.com/DIGI-VUB/torch.ner but certainly having BERT embeddings through torch would be a big plus

[jonthegeek]

It's likely that we'll make a separate repo for torch development (although that isn't 100% decided yet). I'll definitely ping you when we figure it out!