Upcoming in next release! (this week)

[Vaibhavs10]

• Speaker Diarization with Pyannote 🤯
• Fast CPU support 💻
• Streaming ⚡

[thomasmol]

Awesome! I have a pyannote implementation here https://github.com/thomasmol/replicate-whisper-diarization if you want to take a look.
Also really curious how streaming would work!

[omarsiddiqi224]

There is this resource as well to help with diarization: https://github.com/MahmoudAshraf97/whisper-diarization

[zeke]

Looking forward to diarization 🙏🏼

[JuergenFleiss]

Hi, is testing CPU possible already? Would be greatly interested to compare it to faster-whisper. Also, what would the limitations be? Batching should work? Flash attention? Your speeds sound really promising.

[souvikqb]

• Speaker Diarization with Pyannote 🤯
• Fast CPU support 💻
• Streaming ⚡

Hey 👋 any update on these releases @Vaibhavs10 ? They would open up a lot of possibilities

[acul3]

Looking forward to this

Btw @Vaibhavs10 have you consider adding vad(voice activity detection) to the pipeline

By my testing..VAD reduce hallucination espscially with audio lot of silence and noise

Thanks

[BBC-Esq]

Might I suggest using nemo toolkit instead? It seems to avoid pyannote's requirement of using a huggingface key or what not to access their model. omarsiddiqi224 is the one who posted a link to a repository that relies on it instead of pyannote.

[bluusun]

How can the speaker diarization be used? Where does it show? Thanks for adding this!

[TomExMachina]

Does anyone have a streaming script or snippet they can share ahead of the release? If you do I will help iterate on it.

[souvikqb]

Does anyone have a streaming script or snippet they can share ahead of the release? If you do I will help iterate on it.

I had originally asked this question on Distill Whisper, here's a potential script - huggingface/distil-whisper#4 (comment)

Link to my issue - huggingface/distil-whisper#41 (comment)

[Vaibhavs10]

Heu @souvikqb @TomExMachina - Re: Streaming: A community member made this: https://gist.github.com/Oceanswave/32da596e8bb10c928f6c69c889c3c130 (It works quite well)

[Vaibhavs10]

Hey @bluusun - Currently, the API is a bit spaghetti, however, if you pass a parameter --hf_token <HF token> it should automatically diarise.

@kadirnar recently made a PR to make this more clear #83 (we'll make a release tomorrow or on saturday along with some more goodies 🤞 )

[Vaibhavs10]

@BBC-Esq - I'm opening a new issue to discuss this #85, I think adding support for Nvidia NeMo might make sense and give people the option to choose different backends too.

[Vaibhavs10]

(Closing this issue since the release already happened; we need another patch (to fix the current API) before considering the next steps.)

[Tortoise17]

@Vaibhavs10 CPU usage is now possible with the new release?

[Tortoise17]

@Vaibhavs10 Fast CPU support was also mentioned to make available in this release in addition to diarization and streaming

[Vaibhavs10]

Let's discuss that in a seperate issue. (I'll open one)