Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate on a better gpu usage #72

Merged
merged 18 commits into from
Jun 1, 2023
Merged

Conversation

chainyo
Copy link
Contributor

@chainyo chainyo commented Jun 1, 2023

This PR proposes an improvement to transcription by implementing a batch request process.

  • Add README instructions for profiling the container
  • Fix Exception/Error returns through the API -> raised errors should be more transparent for the user
  • VAD use now onnx and faster-whisper implementation
  • Transcription is now batched. Current batch size of 32 (less than 13GB of VRAM) -> TODO: make it configurable based on the GPU space.
  • word_timestamps is disabled for now until it's fully implemented with the batched transcription process

@chainyo chainyo added api Everything related to the API implementation transcription Everything related to the transcription part labels Jun 1, 2023
@chainyo chainyo self-assigned this Jun 1, 2023
@chainyo chainyo linked an issue Jun 1, 2023 that may be closed by this pull request
@chainyo chainyo merged commit 12ec398 into main Jun 1, 2023
@chainyo chainyo deleted the 6-investigate-on-a-better-gpu-usage branch June 1, 2023 17:51
@chainyo chainyo mentioned this pull request Jun 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Everything related to the API implementation transcription Everything related to the transcription part
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate on a better GPU usage
2 participants