-
Notifications
You must be signed in to change notification settings - Fork 157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize for CPU inference #19
Comments
Does Intel CPU have Tensorflow already uses all CPU cores by default, and I'm not sure if there's practically any performance to gain unless we compromise the prediction frequency (using time step > 10 ms) or using a smaller model (like the one I used in the web demo) |
It's not precisely optimize but you might be interested in some of these numbers reported here |
We should add this table to the README file in the section where we explain about the different models available, ideally also including performance drop with respect to the full model (i.e. use a track from MDB for which we have a reference annotation and evaluate each model not only in terms of time, but also accuracy). e.g. something similar to table 1 here: https://www.tensorflow.org/performance/quantization |
Yeah that's why I haven't put the table on README yet! |
Inference on CPU is very slow right now (often too slow for practical application).
I think TensorFlow already uses as many cpu cores as it has access to when running in cpu mode (?), so I'm not sure whether e.g. splitting the audio track and parallelizing inference via e.g. multiprocessing or jobilb would make any difference.
But, it might be worth checking out TF guide on performance, such as the performance guide or the info on model quantization.
The text was updated successfully, but these errors were encountered: