-
Notifications
You must be signed in to change notification settings - Fork 153
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Writing a backend for PyTorch in Kompute #322
Comments
This sounds interesting @han-minhee ,there has been previous implementations of Kompute as backend for ml frameworks such as jax, you can have a look at that to get an idea as well. Writing a backend for PyTorch would be a large undertaking but feel free to share here any questions (or design docs) and we'll be happy to support and point you in the right direction. |
@axsaucedo Thank you for the reply! Did you mean vkJAX? This would be a good reference. In addition, do you think I would need to add much codes to Kompute itself, other than using the functions already implemented in Kompute? I currently don't have any id of how much operators needed for the porting are available in the current Kompute. It would be my pleasure if I can add some more codes to the Kompute itself. |
Reopening the issue as I accidentally closed it |
Yes that's the project I was referencing, there may be others that could be relevant to identify
I can't think of anything specific, but if we come across something then it could certainly be explored
Operators are extensible so that shouldn't be the main issue, but any limitations of the framework can be explored
Great that can be explored |
I see! I'll start working on it, though I don't know how much it will take 🙃 |
hmm, I was thinking about turning Metal Performance Shaders (MPS) into Vulkan via Kompute, then it could function as plugin like pytorch-directml. Nomic-backend for ggml is interesting reference as well. |
@sorasoras that's a super interesting proposal - I would be quite interested to support this as it could indeed provide quite a flexible backend. I believe there's not been much movement here, so it could be a great one to explore especially as now the Nomic-backend for ggml is being integrated and can be used as reference. We'll also be looking at extending the examples sections of the Kompute docs as well as potentially adding tests to ensure we're testing upstream integrations to flag any deltas with changes. |
I'm planning to implement a PyTorch backend in Kompute, first aiming at a few basic operators and moving onto the full set.
Is there any project like this, or what are your thoughts?
I have some experience in porting PyTorch/Tensorflow models to C++ CUDA/OpenCL programs, but don't have any experience with modifying ATen/ or PyTorch code itself. I also have to admit that I've just started trying Vulkan / Kompute, but I already feel attached to it :)
The text was updated successfully, but these errors were encountered: