-
|
Apologies to be the only one asking questions, but we love the project and think it will really help us in evaluating different LLMs for our use cases. We moved out of Colab to make things easier to debug. We are now trying to install on our own hardware - server w/ 3070 desktop card and 64 GB of RAM. We are now able to launch the Chat UI without a problem.
However, we run into the following RuntimeError from there.
Any thoughts? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Similar issues can be found on StackOverflow, I think your model is loaded on CPU instead of your GPU. https://stackoverflow.com/questions/73530569/pytorch-matmul-runtimeerror-addmm-impl-cpu-not-implemented-for-half |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, will take a look there. |
Beta Was this translation helpful? Give feedback.
Similar issues can be found on StackOverflow, I think your model is loaded on CPU instead of your GPU. https://stackoverflow.com/questions/73530569/pytorch-matmul-runtimeerror-addmm-impl-cpu-not-implemented-for-half