If I have an AMD Processor with 128 cores, can DeepSparse fully utilizes all those cores? #1591
Unanswered
CracklyOdin
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello! I recently conducted testing on my WSL2 environment with DeepSparse using the neuralmagic/Nous-Hermes-2-SOLAR-10.7B-pruned50-quant-ds model. I performed this testing on two devices: one with an AMD Ryzen 5 7600 and another with an Intel® Core™ i9-13900KF.
For your reference, the AMD Ryzen 5 7600 has 6 cores, while the Intel® Core™ i9-13900KF has 24 cores (16 efficient cores (E-cores) and 8 high-performance cores (P-cores)).
The following screenshot illustrates that in the AMD Ryzen 5 7600 device, the response generation process is utilizing 598% of CPU usage (approximately 6 cores), fully leveraging all cores of the AMD Ryzen 5 7600 processor.
![image](https://private-user-images.githubusercontent.com/159231669/303256305-fcf9ae6d-e058-4fe4-84ea-bdd03caf89b8.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA4NDk1MzgsIm5iZiI6MTcyMDg0OTIzOCwicGF0aCI6Ii8xNTkyMzE2NjkvMzAzMjU2MzA1LWZjZjlhZTZkLWUwNTgtNGZlNC04NGVhLWJkZDAzY2FmODliOC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzEzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxM1QwNTQwMzhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0zMTE2MGY1NGQwOTEwNGQxZmQ0MDhkYjE5ZGIwYzc3MzhjZDYwMThkYWUyYmY5NTMzOTU3Y2RmZjVjYWVjNGJlJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9._eNfrne_4H270NvTPXNHGgtuxZ5CrlnEVJQGTx23FoQ)
The following screenshot shows the testing result in the Intel® Core™ i9-13900KF device, the response generation process is utilizing 1591% of CPU usage (approximately 16 cores).
![image](https://private-user-images.githubusercontent.com/159231669/303258996-3ba05ddf-487f-4599-b77b-92ee3c0b9f6e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA4NDk1MzgsIm5iZiI6MTcyMDg0OTIzOCwicGF0aCI6Ii8xNTkyMzE2NjkvMzAzMjU4OTk2LTNiYTA1ZGRmLTQ4N2YtNDU5OS1iNzdiLTkyZWUzYzBiOWY2ZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzEzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxM1QwNTQwMzhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT05ZmZhZTcwNmY0MjI3YjU2YWJiNWYwMzhhYmYzNWQzM2ZiODVlZDcxODg5NGZjODU4MWNhNDkxOWE5NDNhYzAyJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.nKQ8HNZVm_Al2rsH6dKnfW4V4u533iv9Qzi4Ih8v4aE)
I've reviewed other discussions in the DeepSparse forum and discovered that DeepSparse doesn't support virtual threading provided by hyper-threading and only utilizes physical cores. Could this be the reason why the response generation process is only utilizing 16 E-cores and not the 8 P-cores, given that P-cores support hyperthreading?
Additionally, if I have an AMD processor with 128 cores, will DeepSparse fully utilize all of them?
Thanks in advanced.
Beta Was this translation helpful? Give feedback.
All reactions