Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Determine the thread pool size based on the physical CPU #191

Closed
marktwtn opened this issue Sep 9, 2019 · 1 comment · Fixed by #195
Closed

Determine the thread pool size based on the physical CPU #191

marktwtn opened this issue Sep 9, 2019 · 1 comment · Fixed by #195
Assignees
@marktwtn

Comments

@marktwtn
Copy link
Collaborator

marktwtn commented Sep 9, 2019

The dcurl before and after integrating the thread pool uses the (maximum logical CPU - 1) as the number of threads for a single PoW.
However, dcurl also allows calculating 2 PoW at the same time, which is the number of threads per core.
And the number of logical CPU is equal to the number of physical CPU * threads per core.

Before integrating the thread pool, dcurl may create the threads as many as the twice number of the logical CPU.
After integrating the thread pool, dcurl set the pool size as (maximum logical CPU - 1).
And it causes the PoW to wait for the threads returning from the previous PoW.

Goal:

  • First of all, the thread pool size must be changed to allow 2 PoW being calculated at the same time.
  • Second, the thread number should be modified since using (maximum logical CPU - 1) number of threads in a single PoW is contradicts to calculating (threads per core) number of PoW at the same time.
@marktwtn marktwtn self-assigned this Sep 9, 2019
@marktwtn
Copy link
Collaborator Author

I found out that it is hard to get the physical CPU number and hyperthreading data without parsing CPU information by the programmer.
get_nprocs_conf() only gives us the logical CPU number, and the related function sysconf() can not give us the physical CPU number, either.

It would be a lot of easier if we just modify dcurl to do 1 PoW at the same time.
However, I would like to try the harder way to fit the original dcurl design.

marktwtn added a commit to marktwtn/dcurl that referenced this issue Sep 19, 2019
@marktwtn
feat: Allocate thread resource based on the physical processor
ed84a19 
The original design has contradiction.
It uses almost all the logical processors for a single PoW,
but allows dcurl to handle 2 PoW at the same time based on the
hyper-threading.

The new function is added in cpu-utils.h to detect the thread number per
physical processor, which also shows whether the hyper-threading is
enabled or not.
Once we know the status of hyper-threading, we can determine the
threadpool size and the thread number for a single PoW.

Close DLTcollab#191.
@marktwtn marktwtn mentioned this issue Sep 19, 2019
Merged
marktwtn added a commit to marktwtn/dcurl that referenced this issue Sep 23, 2019
@marktwtn
feat: Allocate thread resource based on the physical processor
9745f90 
The original design has contradiction.
It uses almost all the logical processors for a single PoW,
but allows dcurl to handle 2 PoW at the same time based on the
hyper-threading.

The new function is added in cpu-utils.h to detect the thread number per
physical processor, which also shows whether the hyper-threading is
enabled or not.
Once we know the status of hyper-threading, we can determine the
threadpool size and the thread number for a single PoW.

Close DLTcollab#191.
marktwtn added a commit to marktwtn/dcurl that referenced this issue Sep 23, 2019
@marktwtn
feat: Allocate thread resource based on the physical processor
cf0db79 
The original design has contradiction.
It uses almost all the logical processors for a single PoW,
but allows dcurl to handle 2 PoW at the same time based on the
hyper-threading.

The new function is added in cpu-utils.h to detect the thread number per
physical processor, which also shows whether the hyper-threading is
enabled or not.
Once we know the status of hyper-threading, we can determine the
threadpool size and the thread number for a single PoW.

Close DLTcollab#191.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@marktwtn marktwtn

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Allocate thread resource based on the physical processor marktwtn/dcurl
1 participant
@marktwtn