Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build and implement new reward stack #126

Open
schampoux opened this issue May 30, 2024 · 3 comments
Open

Build and implement new reward stack #126

schampoux opened this issue May 30, 2024 · 3 comments

Comments

@schampoux
Copy link
Contributor

Currently we use a one-hot rewarding system for miners. The goal of this issue is to build, implement, and test new rewarding systems that are more dependent on the distribution of energies.

  • Sorting energies in ascending order (most negative to 0). Assign rewards linearly such that the most negative energy gets a reward of 1 and energy 0 gets a reward of 0.
  • Scale the rewards using min-max normalization, excluding the zero energy value. This can be done by removing 0 energy values, determine the min and max energies from the filtered set, and apply min-max normalization to scale the rewards between 0 and 1.
  • Rewards can be scaled exponentially to emphasize lower energy values more strongly.
  • Logarithmic scaling can be used to reduce the impact of high energy values and provide a smoother reward distribution.
  • Softmax scaling to convert the energies into a probability distribution, which can be interpreted as scaled rewards.
  • others.
@schampoux
Copy link
Contributor Author

schampoux commented Jun 3, 2024

image

This plot was constructed using the reward distribution from all miners and jobs. Brian had mentioned steering the initiative toward scaling each simulations output to formulate rewards. This is to be done on a batch level per simulation.

@schampoux
Copy link
Contributor Author

schampoux commented Jun 3, 2024

@steffencruz
Copy link
Contributor

@mccrindlebrian can we close?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants