Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Make random profile generation reproducible #295

Closed
tturocy opened this issue Jan 19, 2022 · 1 comment
Closed

ENH: Make random profile generation reproducible #295

tturocy opened this issue Jan 19, 2022 · 1 comment
Assignees
Labels
c++ Items which involve writing in C++ python Items which involve coding in Python
Milestone

Comments

@tturocy
Copy link
Member

tturocy commented Jan 19, 2022

There are a few places in which random numbers are used:

  • In generating starting strategy profiles for some path-following algorithms (such as liap, simpdiv);
  • In generating perturbation vectors for GNM/IPA;
  • In generating random games (in gametracer only, although this is not currently visible to users).

Depending on the origin of the code, these use either rand() or drand48() where the latter is present.

Given the way that random numbers are used, the fact that e.g. rand() does not necessarily have nice properties does not matter very much; in applications we generate a handful of random numbers for the purposes above, and we are not doing anything like Monte Carlo simulations in which having good properties for the pseudo-RNG are relevant for correctness.

That said there are ways in which the code which involves random numbers can and should be improved. In particular, good practice would be that

  • Each place that uses random numbers should be instrumented with a random number source provided by the calling code;
  • The ability to specify seeds should be exposed in every algorithm that offers random re-start or random perturbation, to ensure that results are reproducible across runs.

This is a moderate-sized project but should be straightforward given the random number generation model available in C++11. In the meanwhile we have left calls to rand() or drand48() as they are, including the check in the configure script to use drand48() where the original code has it. It might be desirable to change the latter and use only rand() (or once firmly using C++11, the modern equivalent), because the Python build does not check for drand48() and currently always uses rand() - creating a small difference in how the same algorithm would operate when called in Python versus via the command-line solvers.

@tturocy tturocy added this to the gambit-16.3.0 milestone Sep 29, 2023
@tturocy tturocy modified the milestones: gambit-16.3.0, gambit-16.2.0 Nov 3, 2023
@tturocy tturocy self-assigned this Feb 9, 2024
@tturocy tturocy changed the title Modernise approach to random numbers ENH: Make random profile generation reproducible Feb 9, 2024
@tturocy
Copy link
Member Author

tturocy commented Feb 9, 2024

We now only use random number generation in one (class of) place, the creation of random MixedStrategyProfile and MixedBehaviorProfile objects, and this now only calls std::rand() (so we no longer have drand48/srand48 anywhere).

To finish this off, we should make these reproducible. The most straightforward way to accomplish this is:

(1) For C++, use std::uniform_int_distribution and std::uniform_real_distribution, and expose a seed parameter as a command-line option for tools that use it.

(2) For Python, use the numpy facilities for random-number generation, and optionally allow the calls to .randomize() to accept this as a parameter.

We should note here that creating these distributions at random is mostly a convenience for the user, as in all cases calling code is free to construct these by whatever method it might desire - and that in general it is probably desirable for calling code to take more control rather than worrying about whether a seed results in the same output on different systems at different points in history.

@tturocy tturocy added python Items which involve coding in Python c++ Items which involve writing in C++ and removed python Items which involve coding in Python labels Feb 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Items which involve writing in C++ python Items which involve coding in Python
Projects
None yet
Development

No branches or pull requests

1 participant