Skip to content

Commit

Permalink
Update NNUE architecture to SFNNv9 and net nn-ae6a388e4a1a.nnue
Browse files Browse the repository at this point in the history
Part 1: PyTorch Training, linrock

Trained with a 10-stage sequence from scratch, starting in May 2023:
https://github.com/linrock/nnue-tools/blob/master/exp-sequences/3072-10stage-SFNNv9.yml

While the training methods were similar to the L1-2560 training sequence,
the last two stages introduced min-v2 binpacks,
where bestmove capture and in-check position scores were not zeroed during minimization,
for compatibility with skipping SEE >= 0 positions and future research.

Training data can be found at:
https://robotmoon.com/nnue-training-data

This net was tested at epoch 679 of the 10th training stage:
https://tests.stockfishchess.org/tests/view/65f32e460ec64f0526c48dbc

Part 2: SPSA Training, Viren6

The net was then SPSA tuned.
This consisted of the output weights (32 * 8) and biases (8)
as well as the L3 biases (32 * 8) and L2 biases (16 * 8), totalling 648 params in total.

The SPSA tune can be found here:
https://tests.stockfishchess.org/tests/view/65fc33ba0ec64f0526c512e3

With the help of Disservin , the initial weights were extracted with:
https://github.com/Viren6/Stockfish/tree/new228

The net was saved with the tuned weights using:
https://github.com/Viren6/Stockfish/tree/new241

Earlier nets of the SPSA failed STC compared to the base 3072 net of part 1:
https://tests.stockfishchess.org/tests/view/65ff356e0ec64f0526c53c98
Therefore it is suspected that the SPSA at VVLTC has
added extra scaling on top of the scaling of increasing the L1 size.

Passed VVLTC 1:
https://tests.stockfishchess.org/tests/view/6604a9020ec64f0526c583da
LLR: 2.94 (-2.94,2.94) <0.00,2.00>
Total: 53042 W: 13554 L: 13256 D: 26232
Ptnml(0-2): 12, 5147, 15903, 5449, 10

Passed VVLTC 2:
https://tests.stockfishchess.org/tests/view/660ad1b60ec64f0526c5dd23
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 17506 W: 4574 L: 4315 D: 8617
Ptnml(0-2): 1, 1567, 5362, 1818, 5

STC Elo estimate:
https://tests.stockfishchess.org/tests/view/660b834d01aaec5069f87cb0
Elo: -7.66 ± 3.8 (95%) LOS: 0.0%
Total: 9618 W: 2440 L: 2652 D: 4526
Ptnml(0-2): 80, 1281, 2261, 1145, 42
nElo: -13.94 ± 6.9 (95%) PairsRatio: 0.87

closes https://tests.stockfishchess.org/tests/view/660b834d01aaec5069f87cb0

bench 1823302

Co-Authored-By: Linmiao Xu <lin@robotmoon.com>
  • Loading branch information
2 people authored and vondele committed Apr 2, 2024
1 parent c964942 commit 0716b84
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion src/evaluate.h
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ constexpr inline int SmallNetThreshold = 1165, PsqtOnlyThreshold = 2500;
// for the build process (profile-build and fishtest) to work. Do not change the
// name of the macro or the location where this macro is defined, as it is used
// in the Makefile/Fishtest.
#define EvalFileDefaultNameBig "nn-1ceb1ade0001.nnue"
#define EvalFileDefaultNameBig "nn-ae6a388e4a1a.nnue"
#define EvalFileDefaultNameSmall "nn-baff1ede1f90.nnue"

namespace NNUE {
Expand Down
2 changes: 1 addition & 1 deletion src/nnue/nnue_architecture.h
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ namespace Stockfish::Eval::NNUE {
using FeatureSet = Features::HalfKAv2_hm;

// Number of input feature dimensions after conversion
constexpr IndexType TransformedFeatureDimensionsBig = 2560;
constexpr IndexType TransformedFeatureDimensionsBig = 3072;
constexpr int L2Big = 15;
constexpr int L3Big = 32;

Expand Down

0 comments on commit 0716b84

Please sign in to comment.