From 0716b845fdef8a20102b07eaec074b8da8162523 Mon Sep 17 00:00:00 2001
From: Viren6 <94880762+Viren6@users.noreply.github.com>
Date: Tue, 2 Apr 2024 04:38:54 +0100
Subject: [PATCH] Update NNUE architecture to SFNNv9 and net
 nn-ae6a388e4a1a.nnue
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Part 1: PyTorch Training, linrock

Trained with a 10-stage sequence from scratch, starting in May 2023:
https://github.com/linrock/nnue-tools/blob/master/exp-sequences/3072-10stage-SFNNv9.yml

While the training methods were similar to the L1-2560 training sequence,
the last two stages introduced min-v2 binpacks,
where bestmove capture and in-check position scores were not zeroed during minimization,
for compatibility with skipping SEE >= 0 positions and future research.

Training data can be found at:
https://robotmoon.com/nnue-training-data

This net was tested at epoch 679 of the 10th training stage:
https://tests.stockfishchess.org/tests/view/65f32e460ec64f0526c48dbc

Part 2: SPSA Training, Viren6

The net was then SPSA tuned.
This consisted of the output weights (32 * 8) and biases (8)
as well as the L3 biases (32 * 8) and L2 biases (16 * 8), totalling 648 params in total.

The SPSA tune can be found here:
https://tests.stockfishchess.org/tests/view/65fc33ba0ec64f0526c512e3

With the help of Disservin , the initial weights were extracted with:
https://github.com/Viren6/Stockfish/tree/new228

The net was saved with the tuned weights using:
https://github.com/Viren6/Stockfish/tree/new241

Earlier nets of the SPSA failed STC compared to the base 3072 net of part 1:
https://tests.stockfishchess.org/tests/view/65ff356e0ec64f0526c53c98
Therefore it is suspected that the SPSA at VVLTC has
added extra scaling on top of the scaling of increasing the L1 size.

Passed VVLTC 1:
https://tests.stockfishchess.org/tests/view/6604a9020ec64f0526c583da
LLR: 2.94 (-2.94,2.94) <0.00,2.00>
Total: 53042 W: 13554 L: 13256 D: 26232
Ptnml(0-2): 12, 5147, 15903, 5449, 10

Passed VVLTC 2:
https://tests.stockfishchess.org/tests/view/660ad1b60ec64f0526c5dd23
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 17506 W: 4574 L: 4315 D: 8617
Ptnml(0-2): 1, 1567, 5362, 1818, 5

STC Elo estimate:
https://tests.stockfishchess.org/tests/view/660b834d01aaec5069f87cb0
Elo: -7.66 ± 3.8 (95%) LOS: 0.0%
Total: 9618 W: 2440 L: 2652 D: 4526
Ptnml(0-2): 80, 1281, 2261, 1145, 42
nElo: -13.94 ± 6.9 (95%) PairsRatio: 0.87

closes https://tests.stockfishchess.org/tests/view/660b834d01aaec5069f87cb0

bench 1823302

Co-Authored-By: Linmiao Xu <lin@robotmoon.com>
---
 src/evaluate.h               | 2 +-
 src/nnue/nnue_architecture.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/evaluate.h b/src/evaluate.h
index 5f2b2c5422a..97ca4c4b665 100644
--- a/src/evaluate.h
+++ b/src/evaluate.h
@@ -35,7 +35,7 @@ constexpr inline int SmallNetThreshold = 1165, PsqtOnlyThreshold = 2500;
 // for the build process (profile-build and fishtest) to work. Do not change the
 // name of the macro or the location where this macro is defined, as it is used
 // in the Makefile/Fishtest.
-#define EvalFileDefaultNameBig "nn-1ceb1ade0001.nnue"
+#define EvalFileDefaultNameBig "nn-ae6a388e4a1a.nnue"
 #define EvalFileDefaultNameSmall "nn-baff1ede1f90.nnue"
 
 namespace NNUE {
diff --git a/src/nnue/nnue_architecture.h b/src/nnue/nnue_architecture.h
index 05efb813754..7f73f87fd5e 100644
--- a/src/nnue/nnue_architecture.h
+++ b/src/nnue/nnue_architecture.h
@@ -38,7 +38,7 @@ namespace Stockfish::Eval::NNUE {
 using FeatureSet = Features::HalfKAv2_hm;
 
 // Number of input feature dimensions after conversion
-constexpr IndexType TransformedFeatureDimensionsBig = 2560;
+constexpr IndexType TransformedFeatureDimensionsBig = 3072;
 constexpr int       L2Big                           = 15;
 constexpr int       L3Big                           = 32;