-
Notifications
You must be signed in to change notification settings - Fork 0
/
notes.log
665 lines (552 loc) · 37.7 KB
/
notes.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
06.02.2024
Parsing FEN strings determines ordering of piece lists in Position. Consequently it determines move generation order.
So probably this will cause engine to play color-flipped position much weaker/ stronger.
Should either do some nice move ordering or at least shuffle generated moves at random?
08.02.2024
Thinking about attack LUT and promotions. At first I thought that I should consider promotions in attack LUT
(e.g pawn should square on the other side the board be flagged as 'attacked by pawn on promotion rank'). But then I realized
that the table is indexed by square difference, not by actual squares. So one entry in the table can originate from arbitrary rank-
not necessarily from the promoting rank.
Looks like promoting and then looking up is the way to go. Also.. pretty natural in typical movegen algorithm (generate pseudolegal and then filter out illegal)
15.02.2024
Done with perft test suite. Machess completes the suite in ~46.6 seconds; magog completes the same test in ~27.0 s! Great!
22.03.2024
Done with iterative perft. The implementation is on the "iterativePerft" branch.
Perftsuite (recursive) done in ~27.4 s. Iterative version completed in ~27-29s.
Disappointing result - was expecting some speedup after reading https://www.talkchess.com/forum3/viewtopic.php?t=53408
26.03.2024
Done with basic static position evaluation. Includes material and mobility. I've decided to copy-paste-modify original move generation code rather
than make the old one more general. This way I can skip the need for slice with moves completely. As a bonus I discovered that I can skip
some superfluous legality tests:
-now I have one legality test for all 4 promotions
-no need to do legality test after castling
Another minor optimization: appending four promotions at once
28.03.2024
Movegen with checking for doublechecks: ~23,2s
05.04.2024
Movegen with specialized movegen in leaf nodes (doublechecks, captures, interpo)
23.04.2024
Couldn't get movegen any faster really. Version with pin checking is way slower. Would like to investigate later. All the intermediate work is left at
specialized_movegen branch.
24.04.2024
Testing search and evaluate/score functions. Found an interesting bug while calculating mobility score. When calculating mobility of enemy pieces (the ones that are not moving).
The piece tries to take its own pawn because the enPassant square is set. So simply making a null-move does not guarrantee calculating correct mobility of enemy pieces. Luckily
this caused panic and rather than falfifying mobility score. So clean crash rather than elusive bug. Probably Machess engine has the same bug that went undetected!
09.05.2024
About iterative deepening under time control.
Just finished one of the first games against engine. I added a timeout condition in the search loop. This way engine does not spend too much time searching exhaustively at
depth reached (which makes it loose on time).
However we cannot simply break out from the search loop in arbitrary moment because this will cause engine to miss obvious moves. Specifically we must not quit right after descending
on depth deeper. Here a relevant log from Arena with my comments added:
2024-05-06 19:56:02,164-->1:position startpos moves e2e4 c7c5 g1f3 d8c7 f1c4 e7e6 b1c3 d7d6 d2d3 c7d8 c1e3 g7g6 d1d2 b8c6 e3g5 // black to move, black queen at d8 is hanging
2024-05-06 19:56:02,164-->1:go wtime 239171 btime 236628 winc 0 binc 0
2024-05-06 19:56:02,165<--1:info score 305 pv d8g5 depth 1
2024-05-06 19:56:02,171<--1:info score -180 pv d8a5 c4e6 depth 2
2024-05-06 19:56:02,332<--1:info score 65 pv d8a5 b2b4 c5b4 depth 3
2024-05-06 19:56:08,700<--1:info score -190 pv f8e7 c4b5 e7g5 b5c6 depth 4 // completed search at depth 4 - found reasonable continuation
2024-05-06 19:56:10,052<--1:timeout! // timeout reached while searching at depth 5 - starting to return from recursive calls
2024-05-06 19:56:10,052<--1:timeout!
2024-05-06 19:56:10,053<--1:timeout!
2024-05-06 19:56:10,053<--1:timeout!
2024-05-06 19:56:10,053<--1:timeout!
2024-05-06 19:56:10,053<--1:info score -10 pv f8g7 c3b5 g7c3 d2c3 d8g5 depth 5 // what is returned is a search partially completed searched at depth 5 - it didn't even see that queen is hanging,
2024-05-06 19:56:10,055<--1:info pv f8g7 c3b5 g7c3 d2c3 d8g5
2024-05-06 19:56:10,055<--1:bestmove f8g7 // engine hangs a queen
09.05.2024 - 2
Got it working but still got the following situation where it hangs the queen. But this blunder can be attributed to lack of quiescence search
(obviously it thinks it captured a piece at the end of the line). Log below:
position startpos moves e2e4 e7e6 f1c4 d8f6 b1c3 f6d4 g1f3 d4c4 d2d3
go wtime 332611 btime 261957 winc 0 binc 0
info pv b8a6 score 315
info pv b8c6 score 335
info pv c4c3 score 780
info score 780 pv c4c3 depth 1
info pv c4c3 b2c3 score -320
info pv c4c5 c1g5 score 280
info pv c4b4 c1g5 score 305
info score 305 pv c4b4 c1g5 depth 2
info pv c4b4 e1e2 b4c3 score 645
info score 645 pv c4b4 e1e2 b4c3 depth 3
info pv c4b4 f3g5 b8c6 g5f7 score 230
info score 230 pv c4b4 f3g5 b8c6 g5f7 depth 4
info pv c4b4 a2a3 b4c5 c3e2 c5f2 score 585 // at depth 5 started of with line similar from depth 4
info pv f8d6 f3g5 c4b4 c1d2 b4c3 score 655 // but falsely found "sth even better" - end of this sequence hangs a queen
info score 655 pv f8d6 f3g5 c4b4 c1d2 b4c3 depth 5
info pv f8d6 f3g5 c4b4 c1d2 b4c3 score 655
bestmove f8d6
17.05.2024
Completed quiescence search. Dode a 10 round tournament against Machess (5 openings as both black and white). Results are kind of disappointing :).
It lost all 10 out of 10 rounds! :)
Looks like it is simply slower. In scandinavian defense machess took 6 seconds to find the e4e5.
Magog took 6s for the same move.
Time to investigate - probably it is due to bad move ordering machess searches captures first
(and also has separate piece list per piece type)
but it could also be a problem with small NPS.. (simply slow raw movegen and evaluation function)
22.05.2024
Yup the engine has 200k less nps than Machess (~500k vs ~300k). Did some profiling with go pprof and found out that
isUnderCheckMethod appears quite high on the list.
To be more specific switch in that method is quite hot.
Here's a pprof list (note that it's not exactly from the commited version - this is a color-blind of the switch cases - see lines: 296, 297, 302.. anyway the results are on par)
(pprof) list isUnderCheck
Total: 28.55s
ROUTINE ======================== macsmol/magog/engine.(*Position).isUnderCheck in C:\Users\Maciej\workspaces\go\magog\engine\position.go
3.47s 3.73s (flat, cum) 13.06% of Total
60ms 60ms 292:func (pos *Position) isUnderCheck(enemyPieces []square, enemyKing square, destSquare square) bool {
. . 293: var moveIdx int16
300ms 300ms 294: for _, attackFrom := range enemyPieces {
60ms 260ms 295: moveIdx = moveIndex(attackFrom, destSquare)
1.17s 1.17s 296: switch pos.board[attackFrom]&ColorlessPiece {
. . 297: case Knight:
170ms 170ms 298: if attackTable[moveIdx]&KnightAttacks == 0 {
. . 299: continue
. . 300: }
. . 301: return true
So I thought that maybe the switch in Golang is kindof slow (a premonition I have ever since I
debugged a switch statement in Golang - for some reason you never jump directly to the desired case, you have to step over few times. This never happened in Java).
Following that lead I've made a version that uses a LUT with isCheckedByXX function containing
functions for each piece. This worked much slower (~200k nps).
Here's the relevant listing:
(pprof) list Check
Total: 36.26s
ROUTINE ======================== macsmol/magog/engine.(*Position).isUnderCheck in C:\Users\Maciej\workspaces\go\magog\engine\position.go
3.63s 7.63s (flat, cum) 21.04% of Total
140ms 140ms 332:func (pos *Position) isUnderCheck(enemyPieces []square, enemyKing square, destSquare square) bool {
. . 333: var moveIdx int16
760ms 760ms 334: for _, attackFrom := range enemyPieces {
10ms 2.24s 335: moveIdx = moveIndex(attackFrom, destSquare)
480ms 480ms 336: lutIdx := pos.board[attackFrom] & ColorlessPiece - 1
2.18s 3.92s 337: if checkFunctionsLUT[lutIdx](attackFrom, destSquare, moveIdx, &pos.board) {
. . 338: return true
. . 339: }
. . 340: }
. . 341:
. 30ms 342: moveIdx = moveIndex(enemyKing, destSquare)
40ms 40ms 343: kingAttack := attackTable[moveIdx]&KingAttacks != 0
20ms 20ms 344: return kingAttack
. . 345:}
. . 346:
. . 347:func checkedBySlidingPiece(slidingPieceSquare, destSquare square, moveIndex int16, board *[128]piece) bool {
25.05.2024
Search on depth 7 never finishes! Wait. It was for search from start pos. perft(7, startpos) has 3*10^9 nodes. So maybe not a bug.
anyway
here's log:
go depth 7
info pv e2e3 d7d5 f1b5 c7c6 score cp 25 depth 4 nodes 278733 time 682 nps 408525
info nodes 500000 time 1242 nps 402389
info pv e2e3 e7e5 d1g4 h7h5 g4e4 score cp 60 depth 5 nodes 751466 time 1921 nps 391035
info nodes 1000000 time 2449 nps 408190
info nodes 1500000 time 3512 nps 427077
info nodes 2000000 time 4585 nps 436121
...
info nodes 5500000 time 12584 nps 437040
info pv e2e3 e7e5 d1g4 h7h5 g4e4 score cp 60 depth 5 nodes 5515234 time 12617 nps 437105
info nodes 6000000 time 14202 nps 422472
....
sinfo nodes 14000000 time 37463 nps 373698
top
info pv e2e3 e7e6 d1f3 g8h6 h2h4 f8b4 score cp 5 depth 6 nodes 14154176 time 37965 nps 372812
info pv e2e3 e7e6 d1f3 g8h6 h2h4 f8b4 score cp 5 depth 6 nodes 14154176 time 37965 nps 372812
bestmove e2e3
go depth 7
info pv e2e3 d7d5 f1b5 c7c6 score cp 25 depth 4 nodes 278731 time 679 nps 410017
info nodes 500000 time 1241 nps 402897
info pv e2e3 e7e5 d1g4 h7h5 g4e4 score cp 60 depth 5 nodes 751464 time 1900 nps 395341
info nodes 1000000 time 2423 nps 412608
info nodes 1500000 time 3458 nps 433652
info nodes 2000000 time 4519 nps 442488
info nodes 2500000 time 5622 nps 444664
info nodes 3000000 time 6849 nps 437956
info nodes 3500000 time 7917 nps 442047
info nodes 4000000 time 9040 nps 442470
info nodes 4500000 time 10297 nps 437014
info nodes 5000000 time 11450 nps 436643
info nodes 5500000 time 12513 nps 439512
info pv e2e3 e7e5 d1g4 h7h5 g4e4 score cp 60 depth 5 nodes 5515232 time 12549 nps 439467
info nodes 6000000 time 14121 nps 424869
info nodes 6500000 time 15950 nps 407519
info nodes 7000000 time 17591 nps 397926
[...]
info nodes 19500000 time 52613 nps 370623
info pv e2e3 e7e5 d1g4 h7h5 g4f5 d8f6 score cp 20 depth 6 nodes 19769425 time 53366 nps 370448
info nodes 20000000 time 54043 nps 370069
[...]
info nodes 214000000 time 585191 nps 365692
info pv e2e3 e7e5 d1g4 h7h5 g4f5 d8f6 score cp 20 depth 6 nodes 214337518 time 586065 nps 365722
info nodes 214500000 time 586629 nps 365648
[...]
info nodes 681000000 time 1943027 nps 350483
info pv e2e3 e7e5 d1h5 d8f6 b1c3 f8b4 f1b5 score cp
40 depth 7 nodes 681371532 time 1943941 nps 350510
info nodes 681500000 time 1944231 nps 350524
[...]
info nodes 846000000 time 2353820 nps 359415
info nodes 846500000 time 2355236 nps 359411
stopinfo nodes 847000000 time 2356474 nps 359435
info pv e2e3 e7e5 d1h5 d8f6 b1c3 f8b4 f1b5 score cp 40 depth 7 nodes 847008416 time 2356494 nps 359435
bestmove e2e3
Anyway started new search before leaving house.
perft(7,startpos can get to 3,1*10^9 nodes) so let's see if my search gets there.
25.05.2024 - 2 Came back after 2 hours and perft (7, startpos) still didn't finish (~800mln nodes after 2 hours). What's worse is I've noticed a slow
but steady degradation in nps. When I came back it dropped down to ~131k. Very strange.
Previous run didn' have the same problem! After the same period of time it maintained the tempo.
I thought that maybe it's int overflow but no. Didn't happen. It's a slow gradual slowdown.
Have a log so maybe I can plot it. Or I can profile it at different time points (how?)
25.05.2024 - 3 ok it's speeding back up as I type! It must have been a problem with laptop
energy saving mode. I guess I need to display nps based on speed since the last nps log rather than
cumulative nps.
25.05.2024 - 4 ok. it stopped after 6 hours of searching...
info nodes 4406000000 time 21573542 nps 204231
info nodes 4406500000 time 21574983 nps 204241
info nodes 4407000000 time 21576540 nps 204249
info nodes 4407500000 time 21577877 nps 204260
info pv e2e3 e7e5 d1h5 d8f6 b1c3 f8b4 f1b5 score cp 40 depth 7 nodes 4407845429 time 21578963 nps 204265
bestmove e2e3
The counter didn't overflow - why?
OK. nodes number is higher because of quiescence search.. so that makes sense.
26.05.2024
I've tried to compare prof recordings after my change. I've run go tool as webserver (easiest way to display flamegraphs).
And then I've run into another surprise. I couldn't reliably compare the data - at least the per source line list command.
Old profiles show invalid data interposed on new sources. Basically I was seeing old profile recordings that listed methods that didn't exist at the time.
I've tried to delete the *.exe file but the pprof still run successfully and still has shown the same garbage. I guess I need to recompile exe before running pprof or sth..
27.05.2024
Changed mogeven to generate pawn takes first.
Did test on startpos and search for depth=5 could barely notice any difference in number of nodes evaluated (better
ordering should need fewer nodes evaluation due to some branches being pruned).
Here are 4 consecutives runs for each implementation:
pawn moves mixed
5476069
5499487
5491873
5486023
pawn takes first
5471470
5494910
5497865
5492182
However when I did search depth=4 from kiwipete position the story is different
pawn takes first
104 763 297 nodes evaluated; (nps: 289365; time: 361134)
pawn moves mixed
info nodes 1741500000 time 5665091 nps 307408 (and counting...)
EDIT 02.06.2024 - here are "go depth 4" results after capture ordering (magog.0.6)
info nodes 13000000 time 66124 nps 196599
info nodes 13500000 time 68544 nps 196951
info pv e2a6 e6d5 c3d5 e7e5 score cp 50 depth 4 nodes 13699103 time 69555 nps 196952
info pv e2a6 e6d5 c3d5 e7e5 score cp 50 depth 4 nodes 13699103 time 69555 nps 196952
bestmove e2a6
I had to break the depth=4 (actually kill the program - there's no break flag in
quiescence function so stop command wouldn't work). Here are results for depth=3
pawn takes mixed:
info pv e2a6 e6d5 c3d5 e7e5 score cp 50 depth 3 nodes 134_495_127 time 462728 nps 290656
bestmove e2a6
pawn takes first:
info pv e2a6 e6d5 c3d5 e7e5 score cp 50 depth 3 nodes 26_948_101 time 92187 nps 292319
bestmove e2a6
machess for reference:
info nodes 31172702 pv e2a6 e6d5 c3d5 e7e5 time 142438 depth 3 nps 218849 score cp 50 nodes evaluateddd 31172702
bestmove e2a6
Note that not only the speed but also the ordering is better here than in Machess!
28.05.2024
Did a rematch. Getting pwnd by machess. Somethings bad with move ordering in timebased play..
Not sure what.
In this situation magog is very likely to miss a mate in 1.
2024-05-27 18:34:49,129-->1:position startpos moves e2e4 c7c5 f1c4 e7e5 d1h5
2024-05-27 18:34:49,129-->1:go wtime 285088 btime 280307 winc 0 binc 0
2024-05-27 18:34:49,718<--1:info pv d8f6 c4f7 f6f7 h5e5 score cp -70 depth 4 nodes 168010 time 581 nps 289162
2024-05-27 18:34:50,833<--1:info nodes 500000 time 1690 nps 295765
2024-05-27 18:34:52,566<--1:info nodes 1000000 time 3434 nps 291201
2024-05-27 18:34:54,259<--1:info nodes 1500000 time 5117 nps 293130
2024-05-27 18:34:55,983<--1:info nodes 2000000 time 6844 nps 292197
2024-05-27 18:34:57,689<--1:info nodes 2500000 time 8557 nps 292134
2024-05-27 18:34:58,486<--1:info pv g8f6 g2g4 d8b6 b1c3 f6h5 g4h5 score cp 585 depth 4 nodes 2737382 time 9347 nps 292855
2024-05-27 18:34:58,489<--1:info pv g8f6 g2g4 d8b6 b1c3 f6h5 g4h5 score cp 585 depth 4 nodes 2737382 time 9347 nps 292855
2024-05-27 18:34:58,502<--1:bestmove g8f6
2024-05-27 18:34:58,502*1*Found move:Ng8-f6
If I let it do search to full depth it does not miss it and plays d8f6. However with the bad timing
(quite likely as it turns out) it plays g8f6 and never reaching reply Qh5f7.
__For some reason it does not happen in machess. I need to add extra logging to it and investigate.
Otherwise this blunder looks like could be rectified with killer moves heuristic...
Or trivial fix would be to never stop searching on odd depths (i.e engine's own moves)
29.05.2024
Ok. So the root cause of blunder was the same as was in Machess some years ago. I mistakenly believed that
results from interrupted alphaBeta search at some depth can be any good. No they are not.
In general they're can contain horrible blunder (hanging pieces, mates in 1). If engine had alpha beta interrupted then it should just return
search results from previous iteration of ItDeep (even though some sourcess suggest otherwise).
29.05.2024 - 2
Results after fix:
Engine Score Ma Ma S-B
1: Machess-1.0-SNAPSHOT 7,0/10 ·········· =1110=1=1= 21,00
2: Magog.0.4 3,0/10 =0001=0=0= ·········· 21,00
29.05.2024 - 3
Magog had a bug in parsing moves list in uci position command. It failed to recognize that double pushes make pawns vulnerable
to en-passant attacks. Luckily it was rather easy to find because it made the engine to make illegal moves (tried to capture
with a pawn that wasn't there). So 2 games were lost due to illegal games but
it made at least one other occasion where it made a blunder in equal position by not taking E-P. Bug fixed.
29.05.2024 - 4
Results are only slightly better:
Engine Score Ma Ma S-B
1: Machess-1.0-SNAPSHOT 6,5/10 ·········· =111011=0= 22,75
2: Magog.0.5 3,5/10 =000100=1= ·········· 22,75
Here it strongly prefers to trade bishop for 2 pawns. Very strange as machess with the same evaluation function retreats bishop to b4d6 after finishing depth 3.
However it does play it after full search depth =4 so it's also an ordering/speed issue.
2024-05-29 19:45:30,790-->2:position startpos moves d2d4 e7e6 d1d3 d8h4 c1d2 b8c6 g1f3 h4g4 h2h3 g4h5 d2g5 f7f6 g2g4 h5g6 d3g6 h7g6 g5d2 d7d5 e2e3 f6f5 g4f5 g6f5 b1c3 g8f6 h1g1 c8d7 c3b5 e8d8 f3g5 d8c8 g5f7 h8h4 f2f4 d7e8 f7e5 c6b4 d2b4 f8b4 c2c3 e8b5 e5g6 h4h5 f1b5
2024-05-29 19:45:30,790-->2:go wtime 105073 btime 87638 winc 0 binc 0
2024-05-29 19:45:30,795<--2:info pv a7a6 score cp -30 depth 1 nodes 7 time 0 nps 7000000
2024-05-29 19:45:30,810<--2:info pv c7c6 c3b4 c6b5 score cp -20 depth 1 nodes 17 time 0 nps 17000000
2024-05-29 19:45:30,825<--2:info pv h5h3 c3b4 h3e3 score cp 10 depth 1 nodes 46 time 0 nps 46000000
... no changes in pv line
2024-05-29 19:45:30,905<--2:info pv h5h3 c3b4 h3e3 score cp 10 depth 3 nodes 37634 time 97 nps 387328
2024-05-29 19:45:30,985<--2:info pv h5h3 c3b4 h3e3 e1d2 score cp -145 depth 4 nodes 74809 time 183 nps 408430
2024-05-29 19:45:31,080<--2:info pv a7a6 c3b4 a6b5 g6e7 score cp -125 depth 4 nodes 116083 time 280 nps 413906
2024-05-29 19:45:31,971<--2:info nodes 500000 time 1171 nps 426765
2024-05-29 19:45:32,929<--2:info pv b4d6 h3h4 c7c6 b5d3 score cp -15 depth 4 nodes 914395 time 2136 nps 427968
2024-05-29 19:45:33,136<--2:info nodes 1000000 time 2337 nps 427823
2024-05-29 19:45:33,727<--2:info pv h5h3 c3b4 h3e3 score cp 0 depth 3 nodes 1248174 time 2921 nps 427262
2024-05-29 19:45:33,743<--2:bestmove h5h3
2024-05-29 19:45:33,743*2*Found move:Rh5xh3
01.06.2024
Had great success with magog.0.6 where moves are sorted by capture score. Here are results:
Engine Score Ma Ma S-B
1: Machess-1.0-SNAPSHOT 5,5/10 ииииииииии =1=10=1=0= 24,75
2: Magog.0.6 4,5/10 =0=01=0=1= ииииииииии 24,75
However the big question is. Why it's still worse than Machess? My guess is that it's caused by worse move ordering (raw speed is already better).
And ordering may be worse for two reasons:
1. machess has separate piece lists for every type.
2. It uses copyMakeMove rather than Make/Unmake move. (pieces are shuffled at MoveUnmake)
So I made a version with separate piece list for knights (so pawns, knights and sliding pieces). And this version turned out to be worse
than magog.0.6. Here are arena results:
Engine Score Ma Ma Ma S-B
1: Machess-1.0-SNAPSHOT 11,0/20 ииииииииии =1=10=1=0= =1=101100= 104,50
2: Magog.0.6 10,0/20 =0=01=0=1= ииииииииии 10======1= 99,00
3: Magog.0.7 9,0/20 =0=010011= 01======0= ииииииииии 94,50
It seems that this version is worse because while it has modest ordering improvement for positions with many captures (kiwipete),
it suffers from small decrease in raw speed.
Next to try is the version with copyMakeMove (now I remember that make/unkame version fo Machess was a failure too)
03.06.2024
Finished the copyMakeMove version. It's 3x slower. Profiler shows that there is a lot of heap allocations and gc going on.
Looks like it is related with conversion between values and pointers for method calls. Need to investigate more:
-need to investigate isLegal() version with working on positionStack rather that autoconversions on method calls that golang provides.
04.06.2024
Found a way to make copyMakeMove version work fast. Now it's 5-30% faster than previous one. The problem was related to autoconversions.
For some cases golang decides to place autoconverted variables (pointer -> stack vars) allocated on the heap. I was expecting that it would
be placing them on stack - even if they are big eg. hundreds of bytes (because they are stack variables after all).
This turned out not to be the case.
But anyway the that was only part of the problem. Another thing was that I was using methods on (pos Poision) and (pos *Position) interchangably.
So I decided to stick with (pos *Position). And only in isLegal() I would copy whatever was under pointer to another variable - so this way I
knew that whatever autoconversions were going on they were limited only to this method.
Another crucial thing was using global variable instead of stack variable (variable inside of function). Looks like this pushed golang compiler
to do the right thing:
-it stopped allocating Position on heap
-it stopped doing excessive gc
So now the new version is finally faster than magog.0.6.
04.06.2024
v0.8 lost. Why? It's faster in raw speed Maybe the evaluation function is bad? I should analyze games and look for pattern where Magog.0.8 thinks
it's gaining an edge and then loosing.
-----------------Machess-1.0-SNAPSHOT-----------------
Machess-1.0-SNAPSHOT - Magog.0.6 : 5,0/10 2-2-6 (=1=10===0=) 50% ±0
Machess-1.0-SNAPSHOT - Magog.0.8 : 6,5/10 5-2-3 (1=11101==0) 65% +108
-----------------Magog.0.6-----------------
Magog.0.6 - Machess-1.0-SNAPSHOT : 5,0/10 2-2-6 (=0=01===1=) 50% ±0
Magog.0.6 - Magog.0.8 : 5,0/10 2-2-6 (=1====0=10) 50% ±0
-----------------Magog.0.8-----------------
Magog.0.8 - Machess-1.0-SNAPSHOT : 3,5/10 2-5-3 (0=00010==1) 35% -108
Magog.0.8 - Magog.0.6 : 5,0/10 2-2-6 (=0====1=01) 50% ±0
05.06.2024
Yup looks like it is the case of not so good evaluation function, worse pruning.
Sometimes it barely manages to search one ply deeper than magog 0.6. Searches 4 plies rather than 3
Hypothesis 1: Even with quiescence search, search result with even depth will return more cautious move - because the
line found ends with opponent's reply.
And then it cannot keep up at this depth consistently on next moves.
Here are pairs of games against machess: one game is lost by magog.0.8 and other is drawn by magog.0.6.
Links point to the moves where the games diverge.
https://lichess.org/mE4Sh1X4#18
https://lichess.org/lHYsF7KI#18
https://lichess.org/zbAffPbv#9
https://lichess.org/NY0e6AFO#9
here it actually chose better move according to Stockfish on Lichess
https://lichess.org/NzFITo6Y#17
https://lichess.org/DbXVmMz7#17
In first two of these positions magog0.8 refrained from pushing the pawn so maybe it's high time to
add piece-square tables for pawns.
Hypothesis 2: It's due to bad move ordering: Check version with silent moves descending sorted by type of material moved
(pawns -> knights -> bish -> rook -> queen)
06.06.2024
Checked Hypothesis 1. Played 0.8 agains Machess but this time not at 3 mins but at 4mins. It did a little better:
-----------------Magog.0.8-----------------
Magog.0.8 - Machess-1.0-SNAPSHOT : 4,0/10 2-4-4 (0===00=101) 40% -70
so maybe the the theory about hitting odd depths explains it - but I didn't check the average depths reached really)
Now time to check hypothesis 2...
06.06.2024 - 2
Hypothesis 2 failed. Here are results of magog.0.8 with silent moves ranked by descending value of piece moved.
-----------------Magog.0.8.silentsRanked-----------------
Magog.0.8.silentsRanked - Machess-1.0-SNAPSHOT : 3,0/10 1-5-4 (000===0=10) 30% -147
06.06.2024 - 3
Did a test of magog 0.6 + pawn square tables. Another failure. Played weaker than magog.0.6:
-----------------Magog.0.6.pawnSquareTable-----------------
Magog.0.6.pawnSquareTable - Machess-1.0-SNAPSHOT : 2,0/10 0-6-4 (00==00==00) 20% -241
Machess plays faster - it has expected_moves_to_be_played=40 not 30 like magog.
Hypothesis 3: Maybe magog blunder under time trouble in late game?
06.06.2024 - 4
magog.0.6 + simplified evaluation function for material (P = 100; N = 320; B = 330; R = 500; Q = 900)
Another failure:
-----------------Magog.0.6.sev_material-----------------
Magog.0.6.sev_material - Machess-1.0-SNAPSHOT : 2,5/10 0-5-5 (00=0==00==) 25% -191
08.06.2024
Found what was the problem! It was a bug in beta cutoff in alphaBeta() funcion. I returned
beta only if score > beta. I should have done that when score >= beta. One letter fix!
Started a new tournament. Engine plays noticably better but it run into another bug in the rook endgame.
It exceeded the MaxSearchDepth (which was 40). Could just increase it but even before it got to that
depth it was returning rather strange scores.
2024-06-08 12:50:43,087-->2:go wtime 25596 btime 40009 winc 0 binc 0
2024-06-08 12:50:43,094<--2:info depth 2 pv e4f4 score mate -49949999 nodes 43 time 0 nps 43000000
2024-06-08 12:50:43,109<--2:info depth 3 pv e4f4 score mate -49949999 nodes 63 time 0 nps 63000000
2024-06-08 12:50:43,125<--2:info depth 4 pv e4f4 score mate -49949999 nodes 83 time 0 nps 83000000
...
2024-06-08 12:50:43,436<--2:info depth 24 pv e4f4 score mate -49949999 nodes 483 time 2 nps 167012
2024-06-08 12:50:43,452<--2:info depth 25 pv e4f4 score mate -49949999 nodes 503 time 2 nps 173928
...
2024-06-08 12:50:43,608<--2:info depth 35 pv e4f4 score mate -49949999 nodes 703 time 5 nps 127424
2024-06-08 12:50:43,624<--2:info depth 36 pv e4f4 score mate -49949999 nodes 723 time 5 nps 126354
2024-06-08 12:50:43,639<--2:info depth 37 pv e4f4 score mate -49949999 nodes 743 time 5 nps 129849
2024-06-08 12:50:43,655<--2:info depth 38 pv e4f4 score mate -49949999 nodes 763 time 6 nps 117366
09.06.2024
The bug that cause engine to exceed MaxSearchDepth (and later to overflow the stack) was a rather embarassing one:
When returning a terminal node score I've missed an actual return statement. So on stalemate position rather than returning
stalemate score it would returned minus infinity score (formated as mate -49949999). That's one case of silly error that linter
does not complain about.. Anyway - fixed.
11.06.2024
Tried to apply copyMakeMove again after doing all bugfixes. Great success:
-----------------Magog.0.9.copyMakeMoveBugfixed-----------------
Magog.0.9.copyMakeMoveBugfixed - Magog.0.6.terminalNodeFix : 8,5/10 7-0-3 (11==111=11) 85% +301
Copy make move crushed best version so far that that was using moveMake/Unmake.
Another observation. Need to upgrade set of openings used for testing. Currenlty I use 5:
[Event "1 .King's Pawn Game"]
1.e4
[Event "2. Queen's Pawn Game"]
1.d4
[Event "3. English Opening"]
1.c4
[Event "4. Zukertort Opening"]
1.Nf3
[Event "5. Scandinavian Defense"]
1.e4 d5
And it turns out that both King's Pawn games turned into exact same Scandinavian Defenses anyway (2. ..d5)
12.06.2024
Finetuned material score was better than 0.9 and it is now 0.10.
Tested it against 0.10.pieceSquaresTable version.
-----------------Magog.0.10.pieceSquare-----------------
Magog.0.10.pieceSquare - Magog.0.9.finetunedMaterial : 6,0/14 2-4-8 (=00==1====01=0) 43% -49
pSqT lost but it showed some nice intuition in the endgames.
Probably it lost due to smaller raw speed (couldn't match the depth of prev version). Need to confirm.
Could increase the speed drastically by using Lazy Evaluation (eg. skipping mobility score if score apparently bad - pSq table are good enough replacement-right?)
14.06.2024
Thinking of writing automated testing framework that would save me some time.
Rather than playing test games I could build a base of critical positions (positions when engine blundered).
This data would include both positions and time given for a move - two uci commands)
Framework could then simply plug engine variants and compare what each of them plays and compare it to reference moves
obtained from reference engine.
So by running such test I could quickly have idea how each engine compares to others, is it improving?
14.06.2024 - 2
Did 2 tournaments.
First 0.10.pstLazyEval (meaning piece-square-table+lazy evaluation) against 0.10.
-----------------Magog.0.10.pstLazyEval-----------------
Magog.0.10.pstLazyEval - Magog.0.9.finetunedMaterial : 7,5/14 3-2-9 (=10==1=====1=0) 54% +28
The lazy eval version did a little better than 0.10! An upgrade. However it does hint more that mobility is eval() is a slow thing.
Then I did a tournament 0.10.pstNoMobility against 0.10. The challenger won decisively!
-----------------Magog.0.10.pstNoMobility-----------------
Magog.0.10.pstNoMobility - Magog.0.9.finetunedMaterial : 10,0/14 6-0-8 (=111==1====11=) 71% +156
And few things become apparrent.
1. Boost in nps is very important. Most of the games challenger manages to win by either:
a. winning material with some tactics and overpower defender.
b. winning material with some tactics proceeding to endgame and promoting a pawn
2. However many of the draws that defender obtained were accomplished by infinite checkmate. So the new version that
knows nothing about mobility cannot forsee these situations.
14.06.2024 - 3
Turns out that lazy evaluation with narrower window is even better!
-----------------Magog.0.10.pstLazyEvalNarrow-----------------
Magog.0.10.pstLazyEvalNarrow - Magog.0.10.pstNoMobility : 8,0/14 4-2-8 (0=11=10=1=====) 57% +49
You can see that 0.10.pstNoMobility really does not understand the importance of developing.
Frequently not developing rooks for example.
15.06.2024
Got the results from the games against Enko and DoctorB. Surprise!
It did slightly worse than previous (supposedly weaker) challenger (0.9) few days ago!
-----------------Magog.0.11-----------------
Magog.0.11 - DoctorB : 3,0/14 1-9-4 (0=000===000010) 21% -230
Magog.0.11 - EnkoChess_290818 : 1,5/14 1-12-1 (0000000010000=) 11% -363
old results for reference:
-----------------Magog.0.9-----------------
Magog.0.9 - DoctorB : 3,5/14 2-9-3 (010000=0=001=0) 25% -191
Magog.0.9 - EnkoChess_290818 : 2,0/14 0-10-4 (000000=0=0==00) 14% -315
15.06.2024 - 2
I pit 0.11 against 0.9.
-----------------Magog.0.11-----------------
Magog.0.11 - Magog.0.9 : 7,0/14 3-3-8 (=1==10====1=00) 50% ±0
Looks like 0.11 isn't upgrade over 0.9 after all.
More importantly it's apparent that I need more thorough tests - against more opponents.
This also makes testing with critical positions questionable.
Regarding style of play. All variant of magog are not really good at endings. If some pawn could obviously
be promoted to win but it's there at the starting rank, on a square that PST is suggesting then this
pawn will never move.
Most of the moves in the endings seem rather aimless as soon as the desired result is further than
max depth that's attainable.
And one other thing. I need to include added time in move planning. At the moment engine disregards
added time and always allocates time for next move by dividing time left by expected_moves_to_be_played
(by default 30). So now it's time left goes down asymptotically to 30s. At this point all the moves are garbage.
If it could knew that it has more time then it could keep on playin decently in the crucial part of the game
when positions are still around equal
15.06.2024 - 3
I pit 0.11.addedTime against others. Looks a little better I guess..
-----------------Magog.0.11.addedTime-----------------
Magog.0.11.addedTime - DoctorB : 4,0/14 1-7-6 (=0=0=01===0000) 29% -156
Magog.0.11.addedTime - EnkoChess_290818 : 1,0/14 1-13-0 (00000000010000) 7% -449
Magog.0.11.addedTime - Magog.0.9 : 10,0/14 7-1-6 (1===1111==110=) 71% +156
It lost one game against DoctorB on time - I forgot to use antiflagDuration..
But at least now the log gives me some idea how big should it be. Looks like it should be 10x bigger than initial guess of 4 ms.
Timestamps on the left show that magog was 40 millis late.
2024-06-15 14:07:33,797-->2:go wtime 1 btime 52152 winc 1000 binc 1000
[...]
2024-06-15 14:07:34,715<--2:info currmove g4f5 currmovenumber 1 nodes 1000000 time 870 nps 1148920
2024-06-15 14:07:34,817<--2:info score cp 167 depth 7 nps 1139808 time 1009 nodes 1150405 pv g4f5 d4d6 h2h7 f8g8 h7c7 d6d5 f5e4
2024-06-15 14:07:34,817-->2:stop
2024-06-15 14:07:34,837<--2:bestmove g4f5
2024-06-15 14:07:34,837*2*---------> Arena:Illegal move!: "bestmove g4f5" ()
2024-06-15 14:07:34,837<--2:fatal error: all goroutines are asleep - deadlock!
2024-06-15 14:07:34,837****Tour**Game end: 0-1 {0-1 White forfeits on time}
2024-06-15 14:07:34,837<--2:goroutine 1 [chan send]:
2024-06-15 14:07:34,837<--2:macsmol/magog/engine.ParseInputLine({0xc0003a06c4?, 0xc0000cc02b?})
2024-06-15 14:07:34,837****Tour**Last game won by DoctorB with Black in 141 moves
Also I commit basic openings that I use in tournaments for reference.
16.06.2024
More fixes in time management. Had to do it twice because at one point I've reintroduced bug
fixed on 29.05.2024. Now it's good and playing stronger than ever!:
-----------------Magog.0.11.addedTime3-----------------
Magog.0.11.addedTime3 - DoctorB : 5,0/14 2-6-6 (0==0=01===0001) 36% -100
Magog.0.11.addedTime3 - EnkoChess_290818 : 1,5/14 1-12-1 (00000000010=00) 11% -363
Magog.0.11.addedTime3 - Magog.0.9 : 10,0/14 7-1-6 (=1==1111==110=) 71% +156
But arghh! It still lost on time (against Enko)!
17.06.2024
When it lost against enko another bug become apparent where magog would crash trying to return nil results (because it didn't find anything in ~1 ms).
I've factored out the firs loop of iterative deepening so it will always complete and all the timeout checks are only
done at later stages.
Another thing is that looks like Arena has a bug where it sometimes fails to increment wtime by winc. Or it attributes it to other player.
Not sure exactly. But one thing certain is that in that game Enko was in time trouble (4 consecutive moves with btime=1ms) and then out of thin air it got +1000ms.
These 5 moves by Enko played with the same tempo (~1015ms according to timestamps in arena.debug).
OTOH magog was printing bestmove in ~50 ms (according to timestamps in arena.debug) but still got ~200ms less on successive moves until eventually it crashed.
Will see it gets magical bonus if it does not crash.
17.06.2024 - 2
Fixed the crash and looks like it can exploit the same time measuring quirk in Arena. This time it managed a 3-fold repetition because Enko couldn't mate it with pair
of bishops. Also scores 1.5 point more against DrB than 0.9! By only improving time control.
-----------------Magog.0.11.addedTime4-----------------
Magog.0.11.addedTime4 - DoctorB : 5,0/14 2-6-6 (0=00=01===0=01) 36% -100
Magog.0.11.addedTime4 - EnkoChess_290818 : 2,0/14 1-11-2 (00000000010==0) 14% -315
Magog.0.11.addedTime4 - Magog.0.9 : 10,0/14 7-1-6 (=1==1111==110=) 71% +156
23.06.2024
Good thing that I decided to add tperft from command line.
I discovered two bugs in tactical movegen thanks to that.
Fixed the bugs and crosschecked the fixes against AdaChess.
25.06.2024
Found a bug in time management when lowering movestogo ,eg.
> go wtime 500 btime 501 winc 500 binc 500 movestogo 2
It exceeds time by 205 millis!
< info score cp -50 depth 6 nps 413307 time 705 nodes 291683 pv b8c6 b1c3 e7e5 g1f3 f8c5 f1c4
< bestmove b8c6