chess engines testers

[sarona]

Oliprog - stats :

Code: Select all

013 players                                    : avg. D31, 005 239 ms/move, 002 882 games, 00 175 380 moves

ai 40.3 avx2                                   : avg. D32, 005 293 ms/move, 000 394 games, 00 012 317 moves
ai 40.5 avx2                                   : avg. D32, 005 202 ms/move, 000 624 games, 00 015 865 moves
ai 40.7 avx2                                   : avg. D31, 005 056 ms/move, 000 344 games, 00 009 209 moves
brainlearn 23.1 avx2                           : avg. D31, 005 664 ms/move, 000 300 games, 00 007 291 moves
catropoly v1.8 20230506 avx2                   : avg. D29, 005 376 ms/move, 000 304 games, 00 009 552 moves
corchess 3 dev-20230430-nogit, 64-bit avx2 goi : avg. D29, 005 479 ms/move, 000 288 games, 00 008 519 moves
polyfish 230426 avx2                           : avg. D32, 005 151 ms/move, 000 300 games, 00 010 046 moves
sf pb 050523 avx2 skynet                       : avg. D32, 004 977 ms/move, 000 628 games, 00 022 241 moves
stockfishmz 060523 avx2                        : avg. D32, 005 028 ms/move, 000 306 games, 00 010 678 moves
stockfishmz 280423 avx2                        : avg. D32, 005 404 ms/move, 000 394 games, 00 012 304 moves
sugar ai 040523 avx2                           : avg. D31, 005 274 ms/move, 000 990 games, 00 029 331 moves
sugar ai se avx2                               : avg. D32, 005 290 ms/move, 000 300 games, 00 010 063 moves
zeus 40.5 avx2                                 : avg. D31, 005 265 ms/move, 000 592 games, 00 017 964 moves

Averages :
0 319 sec/game (5 min/game)
079 plies/game (061 played plies/game)

Game stages :
- opening @ avg. D28, 007 734 ms/move
- middleg @ avg. D30, 004 870 ms/move
- endgame @ avg. D35, 003 261 ms/move

Code: Select all

005 players                     : avg. D31, 005 500 ms/move, 000 500 games, 00 029 710 moves

corchess 3, dev-20230423-7a447a : avg. D30, 005 644 ms/move, 000 200 games, 00 005 992 moves
polyfish 230421 avx2            : avg. D32, 005 450 ms/move, 000 200 games, 00 006 041 moves
sf pb 220423 avx2 skynet        : avg. D32, 005 570 ms/move, 000 200 games, 00 005 877 moves
stockfish, dev-20230425-41f50b  : avg. D32, 005 495 ms/move, 000 200 games, 00 005 985 moves
zeus 40.3 avx2                  : avg. D32, 005 336 ms/move, 000 200 games, 00 005 815 moves

Averages :
0 327 sec/game (5 min/game)
075 plies/game (059 played plies/game)

Game stages :
- opening @ avg. D28, 007 526 ms/move
- middleg @ avg. D31, 005 327 ms/move
- endgame @ avg. D35, 003 779 ms/move

79 piles/game and 75 plies/game averages?

Are these numbers correct, Chris?

[deeds]

79 piles/game and 75 plies/game averages?

Are these numbers correct, Chris?

Hi sarona, I think so. I did not recount them myself by hand.

This data comes from a customer who noticed that the majority of OCF's testers produced the worst quality games.

I guess he must have used a modified version of my pgnToDepth tool because by default it is not compatible with Chessbase or Banksia PGNs.

I'm waiting for his chart to be updated because several testers are still missing, he even forgot about me lol

[deeds]

Hi,
About your testers chart, a friend is surprised by the Oliprog stats :

79 piles/game and 75 plies/game averages?

Are these numbers correct, Chris?

Can you tell us more (e.g. your method) or confirm these values ?
Thanks in advance, Chris.

ps : do you have an updated chart ?

Hi Mr JEAN, hope all fine for your family too.

So, pgnToDepth crashes with PGNs commented by Fritz (chessbase) or
pgn4web (tcec)... problem with the durations with Banksia PGNs, wrong values !?

I don't really have a "method",
I just "prepare" the PGNs before using pgnToDepth... Example with the latest Oliprog tests (OCF page #179 and #180 except BIN tests) :



I used the chessbase's 3rd-party tool



now pgnToDepth works...



Thanks for your games i will add in the next chart, maybe next we, not at home now héhé

Ciao.

[deeds]

new additions

[deeds]

Mehmet Karaman - stats :

Code: Select all

006 players               : avg. D24, 001 330 ms/move, 002 602 games, 00 357 709 moves

genko 1.0 bmi2            : avg. D23, 001 317 ms/move, 001 800 games, 00 124 044 moves
genko 1.1 bmi2            : avg. D26, 001 252 ms/move, 000 602 games, 00 044 943 moves
kayra15                   : avg. D25, 001 930 ms/move, 000 200 games, 00 009 882 moves
kayra17                   : avg. D27, 002 001 ms/move, 000 200 games, 00 009 899 moves
stockfish 191122 x64 bmi2 : avg. D23, 001 306 ms/move, 001 800 games, 00 124 019 moves
stockfish 211222 x64 bmi2 : avg. D28, 001 232 ms/move, 000 602 games, 00 044 922 moves

Averages :
0 183 sec/game (3 min/game)
147 plies/game (137 played plies/game)

Game stages :
- opening @ avg. D21, 002 258 ms/move
- middleg @ avg. D23, 001 052 ms/move
- endgame @ avg. D28, 000 588 ms/move

Code: Select all

007 players               : avg. D34, 004 608 ms/move, 002 178 games, 00 319 853 moves

dragon 3.1 x64 avx2       : avg. D35, 004 156 ms/move, 000 220 games, 00 019 022 moves
genko 1.0a bmi2           : avg. D32, 004 697 ms/move, 000 792 games, 00 058 151 moves
kayra 1.5 bmi2            : avg. D33, 004 324 ms/move, 000 440 games, 00 035 601 moves
kayra 1.7 bmi2            : avg. D34, 004 825 ms/move, 001 122 games, 00 076 552 moves
stockfish 071122 x64 bmi2 : avg. D36, 004 938 ms/move, 000 330 games, 00 020 432 moves
stockfish 15.1 x64 bmi2   : avg. D34, 004 589 ms/move, 001 232 games, 00 091 084 moves
stockfish 170822 x64 bmi2 : avg. D34, 004 177 ms/move, 000 220 games, 00 019 011 moves

Averages :
0 677 sec/game (11 min/game)
157 plies/game (147 played plies/game)

Game stages :
- opening @ avg. D28, 007 638 ms/move
- middleg @ avg. D34, 003 856 ms/move
- endgame @ avg. D39, 002 370 ms/move

[deeds]

Christophe Jean - stats :

Code: Select all

010 players                         : avg. D36, 005 178 ms/move, 002 599 games, 00 215 339 moves

2070 lc0 0.28.2 cuda 611245         : avg. D11, 004 600 ms/move, 000 559 games, 00 030 387 moves
20t brainlearn 15.2 nn-ac07bd334b62 : avg. D40, 004 871 ms/move, 000 560 games, 00 023 118 moves
20t brainlearn 17.1 nn-3c0aa92af1da : avg. D37, 005 211 ms/move, 000 480 games, 00 017 853 moves
20t eman 7.80 nn-ac07bd334b62       : avg. D50, 004 922 ms/move, 000 559 games, 00 023 837 moves
20t eman 8.10 nn-3c0aa92af1da       : avg. D37, 005 712 ms/move, 000 480 games, 00 017 582 moves
20t hypnos 040122 nn-ac07bd334b62   : avg. D38, 005 006 ms/move, 000 560 games, 00 023 166 moves
20t hypnos 090622 nn-3c0aa92af1da   : avg. D37, 005 233 ms/move, 000 480 games, 00 018 861 moves
20t judas 090622 nn-3c0aa92af1da    : avg. D37, 006 491 ms/move, 000 480 games, 00 018 627 moves
20t stockfish 14.1 x64 bmi2         : avg. D40, 004 930 ms/move, 000 560 games, 00 023 756 moves
20t stockfish 15.0 x64 bmi2         : avg. D37, 005 459 ms/move, 000 480 games, 00 018 152 moves

Averages :
0 429 sec/game (7 min/game)
108 plies/game (083 played plies/game)

Game stages :
- opening @ avg. D32, 007 775 ms/move
- middleg @ avg. D35, 004 655 ms/move
- endgame @ avg. D40, 003 264 ms/move

[deeds]

Massimiliano Goi - stats :

Code: Select all

011 players                      : avg. D31, 006 573 ms/move, 000 600 games, 00 043 536 moves

blue marlin 15.2a-bmi2           : avg. D33, 006 334 ms/move, 000 100 games, 00 003 462 moves
brainlearn 17                    : avg. D32, 006 804 ms/move, 000 100 games, 00 003 392 moves
corchess 3 020622                : avg. D29, 006 757 ms/move, 000 100 games, 00 003 627 moves
crystal 060622                   : avg. D26, 006 592 ms/move, 000 100 games, 00 004 040 moves
dragon 3 by komodo chess, 64-bit : avg. D30, 006 403 ms/move, 000 100 games, 00 003 902 moves
eman 8.03 64-bit bmi2            : avg. D31, 006 177 ms/move, 000 100 games, 00 003 415 moves
fisherov v0.98c nnue popcnt      : avg. D30, 006 191 ms/move, 000 100 games, 00 003 776 moves
proteussf rbe 008b               : avg. D32, 007 272 ms/move, 000 100 games, 00 003 586 moves
shashchess 22                    : avg. D34, 006 756 ms/move, 000 100 games, 00 003 595 moves
stockfish polyglot 15 x64, bmi2. : avg. D33, 006 684 ms/move, 000 100 games, 00 003 576 moves
stockfishmz 230522 bmi2          : avg. D33, 006 460 ms/move, 000 200 games, 00 007 165 moves

Averages :
0 477 sec/game (8 min/game)
073 plies/game (073 played plies/game)

Game stages :
- opening @ avg. D29, 007 675 ms/move
- middleg @ avg. D30, 006 748 ms/move
- endgame @ avg. D34, 005 297 ms/move

[deeds]

ghppn - stats :

Code: Select all

022 players                   : avg. D33, 030 481 ms/move, 000 402 games, 00 029 551 moves

brainlearn 16.1               : avg. D35, 033 150 ms/move, 000 022 games, 00 000 662 moves
cor pb 100322                 : avg. D29, 033 540 ms/move, 000 050 games, 00 001 723 moves
corchess 3 100122             : avg. D32, 031 546 ms/move, 000 020 games, 00 000 791 moves
corchess 3 100322             : avg. D30, 034 808 ms/move, 000 022 games, 00 000 657 moves
corchess 3 190222             : avg. D31, 028 466 ms/move, 000 120 games, 00 004 856 moves
corchess 3 190422             : avg. D30, 034 962 ms/move, 000 050 games, 00 001 620 moves
corchess 3 200322             : avg. D31, 032 561 ms/move, 000 030 games, 00 000 958 moves
corchess 3 270222             : avg. D30, 034 779 ms/move, 000 020 games, 00 000 653 moves
eman 7.93 64-bit bmi2         : avg. D35, 029 040 ms/move, 000 060 games, 00 002 216 moves
fat titz 110122 64 bmi2 numa  : avg. D32, 029 827 ms/move, 000 020 games, 00 000 791 moves
fat titz 2 130122 64 bmi2 num : avg. D36, 026 435 ms/move, 000 030 games, 00 001 239 moves
kayra 1.1 bmi2                : avg. D40, 025 206 ms/move, 000 030 games, 00 001 378 moves
kayra 1.2 bmi2                : avg. D36, 030 785 ms/move, 000 030 games, 00 000 961 moves
kayra 1.3 bmi2                : avg. D34, 033 426 ms/move, 000 050 games, 00 001 618 moves
raubfisch gtz27d3_nn_sl-bmi2  : avg. D38, 028 294 ms/move, 000 030 games, 00 001 168 moves
sf pb 130322                  : avg. D33, 032 765 ms/move, 000 050 games, 00 001 729 moves
sf pb 170222                  : avg. D36, 029 064 ms/move, 000 030 games, 00 001 086 moves
sf pb 250222                  : avg. D35, 034 547 ms/move, 000 020 games, 00 000 656 moves
stockfishmz 170522 bmi2       : avg. D36, 027 263 ms/move, 000 030 games, 00 001 111 moves
stockfishmz iccf 230522 bmi2  : avg. D25, 033 651 ms/move, 000 030 games, 00 001 101 moves
stockfishmz iccf 240522 bmi2  : avg. D30, 030 065 ms/move, 000 030 games, 00 001 288 moves
sugar ai iccf 2.50 bmi2       : avg. D25, 027 415 ms/move, 000 030 games, 00 001 289 moves

Averages :
2 241 sec/game (37 min/game)
081 plies/game (074 played plies/game)

Game stages :
- opening @ avg. D34, 039 977 ms/move
- middleg @ avg. D32, 029 729 ms/move
- endgame @ avg. D32, 021 737 ms/move

[deeds]

Andreas Strangmuller - stats :

Code: Select all

010 players                : avg. D37, 051 997 ms/move, 006 750 games, 01 248 973 moves

berserk 10                 : avg. D31, 044 251 ms/move, 001 350 games, 00 158 869 moves
dragon by komodo chess 3.1 : avg. D41, 048 424 ms/move, 001 350 games, 00 137 818 moves
ethereal 13.75 nnue        : avg. D41, 051 717 ms/move, 001 350 games, 00 125 811 moves
fat fritz 2 (in sf)        : avg. D46, 051 957 ms/move, 001 350 games, 00 118 521 moves
fire 8 nn.mc.3             : avg. D27, 054 470 ms/move, 001 350 games, 00 117 529 moves
koivisto 8.0               : avg. D37, 052 393 ms/move, 001 350 games, 00 125 767 moves
rubichess 2021 nnue        : avg. D38, 057 405 ms/move, 001 350 games, 00 113 790 moves
seer 2.6.0                 : avg. D36, 054 935 ms/move, 001 350 games, 00 115 988 moves
slowchess blitz nn 2.8     : avg. D27, 053 445 ms/move, 001 350 games, 00 116 735 moves
stockfish 15               : avg. D45, 054 328 ms/move, 001 350 games, 00 118 145 moves

Averages :
9 621 sec/game (160 min/game)
206 plies/game (185 played plies/game)

Game stages :
- opening @ avg. D34, 089 673 ms/move
- middleg @ avg. D37, 040 666 ms/move
- endgame @ avg. D40, 025 604 ms/move

[sarona]

Thanks for the information.

I only download Peter's test results from OCF when available. I have full confidence in his data and include it in the engine saronaBase.

I do not even look at any test pgns/results performed without using increment. I see no logic in testing that way.