OpenChess

I think I may have found some Stockfish settings that do a little better than the default settings. I call them Stockfish 1.7.1 JA SPOON. I also included games from Stockfish 1.7.1 JA Default settings for comparison. Here are some results:

Stockfishtest13 2010

Deep Rybka 4 w32 - Stockfish 1.7.1 JA SPOON 26.0 - 24.0 +20/=12/-18 52.00%
Deep Rybka 4 w32 - Stockfish 1.7.1 JA Default 28.0 - 22.0 +23/=10/-17 56.00%

Stockfishtest14 2010

Deep Rybka 4 w32 - Stockfish 1.7.1 JA Default 70.5 - 49.5 +43/=55/-22 58.75%
Deep Rybka 4 w32 - Stockfish 1.7.1 JA SPOON 68.5 - 51.5 +49/=39/-32 57.08%

5/0 games
Book used = Balanced-12.ctg
All 5 men EGTB
128 MB hash tables
1 CPU Ponder off for both engines

Here are the settings:

Mobility (Middle Game) - 110
Mobility (Endgame) - 95
Pawn Structure (Middle Game) - 110
Pawn Structure (Endgame) - 95
Passed Pawns (Middle Game) - 95
Passed Pawns (Endgame) - 110
Space - 110
Agressiveness - 90
Cowardice - 110

Are there any testers here that can confer with my results?

Could you test with 1000 games at 1 minute per game (1'+0") between the original version and the modified one (It will take about 1,5 days) ?

Thanks

Stockfishtest16-1 2010


1 Stockfish 1.7.1 JA +18/=48/-11 54.55% 42.0/77
2 Stockfish 1.7.1 JA SPOON +11/=48/-18 45.45% 35.0/77

mcostalba wrote:Could you test with 1000 games at 1 minute per game (1'+0") between the original version and the modified one (It will take about 1,5 days) ?

Thanks

I can test it on my laptop. But I have a few questions. My laptop is celeron 1400 and have 32 bit XP. Also I use arena and in 1 min games the engines forfeit on time sometimes are the number of games forfeited negligible?

Taner Altinsoy wrote:

mcostalba wrote:Could you test with 1000 games at 1 minute per game (1'+0") between the original version and the modified one (It will take about 1,5 days) ?

Thanks

I can test it on my laptop. But I have a few questions. My laptop is celeron 1400 and have 32 bit XP. Also I use arena and in 1 min games the engines forfeit on time sometimes are the number of games forfeited negligible?

I am currently testing it at 1/0 with 999 games. I have not been able to go round the clock with it though. Here are current results:

Stockfishtest16-1 2010


1 Stockfish 1.7.1 JA +34/=85/-23 53.87% 76.5/142
2 Stockfish 1.7.1 JA SPOON +23/=85/-34 46.13% 65.5/142

I read another post saying the developers of Stockfish are mystified why the new settings make the program weaker. I've played one game with the SPOON settings and found Stockfish more conservative in approach than the initial parameters. What I'm finding a mystery is why Stockfish is getting rejected by my copy of Chess Assistant. I have an old version (7.1) and for some reason I'm getting that message saying Stockfish is not a UCI engine. I've only been able to configure this with Chessbase, Arena, Jose, ChessPartner and Aquarium. Would purchasing the latest version of Chess Assistant fix this problem?

I'm running a 1 min 1000 game match. So far 396 games played. Stockfish default leading with 205.5/396 vs 190.5/396.

Taner

Taner Altinsoy wrote:I'm running a 1 min 1000 game match. So far 396 games played. Stockfish default leading with 205.5/396 vs 190.5/396.

Taner

Nice test, thanks !

But 396 games are still not much, I have seen one version to seem +15 ELO stronger after 500 games and going to lose once arrived at 1000. Even this could happen although, luckily, is rare.

Regarding Arena I would not suggest that GUI for engine testing at short TC because has a bad time accounting bug that shows when you are few seconds from the clock and makes engines lose by time. To mitigate that you could test at 1'+0.2" instead of 1+0", but the real solution is to use something else.

After 611 games these are my results:

Stockfishtest16-1 2010


1 Stockfish 1.7.1 JA +133/=382/-96 53.03% 324.0/611
2 Stockfish 1.7.1 JA SPOON +96/=382/-133 46.97% 287.0/611

Not looking too grand for my settings =O(

This is more or less in line with Taner's results.

It is interesting to note that changing parameters by a mere 5% gives this considerable effect....

Another observation to take home is how tests based on small number of games could be misleading: you really need at least 1000 games and, if change is small, even 1000 is not enough