Designing an analysis friendly Stockfish?

[LucenaTheLucid]

Code: Select all

   Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini 1.5a                   : 2918   22  22  1040    82.1 %   2654   20.9 %
  2 Deep Rybka 4                   : 2809   19  19   971    70.3 %   2659   29.2 %
  3 Deep Rybka 4.1                 : 2808   26  26   548    71.4 %   2650   29.9 %
  4 Stockfish 2.0.1                : 2777   17  17  1038    65.9 %   2663   35.5 %
  5 Stockfish 2.0.1 PA             : 2775   19  19   920    66.4 %   2657   31.7 %
  6 Critter 0.90                   : 2772   18  18  1040    65.3 %   2663   31.3 %
  7 Stockfish 2.0.1 Lucena 1       : 2768   20  19   880    65.0 %   2660   31.6 %
  8 Armageddon 1.0                 : 2725   44  44   195    61.0 %   2647   21.5 %
  9 Naum 4.2                       : 2688   17  17  1039    52.8 %   2668   32.5 %
 10 Gull 1.2                       : 2676   20  20   864    51.4 %   2666   28.4 %
 11 Spike 1.4                      : 2670   18  18  1040    50.2 %   2669   31.3 %
 12 Komodo 1.3                     : 2631   18  18  1040    44.3 %   2671   28.8 %
 13 Spark 1.0                      : 2620   18  18  1039    42.5 %   2672   27.7 %
 14 Gull 1.1                       : 2614   20  20   861    40.5 %   2680   28.8 %
 15 Thinker 5.4D Inert             : 2588   18  18  1037    37.9 %   2674   27.3 %
 16 TogaII1.4 5c                   : 2546   20  20   964    32.2 %   2675   23.5 %
 17 Protector 1.4.0                : 2522   20  20  1039    28.9 %   2678   23.6 %
 18 Hannibal 1.0a                  : 2522   21  22   865    29.1 %   2676   23.8 %
 19 Loop 2007                      : 2501   47  48   195    28.5 %   2661   19.0 %
 20 Zappa Mexico II                : 2445   22  22  1038    20.3 %   2682   19.6 %
 21 Jonny 4.00                     : 2441   51  53   195    21.5 %   2666   16.4 %

Just 1 minute time controls with .1 second increment. An older machine. No Gaviota tablebases. At these time controls it looks like Jeremy's Stockfish doesn't see a dramatic decrease in playing strength. Perhaps 1-2 points.

[Sarciness]

Thanks for all of your hard work, guys! I am enjoying this offshoot of Stockfish greatly.

I have a couple of questions about future Stockfish versions:
1) You have improved tome control, evaluation jumping and also added GTB support, is that right? Any other changes that I missed?
2) How many of these changes will be absorbed into the next official release of Stockfish?
3) How easy will it be to re-implement any changes not included in next official Stockfish?

Cheers again!
Sarc

[Uly]

>1) You have improved tome control, evaluation jumping and also added GTB support, is that right? Any other changes that I missed?

The evaluation jumping is still there, what was improved was "evaluation remembering", so that it remembers how moves are evaluated, and all the moves that transpose or lead to them have the correct score that they should.

The granularity was also fixed so instead of 0.40 -> 0.44 -> 0.48 scores Stockfish goes 0.41 -> 0.42 -> 0.43 for extra accuracy.

It's also using Smooth Scaling (as in Stockfish 1.6s).

>2) How many of these changes will be absorbed into the next official release of Stockfish?

The remembering of scores will be better.

>3) How easy will it be to re-implement any changes not included in next official Stockfish?

I don't know but my guess is that it'll be trivially easy.

[snino64]

could be added "evaluation Remembering" in gtb?
This seems to work in Houdini 1.5a using Gaviota (Tb).
[FEN "8/4k3/8/3BB3/4K3/8/8/8 w - - 0 1"]

1.Bc3
( { [Houdini 1.5a x64] 1:[M12]} 1.Bc6 Ke6 2.Bb5 Ke7 3.Kf5 Kd8 4.Ke6
Kc8 5.Kd5 Kb7 6.Kc5 Kc8 7.Kb6 Kd8 8.Bd6 Kc8 9.Be7 Kb8 10.Ba6 Ka8 11.
Bb7+ Kb8 12.Bd6# )
( { [Houdini 1.5a x64] 1:[M12]} 1.Bd4 Kd6 2.Bb6 Ke7 3.Ke5 Kd7 4.Ba5
Ke7 5.Be6 Kf8 6.Kf6 Ke8 7.Bb6 Kf8 8.Bd7 Kg8 9.Kg6 Kf8 10.Bc5+ Kg8 11.
Be6+ Kh8 12.Bd4# )
( { [Houdini 1.5a x64] 1:[M12]} 1.Bc3 Kd6 2.Bb4+ Kc7 3.Ke5 Kb6 4.Kd6
Kb5 5.Bc5 Ka4 6.Kc6 Ka5 7.Bb3 Ka6 8.Bb4 Ka7 9.Kc7 Ka6 10.Bc4+ Ka7 11.
Bc5+ Ka8 12.Bd5# )
( { [Houdini 1.5a x64] 1:(M12)} 1.Kf5 )
( { [Houdini 1.5a x64] 1:(M12)} 1.Bg3 )
*
Houdini show:
1 1 [M12] 1.Bc6 Ke6 2.Bb5 Ke7 3.Kf5 Kd8 4.Ke6 Kc8 5.Kd5 Kb7 6.Kc5 Kc8 7.Kb6 Kd8 8.Bd6 Kc8 9.Be7 Kb8 10.Ba6 Ka8 11.Bb7+ Kb8 12.Bd6#
2 1 (M12) 1.Bd4
3 1 (M12) 1.Bc3
4 1 (M12) 1.Kf5
5 1 (M12) 1.Bg3

If we make the move Bd4, and go back to the same position, Houdini shows the following variations (return to starting position):

1 1 [M12] 1.Bc6 Ke6 2.Bb5 Ke7 3.Kf5 Kd8 4.Ke6 Kc8 5.Kd5 Kb7 6.Kc5 Kc8 7.Kb6 Kd8 8.Bd6 Kc8 9.Be7 Kb8 10.Ba6 Ka8 11.Bb7+ Kb8 12.Bd6#
2 1 [M12] 1.Bd4 Kd6 2.Bb6 Ke7 3.Ke5 Kd7 4.Ba5 Ke7 5.Be6 Kf8 6.Kf6 Ke8 7.Bb6 Kf8 8.Bd7 Kg8 9.Kg6 Kf8 10.Bc5+ Kg8 11.Be6+ Kh8 12.Bd4#
3 1 (M12) 1.Bc3
4 1 (M12) 1.Kf5
5 1 (M12) 1.Bg3

That is, stores the analysis of the line, Bc3, and displays it in a previous position (return to starting position).

1 1 [M12] 1.Bc6 Ke6 2.Bb5 Ke7 3.Kf5 Kd8 4.Ke6 Kc8 5.Kd5 Kb7 6.Kc5 Kc8 7.Kb6 Kd8 8.Bd6 Kc8 9.Be7 Kb8 10.Ba6 Ka8 11.Bb7+ Kb8 12.Bd6#
2 1 [M12] 1.Bd4 Kd6 2.Bb6 Ke7 3.Ke5 Kd7 4.Ba5 Ke7 5.Be6 Kf8 6.Kf6 Ke8 7.Bb6 Kf8 8.Bd7 Kg8 9.Kg6 Kf8 10.Bc5+ Kg8 11.Be6+ Kh8 12.Bd4#
3 1 [M12] 1.Bc3 Kd6 2.Bb4+ Kc7 3.Ke5 Kb6 4.Kd6 Kb5 5.Bc5 Ka4 6.Kc6 Ka5 7.Bb3 Ka6 8.Bb4 Ka7 9.Kc7 Ka6 10.Bc4+ Ka7 11.Bc5+ Ka8 12.Bd5#
4 1 (M12) 1.Kf5
5 1 (M12) 1.Bg3

[Uly]

could be added "evaluation Remembering" in gtb?

What do you mean? What's the current behavior of Stockfish Gran2K in those situations?

[snino64]

[FEN "8/4k3/8/3BB3/4K3/8/8/8 w - - 0 1"]

Using 1.2 stockfish PA Gtb 2k (Multipv = 5)

1 1 [M12] 1.Bc6 Ke6 2.Bb5 Kf7 3.Kf5 Ke7 4.Bc7 Kf7 5.Bd8 Kg7 6.Bg5 Kf7 7.Bf6 Kg8 8.Kg6 Kf8 9.Bg5 Kg8 10.Bh6 Kh8 11.Bg7+ Kg8 12.Bc4#
2 1 (M12) 1.Bb7
3 1 (M12) 1.Bg8
4 1 (M12) 1.Bc3
5 1 (M12) 1.Bg3

if I now realize the play Bb7 and undo the move, and the result is the same stockfish

1 1 [M12] 1.Bc6 Ke6 2.Bb5 Kf7 3.Kf5 Ke7 4.Bc7 Kf7 5.Bd8 Kg7 6.Bg5 Kf7 7.Bf6 Kg8 8.Kg6 Kf8 9.Bg5 Kg8 10.Bh6 Kh8 11.Bg7+ Kg8 12.Bc4#
2 1 (M12) 1.Bb7
3 1 (M12) 1.Bg8
4 1 (M12) 1.Bc3
5 1 (M12) 1.Bg3

with houdini also shows the second variation

first initial position
1 1 [M12] 1.Bc6 Ke6 2.Bb5 Kf7 3.Kf5 Ke7 4.Bc7 Kf7 5.Bd8 Kg7 6.Bg5 Kf7 7.Bf6 Kg8 8.Kg6 Kf8 9.Bg5 Kg8 10.Bh6 Kh8 11.Bg7+ Kg8 12.Bc4#
2 1 (M12) 1.Bb7
3 1 (M12) 1.Bg8
4 1 (M12) 1.Bc3
5 1 (M12) 1.Bg3

I now realize the play Bd4 and undo the move, and the result is the same stockfish

1 1 [M12] 1.Bc6 Ke6 2.Bb5 Ke7 3.Kf5 Kd8 4.Ke6 Kc8 5.Kd5 Kb7 6.Kc5 Kc8 7.Kb6 Kd8 8.Bd6 Kc8 9.Be7 Kb8 10.Ba6 Ka8 11.Bb7+ Kb8 12.Bd6#
2 1 [M12] 1.Bd4 Kd6 2.Bb6 Ke7 3.Ke5 Kd7 4.Ba5 Ke7 5.Be6 Kf8 6.Kf6 Ke8 7.Bb6 Kf8 8.Bd7 Kg8 9.Kg6 Kf8 10.Bc5+ Kg8 11.Be6+ Kh8 12.Bd4#
3 1 (M12) 1.Bc3
4 1 (M12) 1.Kf5
5 1 (M12) 1.Bg3

I now realize the play Bb7 and undo the move, and the result is the same houdini (this lack in the variations of houdini)

1 1 [M12] 1.Bc6 Ke6 2.Bb5 Ke7 3.Kf5 Kd8 4.Ke6 Kc8 5.Kd5 Kb7 6.Kc5 Kc8 7.Kb6 Kd8 8.Bd6 Kc8 9.Be7 Kb8 10.Ba6 Ka8 11.Bb7+ Kb8 12.Bd6#
2 1 [M12] 1.Bd4 Kd6 2.Bb6 Ke7 3.Ke5 Kd7 4.Ba5 Ke7 5.Be6 Kf8 6.Kf6 Ke8 7.Bb6 Kf8 8.Bd7 Kg8 9.Kg6 Kf8 10.Bc5+ Kg8 11.Be6+ Kh8 12.Bd4#
3 1 (M12) 1.Bc3
4 1 (M12) 1.Kf5
5 1 (M12) 1.Bg3

'ie houdini retrieves the previous analysis with the play Bd4 and now shows two complete changes, retrieves only the variations that appear in the first place. Bb7 not appear in any variation.

thank you very much for the effort and add support to stockfish gtb. Only missing this little detail.

[Peter C]

>3) How easy will it be to re-implement any changes not included in next official Stockfish?

I don't know but my guess is that it'll be trivially easy.

It won't be particularly hard. I know from experience that adding GTB support is really easy and the other stuff shouldn't be difficult.

Peter

[Uly]

Bb7 not appear in any variation.

Huh, why should this happen? Aren't all these moves mates in 12 so the moves at MultiPV=5 are random moves that mate in 12? Or am I missing something?

[snino64]

8/4k3/8/3BB3/4K3/8/8/8 w - - 0 1
With Houdini, multipv tbGaviota = 5 and shows the following variations
1 1 [M12] 1.Bc6 Ke6 2.Bb5 Ke7 3.Kf5 Kd8 4.Ke6 Kc8 5.Kd5 Kb7 6.Kc5 Kc8 7.Kb6 Kd8 8.Bd6 Kc8 9.Be7 Kb8 10.Ba6 Ka8 11.Bb7+ Kb8 12.Bd6#
2 1 (M12) 1.Bd4
3 1 (M12) 1.Bc3
4 1 (M12) 1.Kf5
5 1 (M12) 1.Bg3

but if I move one ply in each variant and then undo the ply, houdini shows

1 1 [M12] 1.Bc6 Ke6 2.Bb5 Ke7 3.Kf5 Kd8 4.Ke6 Kc8 5.Kd5 Kb7 6.Kc5 Kc8 7.Kb6 Kd8 8.Bd6 Kc8 9.Be7 Kb8 10.Ba6 Ka8 11.Bb7+ Kb8 12.B1 1 [M12] 1.Bc6 Ke6 2.Bb5 Ke7 3.Kf5 Kd8 4.Ke6 Kc8 5.Kd5 Kb7 6.Kc5 Kc8 7.Kb6 Kd8 8.Bd6 Kc8 9.Be7 Kb8 10.Ba6 Ka8 11.Bb7+ Kb8 12.Bd6#
2 1 [M12] 1.Bd4 Kd6 2.Bb6 Ke7 3.Ke5 Kd7 4.Ba5 Ke7 5.Be6 Kf8 6.Kf6 Ke8 7.Bb6 Kf8 8.Bd7 Kg8 9.Kg6 Kf8 10.Bc5+ Kg8 11.Be6+ Kh8 12.Bd4#
3 1 [M12] 1.Bg3 Kf6 2.Bh4+ Kg6 3.Be7 Kh5 4.Be6 Kg6 5.Ke5 Kh5 6.Bf5 Kh6 7.Kf6 Kh5 8.Kf7 Kh6 9.Bg4 Kh7 10.Bf8 Kh8 11.Bg7+ Kh7 12.Bf5#
4 1 [M12] 1.Bc3 Kd6 2.Bb4+ Kc7 3.Ke5 Kb6 4.Kd6 Kb5 5.Bc5 Ka4 6.Kc6 Ka5 7.Bb3 Ka6 8.Bb4 Ka7 9.Kc7 Ka6 10.Bc4+ Ka7 11.Bc5+ Ka8 12.Bd5#
5 1 [M12] 1.Kf5 Kd7 2.Bb7 Ke7 3.Bc8 Kd8 4.Be6 Ke7 5.Bg3 Kd8 6.Kf6 Ke8 7.Bc7 Kf8 8.Bd7 Kg8 9.Kg6 Kf8 10.Bd6+ Kg8 11.Be6+ Kh8 12.Be5#



However, Stockfish always shows the same variations (Stockfish 2.01 pa gtb gran2k, multipv=5, tbgaviota)

1 1 [M12] 1.Bc6 Ke6 2.Bb5 Kf7 3.Kf5 Ke7 4.Bc7 Kf7 5.Bd8 Kg7 6.Bg5 Kf7 7.Bf6 Kg8 8.Kg6 Kf8 9.Bg5 Kg8 10.Bh6 Kh8 11.Bg7+ Kg8 12.Bc4#
2 1 (M12) 1.Bb7
3 1 (M12) 1.Bg8
4 1 (M12) 1.Bc3
5 1 (M12) 1.Bg3

and if I move one ply in each variant and then undo the ply, stockfish shows


1 1 [M12] 1.Bc6 Ke6 2.Bb5 Kf7 3.Kf5 Ke7 4.Bc7 Kf7 5.Bd8 Kg7 6.Bg5 Kf7 7.Bf6 Kg8 8.Kg6 Kf8 9.Bg5 Kg8 10.Bh6 Kh8 11.Bg7+ Kg8 12.Bc4#
2 1 (M12) 1.Bb7
3 1 (M12) 1.Bg8
4 1 (M12) 1.Bc3
5 1 (M12) 1.Bg3

Why not show the moves to mate variations in number 2, 3.4 and 5 and Houdini yes?

[Uly]

could be added "evaluation Remembering" in gtb?

What you show is not evaluation remembering, the evaluation is a mate in 12, Stockfish remembers that it's a mate in 12, so it's remembering the evaluation correctly.

What you are asking for is "show all moves up to mate in mate positions", this could be done even without having to interact with the position and seems reasonable, maybe someone else can help.