Designing an analysis friendly Stockfish?

[Jeremy Bernstein]

OK. 88. Kf5 is suboptimal, dropping us from +M37 after 87.Dd5 back to +M41. 90.Kg6 drops us back to +M52 from +M39. 95.Kg4 +M49 instead of Ke4 +M47 and so on. It's one step forward, five steps backward.

What I don't know is: are the tablebases at fault? Or is the probing code at fault? I'll continue to research it, but maybe someone knows what's going on already.

UPDATE: Houdini finds 88.Kf7 no problem, so there's most likely an issue with the probing code (unless the move is getting chucked for some other reason). I'll see what I can puzzle out in the next day or two.

It looks like, probably under time pressure, Stockfish was grabbing values from the hash and not checking the TBs. Do we need to add a TB check in id_loop, comparing it to any hash value for determining EasyMove? Strangely, I'm not seeing any TB hits in the root search, either.

It appears that, when the probing code is executed, it correctly finds Kf7. For some reason, though, it didn't get executed in this case. Still looking...

UPDATE: if I force a hard TB probe at the root, it is returning Kf5. It looks like alternative moves aren't even being considered.

Jeremy

[mcostalba]

Normally we don't post development code but because I see interest is great and becuase, anyhow, you have started with this idea then here we are

This is the final patch as applied to development branch and backported to 2.0.1 for you to use. We have finished to work on this subject and moved on.

Code: Select all

--- a/src/search.cpp
+++ b/src/search.cpp
@@ -991,14 +991,11 @@ namespace {
     tte = TT.retrieve(posKey);
     ttMove = tte ? tte->move() : MOVE_NONE;
 
-    // At PV nodes, we don't use the TT for pruning, but only for move ordering.
-    // This is to avoid problems in the following areas:
-    //
-    // * Repetition draw detection
-    // * Fifty move rule detection
-    // * Searching for a mate
-    // * Printing of full PV line
-    if (!PvNode && tte && ok_to_use_TT(tte, depth, beta, ply))
+    // At PV nodes we check for exact scores, while at non-PV nodes we check for
+    // and return a fail high/low. Biggest advantage at probing at PV nodes is
+    // to have a smooth experience in analysis mode.
+    if (tte && (PvNode ? tte->depth() >= depth && tte->type() == VALUE_TYPE_EXACT
+                       : ok_to_use_TT(tte, depth, beta, ply)))
     {
         TT.refresh(tte);
         ss->bestMove = ttMove; // Can be MOVE_NONE
diff --git a/src/tt.cpp b/src/tt.cpp
index 63c3e99..5b7e737 100644
--- a/src/tt.cpp
+++ b/src/tt.cpp
@@ -121,7 +121,7 @@ void TranspositionTable::store(const Key posKey, Value v, ValueType t, Depth d,
           continue;
 
       c1 = (replace->generation() == generation ?  2 : 0);
-      c2 = (tte->generation() == generation ? -2 : 0);
+      c2 = (tte->generation() == generation || tte->type() == VALUE_TYPE_EXACT ? -2 : 0);
       c3 = (tte->depth() < replace->depth() ?  1 : 0);
 
       if (c1 + c2 + c3 > 0)

[fruity]

Normally we don't post development code but because I see interest is great and becuase, anyhow, you have started with this idea then here we are

This is the final patch as applied to development branch and backported to 2.0.1 for you to use. We have finished to work on this subject and moved on.

Code: Select all

Biggest advantage at probing at PV nodes is to have a smooth experience in analysis mode.

And the biggest drawbacks

* Repetition draw detection
* Fifty move rule detection
* Searching for a mate
* Printing of full PV line

have dissapeared miraculously?

[Jeremy Bernstein]

OK. 88. Kf5 is suboptimal, dropping us from +M37 after 87.Dd5 back to +M41. 90.Kg6 drops us back to +M52 from +M39. 95.Kg4 +M49 instead of Ke4 +M47 and so on. It's one step forward, five steps backward.

What I don't know is: are the tablebases at fault? Or is the probing code at fault? I'll continue to research it, but maybe someone knows what's going on already.

UPDATE: Houdini finds 88.Kf7 no problem, so there's most likely an issue with the probing code (unless the move is getting chucked for some other reason). I'll see what I can puzzle out in the next day or two.

It looks like, probably under time pressure, Stockfish was grabbing values from the hash and not checking the TBs. Do we need to add a TB check in id_loop, comparing it to any hash value for determining EasyMove? Strangely, I'm not seeing any TB hits in the root search, either.

It appears that, when the probing code is executed, it correctly finds Kf7. For some reason, though, it didn't get executed in this case. Still looking...

UPDATE: if I force a hard TB probe at the root, it is returning Kf5. It looks like alternative moves aren't even being considered.

Jeremy

I've found the problem and should have a fix for it at some point in the next 24 hours or so.

Jeremy

[gaard]

Normally we don't post development code but because I see interest is great and becuase, anyhow, you have started with this idea then here we are

This is the final patch as applied to development branch and backported to 2.0.1 for you to use. We have finished to work on this subject and moved on.

Code: Select all

--- a/src/search.cpp
+++ b/src/search.cpp
@@ -991,14 +991,11 @@ namespace {
     tte = TT.retrieve(posKey);
     ttMove = tte ? tte->move() : MOVE_NONE;
 
-    // At PV nodes, we don't use the TT for pruning, but only for move ordering.
-    // This is to avoid problems in the following areas:
-    //
-    // * Repetition draw detection
-    // * Fifty move rule detection
-    // * Searching for a mate
-    // * Printing of full PV line
-    if (!PvNode && tte && ok_to_use_TT(tte, depth, beta, ply))
+    // At PV nodes we check for exact scores, while at non-PV nodes we check for
+    // and return a fail high/low. Biggest advantage at probing at PV nodes is
+    // to have a smooth experience in analysis mode.
+    if (tte && (PvNode ? tte->depth() >= depth && tte->type() == VALUE_TYPE_EXACT
+                       : ok_to_use_TT(tte, depth, beta, ply)))
     {
         TT.refresh(tte);
         ss->bestMove = ttMove; // Can be MOVE_NONE
diff --git a/src/tt.cpp b/src/tt.cpp
index 63c3e99..5b7e737 100644
--- a/src/tt.cpp
+++ b/src/tt.cpp
@@ -121,7 +121,7 @@ void TranspositionTable::store(const Key posKey, Value v, ValueType t, Depth d,
           continue;
 
       c1 = (replace->generation() == generation ?  2 : 0);
-      c2 = (tte->generation() == generation ? -2 : 0);
+      c2 = (tte->generation() == generation || tte->type() == VALUE_TYPE_EXACT ? -2 : 0);
       c3 = (tte->depth() < replace->depth() ?  1 : 0);
 
       if (c1 + c2 + c3 > 0)

http://dl.dropbox.com/u/11904592/stockfish_201_pamc.7z

This has the code changes backported to 2.0.1. All PGO builds. 64-bit version is under JA's by 2% but the popcnt version is ahead by 2%.

Any other developments you want to share?

[gaard]

Normally we don't post development code but because I see interest is great and becuase, anyhow, you have started with this idea then here we are

This is the final patch as applied to development branch and backported to 2.0.1 for you to use. We have finished to work on this subject and moved on.

Code: Select all

--- a/src/search.cpp
+++ b/src/search.cpp
@@ -991,14 +991,11 @@ namespace {
     tte = TT.retrieve(posKey);
     ttMove = tte ? tte->move() : MOVE_NONE;
 
-    // At PV nodes, we don't use the TT for pruning, but only for move ordering.
-    // This is to avoid problems in the following areas:
-    //
-    // * Repetition draw detection
-    // * Fifty move rule detection
-    // * Searching for a mate
-    // * Printing of full PV line
-    if (!PvNode && tte && ok_to_use_TT(tte, depth, beta, ply))
+    // At PV nodes we check for exact scores, while at non-PV nodes we check for
+    // and return a fail high/low. Biggest advantage at probing at PV nodes is
+    // to have a smooth experience in analysis mode.
+    if (tte && (PvNode ? tte->depth() >= depth && tte->type() == VALUE_TYPE_EXACT
+                       : ok_to_use_TT(tte, depth, beta, ply)))
     {
         TT.refresh(tte);
         ss->bestMove = ttMove; // Can be MOVE_NONE
diff --git a/src/tt.cpp b/src/tt.cpp
index 63c3e99..5b7e737 100644
--- a/src/tt.cpp
+++ b/src/tt.cpp
@@ -121,7 +121,7 @@ void TranspositionTable::store(const Key posKey, Value v, ValueType t, Depth d,
           continue;
 
       c1 = (replace->generation() == generation ?  2 : 0);
-      c2 = (tte->generation() == generation ? -2 : 0);
+      c2 = (tte->generation() == generation || tte->type() == VALUE_TYPE_EXACT ? -2 : 0);
       c3 = (tte->depth() < replace->depth() ?  1 : 0);
 
       if (c1 + c2 + c3 > 0)

http://dl.dropbox.com/u/11904592/stockfish_201_pamc.7z

This has the code changes backported to 2.0.1. All PGO builds. 64-bit version is under JA's by 2% but the popcnt version is ahead by 2%.

Any other developments you want to share?

Actually, there was a gross oversight in the patch I applied from M. Costalba. You will need to re-download.

[BB+]

But at some point, Stockfish is making moves which are wrong, and it loses track of the forced mate.

There could be two problems: not finding the shortest path to mate -- this could be because of how mate scores are hashed w.r.t. transpositions, but if the GTB root probe occurs before a TT lookup, that should not matter.

The second is coming up with bad moves. Looking at the code, the GTB (root) probe seems just to get the score from the tablebase, and doesn't iterate over the first ply so as to get the best move. Then it just returns the value, and I guess the move is determined simply from the last line in id_loop, namely "return rml[0].pv[0]", which should return the first move in the rml (root move list). Now my understanding is that the rml is ordered based upon a qsearch -- but note that the GTB probes don't occur at qsearch nodes (if a GTB probe value is hashed, it should be recalled though). So I think that's your answer -- when there is a GTB probe at root, then code always returns the first move from the root move qsearch ordering, which is not guaranteed to have values determined by GTB probes.

EDIT: From above, it looks like you found some problem also (maybe the same one). The "Decembrists" seem to avoid all this with RobboBases in their ComStock version by doing root probes in id_loop rather than in root_search, and by having a probing system that returns the best move to boot.

[Jeremy Bernstein]

But at some point, Stockfish is making moves which are wrong, and it loses track of the forced mate.

There could be two problems: not finding the shortest path to mate -- this could be because of how mate scores are hashed w.r.t. transpositions, but if the GTB root probe occurs before a TT lookup, that should not matter.

The second is coming up with bad moves. Looking at the code, the GTB (root) probe seems just to get the score from the tablebase, and doesn't iterate over the first ply so as to get the best move. Then it just returns the value, and I guess the move is determined simply from the last line in id_loop, namely "return rml[0].pv[0]", which should return the first move in the rml (root move list). Now my understanding is that the rml is ordered based upon a qsearch -- but note that the GTB probes don't occur at qsearch nodes (if a GTB probe value is hashed, it should be recalled though). So I think that's your answer -- when there is a GTB probe at root, then code always returns the first move from the root move qsearch ordering, which is not guaranteed to have values determined by GTB probes.

EDIT: From above, it looks like you found some problem also (maybe the same one). The "Decembrists" seem to avoid all this with RobboBases in their ComStock version by doing root probes in id_loop rather than in root_search, and by having a probing system that returns the best move to boot.

That's more or less what I've done. I should have looked at the ComStock code to save myself the time... Nevertheless, it was good to have to puzzle out the code a bit, just for the sake of familiarity. In id_loop(), I'm first doing a tb probe before doing the root search. The tb probe now will also now return the fastest mate. The existing code was just returning the first hit, no matter what.

Anyhow, new builds posted (if you downloaded from the post I've deleted in the meantime, please redownload -- there was a "problem"). These incorporate proper root node probing, Peter's other changes, the Marco changes, the Decembrist TT changes, and so on. Should be the best of all worlds, at least from the feature perspective. Please let me know how they do from the functional perspective. 32- and 64-bit builds included.

Enjoy!

[Prima]

gaard,Jeremy, Marcos Costalba etc,

thanks for the patch builds. What version of Preserve Analysis (F - J ?) is this latest patch from Marcos Costalba based on?

[Jeremy Bernstein]

gaard,Jeremy, Marcos Costalba etc,

thanks for the patch builds. What version of Preserve Analysis (F - J ?) is this latest patch from Marcos Costalba based on?

It's Stockfish 2.0.1 + the Stockfish team's own code for PA on top of it.