Strange Stockfish behavior?

[UncombedCoconut]

I can't reproduce it even if I use the "go depth 20" command to do the search.
The best way to solve this mystery might be capturing the actual UCI communication between SF and the GUI.
The GUI might have that option. Otherwise, the InBetween tool here is the best solution I know of on Windows. You would use InBetwee.exe in SF's folder as an engine, with an InBetween.ini file looking something like this:

Code: Select all

[InBetween] 
CommandLine := stockfish.exe
Log := logfile.log 

[Ancalagon]

I'm using Shredder Classic, here, when I put the engine to analyze a position, it goes at it endlessly, but I also can put it in "Human against the Machine mode", there, I can choose time control, I put it at 20ply (so, I make a move, and Stockfish reaches 20 plies and makes a move, then it's my turn again) and force a move (so it's as if I moved) I get different behavior.

If your Fritz GUI gives you that behavior in Infinite analysis, then it doesn't give you the option of analysis mode (which shouldn't matter because the engine's behavior should be the same!) Fritz GUI would use "Play mode" for everything.

So, just GUI quirks, it's still a mystery why Stockfish behaves differently whether is has to "analyze" or "play a move"in Shredder Classic.

I think that could be easily explained, because King Safety is a bit different in real Analyze Mode, there is no difference between own and opponent King Safety in UCI_AnalyseMode. I know that in Chess Partner UCI_AnalyseMode is not used and the engine just goes to infinite, but I don't know if it is the same for Fritz. I thought not but you never know. In endgames it should not matter as then King safety is not computed anymore.

Eelco

From evaluate.cpp:

Code: Select all

  // If running in analysis mode, make sure we use symmetrical king safety. We do this
  // by replacing both Weights[kingDangerUs] and Weights[kingDangerThem] by their average.
  if (Options["UCI_AnalyseMode"].value<bool>())
      Weights[kingDangerUs] = Weights[kingDangerThem] = (Weights[kingDangerUs] + Weights[kingDangerThem]) / 2;

  init_safety();
}

[ernest]

I'm using Shredder Classic, here, when I put the engine to analyze a position, it goes at it endlessly, but I also can put it in "Human against the Machine mode"

Hi Uly,

With my Shredder Classic GUI, I reproduced your "infinite analysis" (with result different from mine in Fritz GUI).

But how do you put it in "Human against the Machine mode" (what Shredder Menu tabs do you use?)? Couldn't find it...

[ernest]

But how do you put it in "Human against the Machine mode" (what Shredder Menu tabs do you use?)? Couldn't find it...

OK Uly, found it: you have to launch the "analysis to depth 20" with the Menu Commands... Compute!

[Uly]

But how do you put it in "Human against the Machine mode" (what Shredder Menu tabs do you use?)? Couldn't find it...

It's Mode > Play against Computer (it switches automatically to it when you tell the engine to make a move, or use the Interrupt button on the toolbar).

But what UncombedCoconut mentions could mean it's an Shredder Classic bug, will investigate when I have time.

[UncombedCoconut]

I think that could be easily explained, because King Safety is a bit different in real Analyze Mode, there is no difference between own and opponent King Safety in UCI_AnalyseMode. I know that in Chess Partner UCI_AnalyseMode is not used and the engine just goes to infinite, but I don't know if it is the same for Fritz. I thought not but you never know. In endgames it should not matter as then King safety is not computed anymore.

You're right; I forgot UCI Aggressiveness and Cowardice were scaled. (It changes weights of 247,259 to 253,253.)
I can now reproduce easily.

Marco or Joona:
In SF-devel, do you still do

Code: Select all

            if (value <= alpha && i >= MultiPV)
                rml[i].pv_score = -VALUE_INFINITE;

after this?

Code: Select all

            // Finished searching the move. If AbortSearch is true, the search
            // was aborted because the user interrupted the search or because we
            // ran out of time. In this case, the return value of the search cannot
            // be trusted, and we break out of the loop without updating the best
            // move and/or PV.
            if (StopRequest)
                break;

The result is that the fail-high/fail-low move will still be played if search stops soon enough.
If I understand correctly this can happen in games. So if current SF tests better than playing these moves, maybe reversing these steps will test better than current SF.

But if things stay as-is, it might be better to have the pv match the move SF will play. That would mean printing unresolved fail-highs, and reprinting the current pv if they resolve as fail-lows. I can write a patch, if it helps...

Another question:
Have you guys tried giving these fail-low/fail-high a bonus in move ordering?

[mcostalba]

The result is that the fail-high/fail-low move will still be played if search stops soon enough.
If I understand correctly this can happen in games. So if current SF tests better than playing these moves, maybe reversing these steps will test better than current SF.

If it is possible I would prefer a patch for this point above because it is not completely clear to me what you are trying to say, so if you write a patch that changes behaviour to what should _fix_ this bug I can understand better.

Another question:
Have you guys tried giving these fail-low/fail-high a bonus in move ordering?

No, and also in this case it is not clear to me what do you mean exactly, care to write a patch also for this ?

Thanks
Marco

[UncombedCoconut]

A fair bit of that was mistaken.

But regarding what I'd patch:
I and others want SF to print a new PV every time it changes its mind about the best move.
If a move fails high, SF will play that move if it's interrupted during resolution.
If the resolution fails low, SF will change its mind.
As a hack, I print the second case as an lowerbound like the first. There's no reason except to tell the GUI there's still no multipv score update, and to make it "look like" part of the same fail high resolution.

The diff in SearchLogs (also visible in UCI output) for Uly's position (which, argh, the forum software is re-indenting):

Code: Select all

--- SearchLog.txt-before	2011-04-13 23:59:42.053061006 -0700
+++ SearchLog.txt-after	2011-04-13 23:48:48.723061006 -0700
@@ -1,36 +1,38 @@
 Searching: r2k1r2/pp1n1q1p/1npP4/4p1P1/4P3/1PbB2NP/P1P1Q1RK/3R4 b - -
 infinite: 0 ponder: 0 time: 0 increment: 0 moves to go: 0
  1     +0.35   00:00      152 Ke8 
  2     -0.27   00:00      196 Ke8 Nf5 
  2     +0.34   00:00      392 Qf3 Rf1 Qxe2 Rxf8+ Nxf8 Nxe2 
  3     -0.14   00:00      705 Qf3 Nf5 Qxe2 Bxe2 
  4     -0.12   00:00     1339 Qf3 Nf5 Qxe2 Bxe2 Nd5 
  5     +0.10   00:00     2080 Qf3 Nf5 Qxe2 Bxe2 Nc5 Bf3 
  6     +0.20   00:00     3717 Qf3 Qxf3 Rxf3 Nf5 Rf4 Kg3 Ke8 
  7  >  +0.44   00:00     6469 Qf3 Qxf3 Rxf3 Nf5 Nc5 Be2 Rf4 
  7     +0.34   00:00     9714 Qf3 a3 Qxe2 Bxe2 Bb2 a4 Nc5 a5 
  7  >  +0.69   00:00    13266 Bb4 Rf1 Qe6 Rxf8+ Nxf8 
+ 7  >  +0.34   00:00    15732 Qf3 a3 Qxe2 Bxe2 Bb2 a4 Nc5 a5 
  8  >  +0.57   00:00    22579 Qf3 a3 Qxe2 Bxe2 a5 Nf5 Nc5 
  8     +0.49   00:00    23800 Qf3 a3 Qxe2 Bxe2 a5 Bg4 a4 Ne2 Bd4 
  9     +0.39   00:00    37107 Qf3 a4 Qxe2 Bxe2 Nc5 Bg4 Ke8 Bf5 Rg8 Ne2 
 10     +0.29   00:00    52600 Qf3 Qxf3 Rxf3 Be2 Rf4 Bg4 Bb4 Bf5 Nc8 Nh5 Rf3 Bg4 
 11  <  +0.11   00:00    69915 Qf3 Qxf3 Rxf3 Be2 Rf4 Bg4 Bb4 Bf5 Nc8 Nh5 Rf3 Nf6 
                               Nxf6 gxf6 Nxd6 Rg8+ 
 11     +0.11   00:00    98989 Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Nh5 a6 Ng7 Nc5 Bf5 
                               Rf7 
 12     +0.07   00:00   123993 Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Bf5 Nc8 c3 Bc5 b4 
                               Bxd6 Bxh7 
 13     +0.00   00:00   224376 Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Bf5 Rf7 Be6 Rf8 Bf5 
 14     +0.00   00:00   287084 Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Bf5 Rf7 Be6 Rf8 Bf5 
 15     +0.00   00:00   426958 Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Bf5 Rf7 Be6 Rf8 Bf5 
 16     +0.00   00:01   833391 Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Bf5 Rf7 Be6 Rf8 Bf5 
 17     +0.00   00:02    1493K Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Bf5 Rf7 Be6 Rf8 Bf5 
 18     +0.00   00:04    2550K Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Bf5 Rf7 Be6 Rf8 Bf5 
 19     +0.00   00:06    3886K Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Bf5 Rf7 Be6 Rf8 Bf5 
 19  >  +0.11   00:12    7460K a5 Nf5 a4 Qe3 axb3 cxb3 Bb4 Rc2 Qe6 Be2 Nd5 
 19  >  +0.22   00:15    9357K a5 Nf5 a4 Qe3 axb3 cxb3 Bb4 Rc2 Qe6 Be2 Nd5 
+19  >  +0.00   00:32   18990K Qf3 Qxf3 Rxf3 Be2 Rf8 Bg4 Bb4 Bf5 Rf7 Be6 Rf8 Bf5 
 
 Nodes: 19001715
-Nodes/second: 604630
+Nodes/second: 591493
 Best move: Qf3
 Ponder move: Qxf3

So Uly could have used depth 7 here and saved people computer time.

The change:

Code: Select all

--- search.cpp  2011-04-13 23:46:13.063061007 -0700
+++ search.cpp-after    2011-04-13 23:42:45.393061006 -0700
@@ -811,6 +811,7 @@
         // Sort the moves before to (re)search
         rml.set_non_pv_scores(pos);
         rml.sort();
+        int bestMoveIndex = 0;

         // Step 10. Loop through all moves in the root move list
         for (int i = 0; i < (int)rml.size() && !StopRequest; i++)
@@ -944,12 +945,22 @@

             // Step 17. Check for new best move
             if (value <= alpha && i >= MultiPV)
+            {
                 rml[i].pv_score = -VALUE_INFINITE;
+
+                if (ss->bestMove == move)
+                {
+                    // If we're reverting a fail-high, send the GUI another
+                    // "lower bound" line with the former PV.
+                    cout << rml[bestMoveIndex].pv_info_to_uci(pos, alpha-1, alpha) << endl;
+                }
+            }
             else
             {
                 // PV move or new best move!

                 // Update PV
+                bestMoveIndex = i;
                 ss->bestMove = move;
                 rml[i].pv_score = value;
                 rml[i].extract_pv_from_tt(pos);

BTW, inside the new if clause, should we reset ss->bestMove to what it was before the fail-high SF is no longer trusting? What's the effect of setting it at the root?

[mcostalba]

If I read your patch correctly it seems there is no functional change here, just an extra update to the GUI in case of fail low. Is it correct ?

[UncombedCoconut]

If I read your patch correctly it seems there is no functional change here, just an extra update to the GUI in case of fail low. Is it correct ?

Correct. That was my intent. However, I also wonder whether updating ss->bestMove in this case would cause a functional change. (I'm away from the code right now.) It seems like it should either be updated when a fail-low causes a change in best move, or never updated at the root