Censor in the Rybka forum

[benstoker]

As I was impressed by the results you reported (quite unexpected for me), I downloaded the "IvanHoe999953.exe" version from the ChessLogik site.
It cannot solve any of the previously mentioned positions. I've also tried "IvanHoe999952a.exe" from the same web site, same negative results.

Below the results I obtain with IvanHoe999952a.exe after 30 to 60 seconds in each position:

Position 1: 1Q6/8/8/2pK3p/8/5N1q/1P4pp/5Rbk w - - 0 1
29 00:28,423 59.130.699 2.130.000 +0,05 1.Qxh2+ Qxh2 2.Rc1 h4 3.Ke4 c4 4.Rd1 c3 5.bxc3 h3 6.Rc1 Qe5+ 7.Kxe5 h2 8.Rc2 Bb6 9.Ke4 g1Q 10.Rxh2+ Qxh2 11.Nxh2 Kxh2 12.Kd5 Kg2 13.c4 Bf2 14.Kd6 Kf3 15.Kd7 Ba7 16.Kc7 Bd4 17.Kb7 Ke3 18.Kc6 Ke4 19.c5 Kf3 20.Kd5 Bxc5 21.Kc4 Bf2

Position 2: 3Q4/3p4/P2p4/N2b4/8/4P3/5p1p/5Kbk w - - 0 1
23 00:31,949 56.383.647 1.802.000 +0,28 1.Qf6 Be4 2.Qxf2 Bd3+ 3.Qe2 Bxe2+ 4.Kxe2 Kg2 5.a7 h1Q 6.a8Q+ d5 7.Qxd5+ Kh2 8.Qh5+ Kg2 9.Qg4+ Kh2 10.Qf4+ Kh3 11.Qh6+ Kg2 12.Qg7+ Kh2 13.Qh7+ Kg2 14.Qe4+ Kh2 15.Qe5+ Kh3 16.Qf5+ Kg3 17.Nc4 Qg2+ 18.Kd3 Qf3 19.Qg5+ Kf2 20.Qf4 Kg2 21.Qd6 Qf5+ 22.Kd4 Qg4+ 23.Kc3 Qe6 24.Qd4 d5 25.Qg7+ Kf3

Position 3: 5q1k/5P1p/Q7/5n1p/6P1/7K/8/8 w - - 0 1
29 00:59,124 121.196.546 2.073.000 0,00 1.gxf5 Qxf7 2.Qa1+ Kg8 3.Qe5 Kf8 4.f6 Qd7+ 5.Kg3 Qg4+ 6.Kf2 Qb4 7.Qxh5 Qb2+ 8.Kg3 Qxf6 9.Qxh7 Qc3+ 10.Kg2 Qd2+ 11.Kh3 Qc3+ 12.Kh4 Qc4+ 13.Kg5 Qc1+ 14.Kg4 Qd1+ 15.Kf4 Qc1+ 16.Ke5 Qa1+ 17.Kf4 Qc1+

Is there something wrong with the KLO compiles on the ChessLogik site?

Robert

That's because the original source code is screwed up - or shall we say, backwards. For some very strange unexplained reason. The booleans bzug and wzug are negated in evaluation.c and mobility.c. It's quite strange that they would publish this in this way. It literally turns OFF the code. Just switch the declarations to false and the conditions to true. Maybe they were testing things and uploaded a test version. Here's the diff for evaluation.c and then mobility.c below that, which will switch the TRUE/FALSE on bzug and wzug:

Code: Select all

--- ../IvanHoe-Beta-999953-Beta/evaluation.c	2010-07-16 09:22:00.000000000 -0500
+++ evaluation.c	2010-08-02 12:27:09.000000000 -0500
@@ -354,7 +354,7 @@
   typePawnEval PAWN_INFO[1];
   int ch;
 #ifdef ZUGZWANG_DETECT
-  boolean wzug = TRUE, bzug = TRUE;
+  boolean wzug = FALSE, bzug = FALSE;
 #endif
 
   PAWN_POINTER = PawnHash + (POSITION->DYN->PAWN_HASH & (PawnHashSize - 1));
@@ -633,7 +633,7 @@
       T = A & wSafeMob;
 #ifdef ZUGZWANG_DETECT
       if (A & ~wBitboardOcc)
-	wzug = FALSE;
+	wzug = TRUE;
 #endif
       POSITION->DYN->wAtt |= A;
       if (A & bKatt)
@@ -669,7 +669,7 @@
       A = AttR (b);
 #ifdef ZUGZWANG_DETECT
       if (A & ~wBitboardOcc)
-	wzug = FALSE;
+	wzug = TRUE;
 #endif
       POSITION->DYN->wAtt |= A;
       if (bBitboardK & ORTHO[b])
@@ -777,7 +777,7 @@
       A = AttB (b);
 #ifdef ZUGZWANG_DETECT
       if (A & ~wBitboardOcc)
-	wzug = FALSE;
+	wzug = TRUE;
 #endif
       POSITION->DYN->wAtt |= A;
       if (bBitboardK & DIAG[b])
@@ -853,7 +853,7 @@
       A = AttN[b];
 #ifdef ZUGZWANG_DETECT
       if (A & ~wBitboardOcc)
-	wzug = FALSE;
+	wzug = TRUE;
 #endif
       POSITION->DYN->wAtt |= A;
       if (A & (bKatt | bBitboardK))
@@ -942,7 +942,7 @@
       A = AttB | AttR;
 #ifdef ZUGZWANG_DETECT
       if (A & ~bBitboardOcc)
-	bzug = FALSE;
+	bzug = TRUE;
 #endif
       T = A & bSafeMob;
       POSITION->DYN->bAtt |= A;
@@ -981,7 +981,7 @@
       A = AttR (b);
 #ifdef ZUGZWANG_DETECT
       if (A & ~bBitboardOcc)
-	bzug = FALSE;
+	bzug = TRUE;
 #endif
       POSITION->DYN->bAtt |= A;
       if (wBitboardK & ORTHO[b])
@@ -1089,7 +1089,7 @@
       A = AttB (b);
 #ifdef ZUGZWANG_DETECT
       if (A & ~bBitboardOcc)
-	bzug = FALSE;
+	bzug = TRUE;
 #endif
       POSITION->DYN->bAtt |= A;
       if (wBitboardK & DIAG[b])
@@ -1167,7 +1167,7 @@
       A = AttN[b];
 #ifdef ZUGZWANG_DETECT
       if (A & ~bBitboardOcc)
-	bzug = FALSE;
+	bzug = TRUE;
 #endif
       POSITION->DYN->bAtt |= A;
       if (A & (wKatt | wBitboardK))

And mobility.c

Code: Select all

--- ../IvanHoe-Beta-999953-Beta/mobility.c	2010-07-16 09:22:00.000000000 -0500
+++ mobility.c	2010-08-02 12:28:53.000000000 -0500
@@ -6,7 +6,7 @@
   uint64 U, A, T, AttB, AttR;
   int b;
 #ifdef ZUGZWANG_DETECT
-  boolean wzug = TRUE, bzug = TRUE;
+  boolean wzug = FALSE, bzug = FALSE;
 #endif
   POSITION->DYN->wXray = 0;
   POSITION->DYN->bXray = 0;
@@ -28,7 +28,7 @@
       A = AttN[b];
 #ifdef ZUGZWANG_DETECT
       if (A & ~wBitboardOcc)
-	wzug = FALSE;
+	wzug = TRUE;
 #endif
       POSITION->DYN->wAtt |= A;
       if (A & bBitboardK)
@@ -40,7 +40,7 @@
       A = AttB (b);
 #ifdef ZUGZWANG_DETECT
       if (A & ~wBitboardOcc)
-	wzug = FALSE;
+	wzug = TRUE;
 #endif
       POSITION->DYN->wAtt |= A;
       if (A & bBitboardK)
@@ -59,7 +59,7 @@
       A = AttR (b);
 #ifdef ZUGZWANG_DETECT
       if (A & ~wBitboardOcc)
-	wzug = FALSE;
+	wzug = TRUE;
 #endif
       POSITION->DYN->wAtt |= A;
       if (A & bBitboardK)
@@ -80,7 +80,7 @@
       A = AttB | AttR;
 #ifdef ZUGZWANG_DETECT
       if (A & ~wBitboardOcc)
-	wzug = FALSE;
+	wzug = TRUE;
 #endif
       POSITION->DYN->wAtt |= A;
       if (A & bBitboardK)
@@ -106,7 +106,7 @@
       A = AttN[b];
 #ifdef ZUGZWANG_DETECT
       if (A & ~bBitboardOcc)
-	bzug = FALSE;
+	bzug = TRUE;
 #endif
       POSITION->DYN->bAtt |= A;
       if (A & wBitboardK)
@@ -118,7 +118,7 @@
       A = AttB (b);
 #ifdef ZUGZWANG_DETECT
       if (A & ~bBitboardOcc)
-	bzug = FALSE;
+	bzug = TRUE;
 #endif
       POSITION->DYN->bAtt |= A;
       if (A & wBitboardK)
@@ -137,7 +137,7 @@
       A = AttR (b);
 #ifdef ZUGZWANG_DETECT
       if (A & ~bBitboardOcc)
-	bzug = FALSE;
+	bzug = TRUE;
 #endif
       POSITION->DYN->bAtt |= A;
       if (A & wBitboardK)
@@ -158,7 +158,7 @@
       A = AttB | AttR;
 #ifdef ZUGZWANG_DETECT
       if (A & ~bBitboardOcc)
-	bzug = FALSE;
+	bzug = TRUE;
 #endif
       POSITION->DYN->bAtt |= A;
       if (A & wBitboardK)

[benstoker]

This post was deleted in the Rybka forum.

Rybka Forum 29-07-2010 ----- Test Positions ---- "Zugzwang with a capital Z"

By stephanie Date 2010-07-30 02:05 H**d*n* 1.03a Finds rb1 in 46 seconds on my slow laptop:

New game
1Q6/8/8/2pK3p/8/5N1q/1P4pp/5Rbk w - - 0 1

For what it's worth, stockfish 1.8 doesn't get this one after depth 46; it's stuck at score 0.00, 845mN. But, it did get the one in RH's other post.

[Robert Houdart]

That's because the original source code is screwed up - or shall we say, backwards. For some very strange unexplained reason. The booleans bzug and wzug are negated in evaluation.c and mobility.c. It's quite strange that they would publish this in this way. It literally turns OFF the code. Just switch the declarations to false and the conditions to true. Maybe they were testing things and uploaded a test version. Here's the diff for evaluation.c and then mobility.c below that, which will switch the TRUE/FALSE on bzug and wzug.

OK, you're playing with a very different compile than the original version. You could/should have mentioned this before, as your changes are far from trivial.
In fact, in my opinion the original code is correct. Your changes in the soure code effectively disable null move in nearly every position, except the positions without a piece move. This will obviously make your version perfect for zugzwang situations, but it should have a significant impact on the playing strength. Does your version go to the same depth and play with the same strength as the original version?

Based on my cursory analysis of the original IH-53 source code, I don't see how it could be able to handle the zugzwang positions that were discussed in this thread.

Robert

[benstoker]

That's because the original source code is screwed up - or shall we say, backwards. For some very strange unexplained reason. The booleans bzug and wzug are negated in evaluation.c and mobility.c. It's quite strange that they would publish this in this way. It literally turns OFF the code. Just switch the declarations to false and the conditions to true. Maybe they were testing things and uploaded a test version. Here's the diff for evaluation.c and then mobility.c below that, which will switch the TRUE/FALSE on bzug and wzug.

OK, you're playing with a very different compile than the original version. You could/should have mentioned this before, as your changes are far from trivial.
In fact, in my opinion the original code is correct. Your changes in the soure code effectively disable null move in nearly every position, except the positions without a piece move. This will obviously make your version perfect for zugzwang situations, but it should have a significant impact on the playing strength. Does your version go to the same depth and play with the same strength as the original version?

Based on my cursory analysis of the original IH-53 source code, I don't see how it could be able to handle the zugzwang positions that were discussed in this thread.

Robert

I see these changes as quite trivial.

Here's a clearer code quote. All that was changed is TRUE to FALSE and FALSE to TRUE. I am no programmer, but why would you start by assigning TRUE to wzug/bzug, and then if you detect zz, change the values to FALSE?! In any event, this switch results in ih finding the zz. It obviously works for zz. I have no idea what it does to the actual strength of the program. But, I will try to run some A vs. A' games tonight and see what happens. Can't you also compile a 53 with these simple changes and run out a few games against Houdini or whatever? [Does the windoze world use the diff and patch utilities? ]

Is this not a bit odd to add so-called zugzwang detection code that absolutely does NOT work because these boolean values are switched? If the tests result in lower ELO, then it is way beyond me what's going on. Maybe Vincent is right and NASA is behind all of this.

Code: Select all

#ifdef ZUGZWANG_DETECT
  boolean wzug = FALSE, bzug = FALSE;  // Original is TRUE
#endif

 * * * 

#ifdef ZUGZWANG_DETECT
       if (A & ~wBitboardOcc)
        wzug = TRUE;  // Original is FALSE
#endif

 * * * 

[BB+]

This post was deleted in the Rybka forum.Probably it was moved to the "secret subforum" --- I can't say I'm surprised really.

[Robert Houdart]

I see these changes as quite trivial.

Here's a clearer code quote. All that was changed is TRUE to FALSE and FALSE to TRUE. I am no programmer, but why would you start by assigning TRUE to wzug/bzug, and then if you detect zz, change the values to FALSE?! In any event, this switch results in ih finding the zz. It obviously works for zz. I have no idea what it does to the actual strength of the program. But, I will try to run some A vs. A' games tonight and see what happens. Can't you also compile a 53 with these simple changes and run out a few games against Houdini or whatever? [Does the windoze world use the diff and patch utilities? ]

Is this not a bit odd to add so-called zugzwang detection code that absolutely does NOT work because these boolean values are switched? If the tests result in lower ELO, then it is way beyond me what's going on. Maybe Vincent is right and NASA is behind all of this.

Your change is *far* from trivial.

You switch the value of a variable that decides whether null moves are allowed in the position. In your version null moves are mostly DISABLED, which works well in zugzwang positions, but at the price of severely disabling the engine in normal play. Null moves are vital for any modern chess engine, disabling them may cost you 50 to 100 Elo.

The intent of the original code is that most of the time null moves are ENABLED, except when one of the sides doesn't have a piece move. This is not a very successful way of detecting zugzwang positions, as none of the positions discussed in this thread fall in this category.

Like I said above, Houdini 1.03a uses a more sophisticated approach for detecting zugzwang positions.

Robert

[benstoker]

I see these changes as quite trivial.

Here's a clearer code quote. All that was changed is TRUE to FALSE and FALSE to TRUE. I am no programmer, but why would you start by assigning TRUE to wzug/bzug, and then if you detect zz, change the values to FALSE?! In any event, this switch results in ih finding the zz. It obviously works for zz. I have no idea what it does to the actual strength of the program. But, I will try to run some A vs. A' games tonight and see what happens. Can't you also compile a 53 with these simple changes and run out a few games against Houdini or whatever? [Does the windoze world use the diff and patch utilities? ]

Is this not a bit odd to add so-called zugzwang detection code that absolutely does NOT work because these boolean values are switched? If the tests result in lower ELO, then it is way beyond me what's going on. Maybe Vincent is right and NASA is behind all of this.

Your change is *far* from trivial.

You switch the value of a variable that decides whether null moves are allowed in the position. In your version null moves are mostly DISABLED, which works well in zugzwang positions, but at the price of severely disabling the engine in normal play. Null moves are vital for any modern chess engine, disabling them may cost you 50 to 100 Elo.

The intent of the original code is that most of the time null moves are ENABLED, except when one of the sides doesn't have a piece move. This is not a very successful way of detecting zugzwang positions, as none of the positions discussed in this thread fall in this category.

Like I said above, Houdini 1.03a uses a more sophisticated approach for detecting zugzwang positions.

Robert

Help me understand what the heck is going on. It's possible to switch off ALL zz detection code as follows (in RobboLito.h):

Code: Select all

// from RobboLito.h v. 53 
#if 1
#define MAGIC_BITBOARDS TRUE
#define ZUGZWANG_DETECT FALSE  //Original is TRUE
#endif

Since this would bypass entirely all of the bzug/wzug=false/true, etc. code in evalulation.c and mobility.c, then why does that not also kill null moves similar to my hack job switching of the the false/true? To my naive, non-chess-engine-programmer-limited-understanding mind, following your logic, taking this code that is necessary for null moves (as you say) completely out would appear to kill the null move algorithm. Therefore, there must be something more fundamental going on that I don't grasp.

Also, can you explain more why my true-false-switheroo hack job causes 53 to detect all 3 of the above zugzwang positions by killing null moves? In other words, why would an engine that has a retarded null move function somehow be better at detecting zz positions and yet be severely diminished in normal play? What is it about null move that blinds an engine's detection of zz?

Is there perhaps a set of conditions that could properly switch on my zz hack job only when and if it appears a zz position is likely? Such as what if you just switch on this hack job code when the total pieces on the board < 6, etc.? Would diminishing null move in the endgame damage the engine also?

[Robert Houdart]

Since this would bypass entirely all of the bzug/wzug=false/true, etc. code in evalulation.c and mobility.c, then why does that not also kill null moves similar to my hack job switching of the the false/true? To my naive, non-chess-engine-programmer-limited-understanding mind, following your logic, taking this code that is necessary for null moves (as you say) completely out would appear to kill the null move algorithm. Therefore, there must be something more fundamental going on that I don't grasp.

By disabling ZUGZWANG_DETECT, you also remove the code that disables the null move, see e.g. in mobility.c:

Code: Select all

#ifdef ZUGZWANG_DETECT
  if (wzug)
    POSITION->DYN->flags &= ~2;
  if (bzug)
    POSITION->DYN->flags &= ~1;
#endif

The two bits in the POSITION->DYN->flags value are used in the search algorithm to decide whether a null move can be attempted or not.

Also, can you explain more why my true-false-switheroo hack job causes 53 to detect all 3 of the above zugzwang positions by killing null moves? In other words, why would an engine that has a retarded null move function somehow be better at detecting zz positions and yet be severely diminished in normal play? What is it about null move that blinds an engine's detection of zz?

A null move basically means that one side (say White) skips a move and lets the opponent (Black) play two moves in a row. If Black doesn't achieve a good score even by playing two consecutive moves, it can be assumed that this variation is safe for White. This assumption is only correct if White is not in zugzwang, because in that case a White move between the two Black moves would be favorable for Black.
See Wikipedia for a more detailed explanation: http://en.wikipedia.org/wiki/Null-move_heuristic .

Is there perhaps a set of conditions that could properly switch on my zz hack job only when and if it appears a zz position is likely? Such as what if you just switch on this hack job code when the total pieces on the board < 6, etc.? Would diminishing null move in the endgame damage the engine also?

In most endgames null moves improve the strength as they allow the engine to go deeper much faster.
Only in very late endgames (pawn endgames or one piece per side) will most programs switch off null moves.

Robert

[benstoker]

Since this would bypass entirely all of the bzug/wzug=false/true, etc. code in evalulation.c and mobility.c, then why does that not also kill null moves similar to my hack job switching of the the false/true? To my naive, non-chess-engine-programmer-limited-understanding mind, following your logic, taking this code that is necessary for null moves (as you say) completely out would appear to kill the null move algorithm. Therefore, there must be something more fundamental going on that I don't grasp.

By disabling ZUGZWANG_DETECT, you also remove the code that disables the null move, see e.g. in mobility.c:

Code: Select all

#ifdef ZUGZWANG_DETECT
  if (wzug)
    POSITION->DYN->flags &= ~2;
  if (bzug)
    POSITION->DYN->flags &= ~1;
#endif

The two bits in the POSITION->DYN->flags value are used in the search algorithm to decide whether a null move can be attempted or not.

Also, can you explain more why my true-false-switheroo hack job causes 53 to detect all 3 of the above zugzwang positions by killing null moves? In other words, why would an engine that has a retarded null move function somehow be better at detecting zz positions and yet be severely diminished in normal play? What is it about null move that blinds an engine's detection of zz?

A null move basically means that one side (say White) skips a move and lets the opponent (Black) play two moves in a row. If Black doesn't achieve a good score even by playing two consecutive moves, it can be assumed that this variation is safe for White. This assumption is only correct if White is not in zugzwang, because in that case a White move between the two Black moves would be favorable for Black.
See Wikipedia for a more detailed explanation: http://en.wikipedia.org/wiki/Null-move_heuristic .

Is there perhaps a set of conditions that could properly switch on my zz hack job only when and if it appears a zz position is likely? Such as what if you just switch on this hack job code when the total pieces on the board < 6, etc.? Would diminishing null move in the endgame damage the engine also?

In most endgames null moves improve the strength as they allow the engine to go deeper much faster.
Only in very late endgames (pawn endgames or one piece per side) will most programs switch off null moves.

Robert

Here's some 200 game results

Code: Select all

200 game(s) loaded, 0 game(s) with unknown result ignored.
Rank Name          Elo    +    - games score oppo. draws 
   1 ih53orig        0   37   37   200   53%   -17   59% 
   2 ih53zughack   -17   37   37   200   47%     0   59%