Important IvanHoe Settings

kingliveson · Post by **kingliveson** » Thu Jul 15, 2010 4:00 am

1/3rd way through:

Time Management 5402  2010     
 	 		       1            2            3            4            5            6
1 IvanHoe 9.54b-TM11 x64  ***********  33.0 - 37.0  37.0 - 33.0  40.0 - 30.0  39.0 - 31.0  45.0 - 25.0  194.0/350   
2 IvanHoe 9.54b-TM01 x64  37.0 - 33.0  ***********  30.0 - 40.0  40.0 - 30.0  37.0 - 33.0  39.5 - 30.5  183.5/350   
3 IvanHoe 9.54b-TM10 x64  33.0 - 37.0  40.0 - 30.0  ***********  34.0 - 36.0  35.5 - 34.5  39.5 - 30.5  182.0/350   
4 Deep Rybka 4 x64        30.0 - 40.0  30.0 - 40.0  36.0 - 34.0  ***********  42.0 - 28.0  42.0 - 28.0  180.0/350   
5 IvanHoe 9.54b-TM00 x64  31.0 - 39.0  33.0 - 37.0  34.5 - 35.5  28.0 - 42.0  ***********  38.5 - 31.5  165.0/350   
6 Stockfish 1.8 JA 64bit  25.0 - 45.0  30.5 - 39.5  30.5 - 39.5  28.0 - 42.0  31.5 - 38.5  ***********  145.5/350

IvanHoe 9.54b-TM00 x64 + Neither Option
IvanHoe 9.54b-TM01 x64 + Option 2
IvanHoe 9.54b-TM10 x64 + Option 1
IvanHoe 9.54b-TM11 x64 + Option 1 & 2

Option 1

Code: Select all

#if 0 /* about the 0.6 elo for the hasslings */ /* measure to UCI? */
  if (!mtg)
    {
      rat = ((double) (TIME + 1)) / ((double) (OPP_TIME + 1));
      if (rat > 2.0)
	rat = 2.0;
      if (rat < 0.5)
	rat = 0.5;
      if (rat > 1.0)
	{
	  mult = ((int) (1024.0 * rat)) - 1024;
	  DESIRED_TIME += (mult * DESIRED_TIME) >> (10 + 2); /* max 1/4 */ 
	}
      else
	{
	  mult = ((int) (1024.0 / rat)) - 1024;
	  DESIRED_TIME -= (mult * DESIRED_TIME) >> (10 + 2); /* max 1/4 */ 
	}
    }
#endif

Option 2

Code: Select all

#if 0
  if (ROOT_DEPTH && ROOT_SCORE < -50)
    DESIRED_TIME += DESIRED_TIME >> 3;
  if (ROOT_DEPTH && ROOT_SCORE < -100)
    DESIRED_TIME += DESIRED_TIME >> 3;
#endif

The code snippets are additional time management functions -- disabled by default in control.c.

kingliveson · Post by **kingliveson** » Sat Jul 17, 2010 1:15 am

2/3rd way through:

Time Management 5402  2010
                               1           2           3           4           5           6
1 IvanHoe 9.54b-TM11 x64  *********** 72.0 - 71.0 74.0 - 69.0 70.5 - 72.5 75.5 - 67.5 91.5 - 51.5  383.5/715   
2 Deep Rybka 4 x64        71.0 - 72.0 *********** 74.5 - 68.5 68.5 - 74.5 79.5 - 63.5 87.0 - 56.0  380.5/715   
3 IvanHoe 9.54b-TM10 x64  69.0 - 74.0 68.5 - 74.5 *********** 76.5 - 66.5 76.0 - 67.0 82.0 - 61.0  372.0/715   
4 IvanHoe 9.54b-TM01 x64  72.5 - 70.5 74.5 - 68.5 66.5 - 76.5 *********** 76.5 - 66.5 80.0 - 63.0  370.0/715   
5 IvanHoe 9.54b-TM00 x64  67.5 - 75.5 63.5 - 79.5 67.0 - 76.0 66.5 - 76.5 *********** 81.0 - 62.0  345.5/715   
6 Stockfish 1.8 JA 64bit  51.5 - 91.5 56.0 - 87.0 61.0 - 82.0 63.0 - 80.0 62.0 - 81.0 ***********  293.5/715

IvanHoe 9.54b-TM00 x64 + Neither Option
IvanHoe 9.54b-TM01 x64 + Option 2
IvanHoe 9.54b-TM10 x64 + Option 1
IvanHoe 9.54b-TM11 x64 + Option 1 & 2

Option 1

Code: Select all

#if 0 /* about the 0.6 elo for the hasslings */ /* measure to UCI? */
  if (!mtg)
    {
      rat = ((double) (TIME + 1)) / ((double) (OPP_TIME + 1));
      if (rat > 2.0)
	rat = 2.0;
      if (rat < 0.5)
	rat = 0.5;
      if (rat > 1.0)
	{
	  mult = ((int) (1024.0 * rat)) - 1024;
	  DESIRED_TIME += (mult * DESIRED_TIME) >> (10 + 2); /* max 1/4 */ 
	}
      else
	{
	  mult = ((int) (1024.0 / rat)) - 1024;
	  DESIRED_TIME -= (mult * DESIRED_TIME) >> (10 + 2); /* max 1/4 */ 
	}
    }
#endif

Option 2

Code: Select all

#if 0
  if (ROOT_DEPTH && ROOT_SCORE < -50)
    DESIRED_TIME += DESIRED_TIME >> 3;
  if (ROOT_DEPTH && ROOT_SCORE < -100)
    DESIRED_TIME += DESIRED_TIME >> 3;
#endif

The code snippets are additional time management functions -- disabled by default in control.c.

kingliveson · Post by **kingliveson** » Sun Jul 18, 2010 2:33 pm

3000 Games Completed

Time Management 5402  2010	
				1	       2	      3	             4	           5	          6	
1 IvanHoe 9.54b-TM11 x64  *************  103.5 - 96.5   100.0 - 100.0  100.5 - 99.5   104.0 - 96.0   124.0 - 76.0    532.0/1000
2 Deep Rybka 4 x64	  96.5 - 103.5   *************  107.0 - 93.0   100.0 - 99.0   105.5 - 94.5   120.5 - 79.5    529.5/999
3 IvanHoe 9.54b-TM10 x64  100.0 - 100.0  93.0 - 107.0   *************  102.0 - 98.0   105.5 - 94.5   113.0 - 87.0    513.5/1000
4 IvanHoe 9.54b-TM01 x64  99.5 - 100.5   99.0 - 100.0   98.0 - 102.0   *************  101.0 - 99.0   115.0 - 85.0    512.5/999
5 IvanHoe 9.54b-TM00 x64  96.0 - 104.0   94.5 - 105.5   94.5 - 105.5   99.0 - 101.0   *************  113.5 - 86.5    497.5/1000
6 Stockfish 1.8 JA 64bit  76.0 - 124.0   79.5 - 120.5   87.0 - 113.0   85.0 - 115.0   86.5 - 113.5   *************   414.0/1000

IvanHoe 9.54b-TM00 x64 + Neither Option
IvanHoe 9.54b-TM01 x64 + Option 2
IvanHoe 9.54b-TM10 x64 + Option 1
IvanHoe 9.54b-TM11 x64 + Option 1 & 2

Option 1

Code: Select all

#if 0 /* about the 0.6 elo for the hasslings */ /* measure to UCI? */
  if (!mtg)
    {
      rat = ((double) (TIME + 1)) / ((double) (OPP_TIME + 1));
      if (rat > 2.0)
	rat = 2.0;
      if (rat < 0.5)
	rat = 0.5;
      if (rat > 1.0)
	{
	  mult = ((int) (1024.0 * rat)) - 1024;
	  DESIRED_TIME += (mult * DESIRED_TIME) >> (10 + 2); /* max 1/4 */ 
	}
      else
	{
	  mult = ((int) (1024.0 / rat)) - 1024;
	  DESIRED_TIME -= (mult * DESIRED_TIME) >> (10 + 2); /* max 1/4 */ 
	}
    }
#endif

Option 2

Code: Select all

#if 0
  if (ROOT_DEPTH && ROOT_SCORE < -50)
    DESIRED_TIME += DESIRED_TIME >> 3;
  if (ROOT_DEPTH && ROOT_SCORE < -100)
    DESIRED_TIME += DESIRED_TIME >> 3;
#endif

The code snippets are additional time management functions -- disabled by default in control.c.

Code: Select all

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 IvanHoe 9.54b-TM11 x64         : 3019   14  14  1000    53.2 %   2996   55.4 %
  2 Deep Rybka 4 x64               : 3017   16  16   999    53.0 %   2997   42.7 %
  3 IvanHoe 9.54b-TM10 x64         : 3008   14  14  1000    51.3 %   2999   56.5 %
  4 IvanHoe 9.54b-TM01 x64         : 3008   14  14   999    51.3 %   2999   57.6 %
  5 IvanHoe 9.54b-TM00 x64         : 2999   14  14  1000    49.8 %   3000   57.9 %
  6 Stockfish 1.8 JA 64bit         : 2950   17  17  1000    41.4 %   3010   37.2 %

Code: Select all

Individual statistics:

1 IvanHoe 9.54b-TM11 x64    : 3019  1000 (+255,=554,-191), 53.2 %

Deep Rybka 4 x64              : 200 (+ 55,= 97,- 48), 51.7 %
Stockfish 1.8 JA 64bit        : 200 (+ 92,= 64,- 44), 62.0 %
IvanHoe 9.54b-TM00 x64        : 200 (+ 40,=128,- 32), 52.0 %
IvanHoe 9.54b-TM01 x64        : 200 (+ 33,=135,- 32), 50.2 %
IvanHoe 9.54b-TM10 x64        : 200 (+ 35,=130,- 35), 50.0 %

2 Deep Rybka 4 x64          : 3017  999 (+316,=427,-256), 53.0 %

Stockfish 1.8 JA 64bit        : 200 (+ 91,= 59,- 50), 60.2 %
IvanHoe 9.54b-TM00 x64        : 200 (+ 59,= 93,- 48), 52.8 %
IvanHoe 9.54b-TM11 x64        : 200 (+ 48,= 97,- 55), 48.2 %
IvanHoe 9.54b-TM01 x64        : 199 (+ 59,= 82,- 58), 50.3 %
IvanHoe 9.54b-TM10 x64        : 200 (+ 59,= 96,- 45), 53.5 %

3 IvanHoe 9.54b-TM10 x64    : 3008  1000 (+231,=565,-204), 51.4 %

Deep Rybka 4 x64              : 200 (+ 45,= 96,- 59), 46.5 %
Stockfish 1.8 JA 64bit        : 200 (+ 74,= 78,- 48), 56.5 %
IvanHoe 9.54b-TM00 x64        : 200 (+ 38,=135,- 27), 52.8 %
IvanHoe 9.54b-TM11 x64        : 200 (+ 35,=130,- 35), 50.0 %
IvanHoe 9.54b-TM01 x64        : 200 (+ 39,=126,- 35), 51.0 %

4 IvanHoe 9.54b-TM01 x64    : 3008  999 (+225,=575,-199), 51.3 %

Deep Rybka 4 x64              : 199 (+ 58,= 82,- 59), 49.7 %
Stockfish 1.8 JA 64bit        : 200 (+ 70,= 90,- 40), 57.5 %
IvanHoe 9.54b-TM00 x64        : 200 (+ 30,=142,- 28), 50.5 %
IvanHoe 9.54b-TM11 x64        : 200 (+ 32,=135,- 33), 49.8 %
IvanHoe 9.54b-TM10 x64        : 200 (+ 35,=126,- 39), 49.0 %

5 IvanHoe 9.54b-TM00 x64    : 2999  1000 (+208,=579,-213), 49.8 %

Deep Rybka 4 x64              : 200 (+ 48,= 93,- 59), 47.2 %
Stockfish 1.8 JA 64bit        : 200 (+ 73,= 81,- 46), 56.8 %
IvanHoe 9.54b-TM11 x64        : 200 (+ 32,=128,- 40), 48.0 %
IvanHoe 9.54b-TM01 x64        : 200 (+ 28,=142,- 30), 49.5 %
IvanHoe 9.54b-TM10 x64        : 200 (+ 27,=135,- 38), 47.2 %

6 Stockfish 1.8 JA 64bit    : 2950  1000 (+228,=372,-400), 41.4 %

Deep Rybka 4 x64              : 200 (+ 50,= 59,- 91), 39.8 %
IvanHoe 9.54b-TM00 x64        : 200 (+ 46,= 81,- 73), 43.2 %
IvanHoe 9.54b-TM11 x64        : 200 (+ 44,= 64,- 92), 38.0 %
IvanHoe 9.54b-TM01 x64        : 200 (+ 40,= 90,- 70), 42.5 %
IvanHoe 9.54b-TM10 x64        : 200 (+ 48,= 78,- 74), 43.5 %

Code: Select all

Games        :   2999 (finished)

White Wins   :    849 (28.3 %)
Black Wins   :    614 (20.5 %)
Draws        :   1536 (51.2 %)
Unfinished   :      1

White Perf.  : 53.9 %
Black Perf.  : 46.1 %

ECO A =    421 Games (14.0 %)
ECO B =    942 Games (31.4 %)
ECO C =    543 Games (18.1 %)
ECO D =    619 Games (20.6 %)
ECO E =    474 Games (15.8 %)

Time Losses

IvanHoe 9.54b-TM00 x64: 18 
IvanHoe 9.54b-TM01 x64: 20
IvanHoe 9.54b-TM10 x64: 15
IvanHoe 9.54b-TM11 x64: 7

† 100 Positions (test suite)
† EGTB off
† Ponder off
† 128 MB Hash
† 8 MB Pawn Hash
† Athlon 64 X2 6000+ @ 3100 MHz
† 2 Cores
† 1+0 Time Control

Match data available: http://chess.cygnitec.com/pgn/

BB+ · Post by **BB+** » Mon Jul 19, 2010 6:11 am

Code: Select all

  1 IvanHoe 9.54b-TM11 x64         : 3019   14  14  1000    53.2 %   2996   55.4 %
  2 Deep Rybka 4 x64               : 3017   16  16   999    53.0 %   2997   42.7 %
  3 IvanHoe 9.54b-TM10 x64         : 3008   14  14  1000    51.3 %   2999   56.5 %
  4 IvanHoe 9.54b-TM01 x64         : 3008   14  14   999    51.3 %   2999   57.6 %
  5 IvanHoe 9.54b-TM00 x64         : 2999   14  14  1000    49.8 %   3000   57.9 %
  6 Stockfish 1.8 JA 64bit         : 2950   17  17  1000    41.4 %   3010   37.2 %

So in other words, both the adjustments give a slight edge, both within the error margin.

kingliveson · Post by **kingliveson** » Mon Jul 19, 2010 9:22 pm

BB+ wrote:

Code: Select all

  1 IvanHoe 9.54b-TM11 x64         : 3019   14  14  1000    53.2 %   2996   55.4 %
  2 Deep Rybka 4 x64               : 3017   16  16   999    53.0 %   2997   42.7 %
  3 IvanHoe 9.54b-TM10 x64         : 3008   14  14  1000    51.3 %   2999   56.5 %
  4 IvanHoe 9.54b-TM01 x64         : 3008   14  14   999    51.3 %   2999   57.6 %
  5 IvanHoe 9.54b-TM00 x64         : 2999   14  14  1000    49.8 %   3000   57.9 %
  6 Stockfish 1.8 JA 64bit         : 2950   17  17  1000    41.4 %   3010   37.2 %

So in other words, both the adjustments give a slight edge, both within the error margin.

The plan was 7500 games to reduce error bar as possible, but got lazy. Looking at the individual stats, when separated by number of wins, draws, losses, kind of an interesting outcome. all in all, there is probably good chance adjustment 1 and 2 combined give better performance. The time management still needs a little work. Stockfish did not lose a single game on time. Rybka was using its default settings so Buffer Time was 3 seconds.

kingliveson · Post by **kingliveson** » Sun Jul 25, 2010 8:55 pm

Something interesting -- assuming results below come from 30,000 games and the ratio is equivalent, how do you go about picking the best IvanHoe version?

1 Deep Rybka 4 SSE42 x64  ***********  22.5 - 13.5  17.5 - 18.5   40.0/72
2 IvanHoe 9.53b x64 A     13.5 - 22.5  ***********  21.0 - 15.0   34.5/72
3 IvanHoe 9.53b x64 B     18.5 - 17.5  15.0 - 21.0  ***********   33.5/72

Robert Houdart · Post by **Robert Houdart** » Sun Jul 25, 2010 9:50 pm

kingliveson wrote:Something interesting -- assuming results below come from 30,000 games and the ratio is equivalent, how do you go about picking the best IvanHoe version?

By playing matches against many other opponents, similar to the rating lists.
There is no way you can pick "the best" Ivanhoe by just playing against Rybka 4.

Robert

kingliveson · Post by **kingliveson** » Sun Jul 25, 2010 10:54 pm

Robert Houdart wrote:
kingliveson wrote:Something interesting -- assuming results below come from 30,000 games and the ratio is equivalent, how do you go about picking the best IvanHoe version?
By playing matches against many other opponents, similar to the rating lists.
There is no way you can pick "the best" Ivanhoe by just playing against Rybka 4.

Robert

Appreciate the input. I think it's still possible to end up with similar scenario even with variety of engines included in the test -- where version A is out-performing version B, but version B is out-performing top engine on the list, and top engine is out-performing version A.

1 Deep Rybka 4 SSE42 x64  ***********  28.0 - 22.0  25.0 - 25.0  53.0/100
2 IvanHoe 9.53b x64 A     22.0 - 28.0  ***********  30.0 - 20.0  52.0/100
3 IvanHoe 9.53b x64 B     25.0 - 25.0  20.0 - 30.0  ***********  45.0/100

Robert Houdart · Post by **Robert Houdart** » Sun Jul 25, 2010 11:21 pm

kingliveson wrote:Appreciate the input. I think it's still possible to end up with similar scenario even with variety of engines included in the test -- where version A is out-performing version B, but version B is out-performing top engine on the list, and top engine is out-performing version A.

True, but your "A>B, B>C, C>A" result is just a demonstration of how much random variation exists in the outcome of engine matches.
Even without this amusing quirk you wouldn't be able to draw any meaningful conclusion from only 150 games between 3 engines that are very close in strength.

Robert

kingliveson · Post by **kingliveson** » Tue Jul 27, 2010 9:41 pm

Pawn Hash Matters

Pawn Hash Battle  2010
                                  1            2            3            4                                
1 Deep Rybka 4 SSE42 x64     ***********  32.5 - 33.5  36.0 - 31.0  40.5 - 26.5   109.0/200
2 IvanHoe 9.53b x64 PH 1/8   33.5 - 32.5  ***********  38.0 - 29.0  37.0 - 30.0   108.5/200
3 IvanHoe 9.53b x64 PH 1/4   31.0 - 36.0  29.0 - 38.0  ***********  32.5 - 33.5   92.5/200
4 IvanHoe 9.53b x64 PH 1/16  26.5 - 40.5  30.0 - 37.0  33.5 - 32.5  ***********   90.0/200

† 50 Positions (test suite)
† EGTB Off
† Ponder Off
† 128 MB Hash
† Phenom II 940
† 4 Cores
† 1+0 Time Control

Rybka
TC Buffer 2
TC Normal Move Time 72
TC Max Move Time 115

IvanHoe
PH 1/16: 8 MB Pawn Hash
PH 1/8: 16 MB Pawn Hash
PH 1/4: 32 MB Pawn Hash

Game data available: http://chess.cygnitec.com/pgn/

Tournament ended before completion -- setting Pawn Hash to 1/8th Hash looks OK to me.

OpenChess

OpenChess

Important IvanHoe Settings

Re: Important IvanHoe Settings

Re: Important IvanHoe Settings

Re: Important IvanHoe Settings

Re: Important IvanHoe Settings

Re: Important IvanHoe Settings

Re: Important IvanHoe Settings

Re: Important IvanHoe Settings

Re: Important IvanHoe Settings

Re: Important IvanHoe Settings

Re: Important IvanHoe Settings