Endgame testing

[gaard]

Code: Select all

Rank Name           Elo    +    - games score oppo. draws 
   1 Rybka 4.1       16   23   23   300   54%    -5   56% 
   2 Komodo 3         0   23   23   300   50%     0   59% 
   3 Houdini 2.0c    -3   22   22   300   49%     1   64% 
   4 Critter 1.4    -13   23   23   300   47%     4   60% 

threads: 1
ponder: off
time control: 2/4"

64-bit and SSE4.2 versions when available
Nalimov and Gaviota 5 piece EGTB's
Nalimov and Gaviota cache: 256MB
Core i5 2.4GHz

I used the -repeat switch for the positions to be repeated with colors swapped, but apparently cutechess ignores this if the tournament is a round robin. Is this the expected behavior?

endgame suite: http://dl.dropbox.com/u/11904592/EG_MSb.pgn
games: http://dl.dropbox.com/u/11904592/eg.pgn

[kingliveson]

This is very interesting as I thought the order would've been:

1. Houdini
2. Critter
3. Rybka
4. Komodo

[gaard]

This is very interesting as I thought the order would've been:

1. Houdini
2. Critter
3. Rybka
4. Komodo

My earlier testing had Houdini at the top. Who knows what it will look like with another 600 (or more) games added on Friday.

[ernest]

I thought the order would've been:

If you look at the error bars, you can see that all 4 programs are in the same ballpark...

[gaard]

This is very interesting as I thought the order would've been:

1. Houdini
2. Critter
3. Rybka
4. Komodo

Code: Select all

Rank Name           Elo    +    - games score oppo. draws 
   1 Rybka 4.1       11   17   17   553   53%    -4   58% 
   2 Houdini 2.0c     4   17   17   550   51%    -1   65% 
   3 Komodo 3        -2   17   17   551   50%     1   60% 
   4 Critter 1.4    -13   17   17   552   47%     4   62% 

Still more to come...

links remain the same

[Ilari Pihlajisto]

I used the -repeat switch for the positions to be repeated with colors swapped, but apparently cutechess ignores this if the tournament is a round robin. Is this the expected behavior?

I guess it's finally time to post my first message on this board. Cutechess-cli does indeed ignore the "-repeat" option in tournament mode if only one game per encounter is played. But if you set "-games" to 2 (or any even number), each opening is repeated once and the colors are swapped.

[gaard]

I used the -repeat switch for the positions to be repeated with colors swapped, but apparently cutechess ignores this if the tournament is a round robin. Is this the expected behavior?

I guess it's finally time to post my first message on this board. Cutechess-cli does indeed ignore the "-repeat" option in tournament mode if only one game per encounter is played. But if you set "-games" to 2 (or any even number), each opening is repeated once and the colors are swapped.

I am only using the -rounds switch with an even number and not specifying the number of games. From what I read here, I should be using -games 2 for repeated positions with colors swapped in one round?

[Ilari Pihlajisto]

I used the -repeat switch for the positions to be repeated with colors swapped, but apparently cutechess ignores this if the tournament is a round robin. Is this the expected behavior?

I guess it's finally time to post my first message on this board. Cutechess-cli does indeed ignore the "-repeat" option in tournament mode if only one game per encounter is played. But if you set "-games" to 2 (or any even number), each opening is repeated once and the colors are swapped.

I am only using the -rounds switch with an even number and not specifying the number of games. From what I read here, I should be using -games 2 for repeated positions with colors swapped in one round?

Yes. "-rounds <n>" will just repeat the tournament <n> times, and "-games <n>" means that each encounter between any two engines consists of <n> games. You can of course combine "-rounds" and "-games".