Page 1 of 1
Endgame testing
Posted: Wed Feb 08, 2012 9:09 pm
by gaard
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Rybka 4.1 16 23 23 300 54% -5 56%
2 Komodo 3 0 23 23 300 50% 0 59%
3 Houdini 2.0c -3 22 22 300 49% 1 64%
4 Critter 1.4 -13 23 23 300 47% 4 60%
threads: 1
ponder: off
time control: 2/4"
64-bit and SSE4.2 versions when available
Nalimov and Gaviota 5 piece EGTB's
Nalimov and Gaviota cache: 256MB
Core i5 2.4GHz
I used the -repeat switch for the positions to be repeated with colors swapped, but apparently cutechess ignores this if the tournament is a round robin. Is this the expected behavior?
endgame suite:
http://dl.dropbox.com/u/11904592/EG_MSb.pgn
games:
http://dl.dropbox.com/u/11904592/eg.pgn
Re: Endgame testing
Posted: Thu Feb 09, 2012 3:33 am
by kingliveson
This is very interesting as I thought the order would've been:
1. Houdini
2. Critter
3. Rybka
4. Komodo
Re: Endgame testing
Posted: Thu Feb 09, 2012 5:39 am
by gaard
kingliveson wrote:This is very interesting as I thought the order would've been:
1. Houdini
2. Critter
3. Rybka
4. Komodo
My earlier testing had Houdini at the top. Who knows what it will look like with another 600 (or more) games added on Friday.
Re: Endgame testing
Posted: Thu Feb 09, 2012 1:46 pm
by ernest
kingliveson wrote:I thought the order would've been:
If you look at the error bars, you can see that all 4 programs are in the same ballpark...
Re: Endgame testing
Posted: Thu Feb 09, 2012 5:13 pm
by gaard
kingliveson wrote:This is very interesting as I thought the order would've been:
1. Houdini
2. Critter
3. Rybka
4. Komodo
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Rybka 4.1 11 17 17 553 53% -4 58%
2 Houdini 2.0c 4 17 17 550 51% -1 65%
3 Komodo 3 -2 17 17 551 50% 1 60%
4 Critter 1.4 -13 17 17 552 47% 4 62%
Still more to come...
links remain the same
Re: Endgame testing
Posted: Tue Feb 21, 2012 11:59 am
by Ilari Pihlajisto
gaard wrote:I used the -repeat switch for the positions to be repeated with colors swapped, but apparently cutechess ignores this if the tournament is a round robin. Is this the expected behavior?
I guess it's finally time to post my first message on this board. Cutechess-cli does indeed ignore the "-repeat" option in tournament mode if only one game per encounter is played. But if you set "-games" to 2 (or any even number), each opening is repeated once and the colors are swapped.
Re: Endgame testing
Posted: Wed Feb 22, 2012 12:10 am
by gaard
Ilari Pihlajisto wrote:gaard wrote:I used the -repeat switch for the positions to be repeated with colors swapped, but apparently cutechess ignores this if the tournament is a round robin. Is this the expected behavior?
I guess it's finally time to post my first message on this board. Cutechess-cli does indeed ignore the "-repeat" option in tournament mode if only one game per encounter is played. But if you set "-games" to 2 (or any even number), each opening is repeated once and the colors are swapped.
I am only using the -rounds switch with an even number and not specifying the number of games. From what I read here, I should be using -games 2 for repeated positions with colors swapped in one round?
Re: Endgame testing
Posted: Wed Feb 22, 2012 10:38 am
by Ilari Pihlajisto
gaard wrote:Ilari Pihlajisto wrote:gaard wrote:I used the -repeat switch for the positions to be repeated with colors swapped, but apparently cutechess ignores this if the tournament is a round robin. Is this the expected behavior?
I guess it's finally time to post my first message on this board. Cutechess-cli does indeed ignore the "-repeat" option in tournament mode if only one game per encounter is played. But if you set "-games" to 2 (or any even number), each opening is repeated once and the colors are swapped.
I am only using the -rounds switch with an even number and not specifying the number of games. From what I read here, I should be using -games 2 for repeated positions with colors swapped in one round?
Yes. "-rounds <n>" will just repeat the tournament <n> times, and "-games <n>" means that each encounter between any two engines consists of <n> games. You can of course combine "-rounds" and "-games".