Page 1 of 2
Android UCI engine test tournament
Posted: Sun May 15, 2011 7:39 am
by AartBik
I am generalizing the engine match feature in
Chess for Android into an engine tournament feature and, as a test, ran a tournament between a few of the Android UCI engines recently released on
Jim Ablett's Chess Projects Website, as well as the built-in Java engine (using the fast-move setting, 32MB hash, and Nalimov, Gaviota, Scorpio, Robbobases endgame tablebases on SD card). More elaborate tournament results to follow (new engines still welcome!).
Code: Select all
1 2 3 4 5 6
1 Stockfish 2.1.1 JA ********** 11110½11½1 111½111111 1111111111 1111111111 1111111111 47.5/50
2 Komodo32 1.3 JA 00001½00½0 ********** 1111111111 1111111111 1111111111 1111111111 42.0/50
3 Rotor 0.4 000½000000 0000000000 ********** 1111011111 1111111111 1111111111 29.5/50
4 Diablo 0.5.1 0000000000 0000000000 0000100000 ********** 1111111111 1111111111 21.0/50
5 Chess for Android 0000000000 0000000000 0000000000 0000000000 ********** 1111½111½1 9.0/50
6 Simplex 0.9.7 rev 180 0000000000 0000000000 0000000000 0000000000 0000½000½0 ********** 1.0/50
Re: Android UCI engine test tournament
Posted: Sun May 15, 2011 10:53 am
by thorstenczub
so we are only a few steps away from having an ARENA-like program done for chess under android.
wow !!
one of the strongest features of arena is, that you can edit the tournaments,
e.g. throw out an engine, replace it with another engine.
and NOW your results from the last run are not overwritten but
used for the NEW tournament.
this way you can enhance tournaments with new versions of a program.
e.g. you had a big tournament with 6 participants and komodo1.3.
now komodo1.4 is coming out, you put komodo in, and another engine
and run the REST of the games and you will see how komodo1.4 ranks in relation to the old komodo1.3 results in the tournament.
Re: Android UCI engine test tournament
Posted: Sun May 15, 2011 10:28 pm
by AartBik
thorstenczub wrote:one of the strongest features of arena is, that you can edit the tournaments,
e.g. throw out an engine, replace it with another engine.
Thorsten, always a few feature requests ahead of me
For now, I am finalizing the last details of the new tournament feature, such as continuing a tournament in progress later (useful if you need the device for something else, or when an UCI engine crashes).
Re: Android UCI engine test tournament
Posted: Sun May 15, 2011 11:20 pm
by AartBik
Below the results of a tournament between the different Android Stockfish versions and compilations (from
Jim Ablett and
myself).
Code: Select all
Chess for Android Tournament May 2011
1 2 3 4 5 6
1 Stockfish 2.0 ********** 01½11½01½0 0½1½0100½½ 010½½½½½½1 ½1011½½½½1 ½11001½½11 27.5/50
2 Stockfish 2.1 JA 10½00½10½1 ********** 1½½1½1½100 ½½11½00111 1½011½½010 1½00001½1½ 27.0/50
3 Stockfish 2.1.1 JA 1½0½1011½½ 0½½0½0½011 ********** ½½½1½10½½½ 0½0010½½½½ ½½1½101½½1 25.5/50
4 Stockfish 1.9 101½½½½½½0 ½½00½11000 ½½½0½01½½½ ********** 100½1½0½1½ 1½10½0111½ 24.5/50
5 Stockfish 2.1 ½0100½½½½0 0½100½½101 1½1101½½½½ 011½0½1½0½ ********** ½½010½0101 24.0/50
6 Stockfish 1.8 ½00110½½00 0½11110½0½ ½½0½010½½0 0½01½1000½ ½½101½1010 ********** 21.5/50
Re: Android UCI engine test tournament
Posted: Mon May 16, 2011 1:48 am
by AartBik
Test tournament with different Android versions of Miguel Ballicora's Gaviota and Michel Van den Bergh's GNU chess.
Code: Select all
1 2 3 4 5
1 GNU Chess 5.07.170.7b ********** 000½10011½ 11½1110110 1100½01001 0111101111 24.0/40
2 gaviota v0.80.0.107 111½01100½ ********** ½010110100 0100½11110 ½101000011 20.5/40
3 GNU Chess 5.07.153.3b 00½0001001 ½101001011 ********** 0101010½01 0111111½10 20.0/40
4 gaviota v0.82-beta2 0011½10110 1011½00001 1010101½10 ********** 00110001½0 19.0/40
5 gaviota v0.83 1000010000 ½010111100 1000000½01 11001110½1 ********** 16.5/40
Re: Android UCI engine test tournament
Posted: Mon May 16, 2011 4:39 am
by AartBik
Yet another test tournament with different Android versions of Jan Brouwer's Rotor and Ben-Hur Carlos Vieira Langoni Jr.'s RedQueen.
Code: Select all
1 2 3 4 5
1 Rotor 0.6a ********** ½111111011 1011111101 111½111111 11½1111111 35.5/40
2 RedQueen 0.9.8 ½000000100 ********** 1½01½½010½ 11½1½0110½ ½1½1½011½1 20.0/40
3 Rotor 0.4 JA 0100000010 0½10½½101½ ********** 1½0½010110 10011½½101 18.0/40
4 RedQueen 0.9.5 000½000000 00½0½1001½ 0½1½101001 ********** ½1101½10½½ 15.0/40
5 RedQueen 0.9.8 JA 00½0000000 ½0½0½100½0 01100½½010 ½0010½01½½ ********** 11.5/40
Re: Android UCI engine test tournament
Posted: Mon May 16, 2011 9:38 am
by thorstenczub
i see you have fun with these tournaments
)
the clou of the reuse of the old data is, that your tournament grows and grows,
and is suddenly a rating list instead of a small tournament.
all the old results were taken over to the new tournament.
if in the new tournament a new version of komodo gets a late entry,
there will be many komodo new versus other participants games.
but the result/game stockfish vs. fruit, which was played
in the old tournament, can be used without a replay because
its data can be used without changes for the new tournament.
this way you only have to play the new komodo games, and the tournament
is finished.
i appreciate that you implemented the feature to interrupt and later continue
the tournament.
thanks aart !!!
Re: Android UCI engine test tournament
Posted: Wed May 18, 2011 8:35 am
by AartBik
I conducted a full tournament between 17 currently available
UCI engines for Android as well as the simple built-in Java engine. The tournament consisted of pair-wise matches between all engines for 10 games with random book opening play, one second-per-move, 32MB hash, and 8MB tablebases cache with the 3- and 4-piece Nalimov, Gaviota, Scorpio, Robbobases endgames on SD card. The 1530 games lasted about two days running full time on a Nexus One phone (without a single crash!). The results are shown below.
Code: Select all
1 Stockfish 2.0 157.5/170
2 RobboLito 0.085g3l 152.5/170
3 IvanHoe-Beta version 999947c 145.0/170
4 Komodo32 1.3 JA 137.0/170
5 Toga II 1.4.1SE 130.5/170
6 Pepito v1.59 104.0/170
7 Daydreamer 1.75 JA 100.5/170
8 Rotor 0.6a 98.0/170
9 gaviota v0.80.0.107 87.5/170 4647.75
10 GNU Chess 5.07.170.7b 87.5/170 4455.25
11 RedQueen 0.9.8 75.5/170
12 GreKo 8.0 71.0/170
13 Jazz v444 JA 50.5/170
14 BikJump v2.1P 49.0/170
15 Diablo 0.5.1 38.5/170
16 ZCT-0.3.2500 34.0/170
17 Chess for Android 10.5/170
18 Simplex 0.9.7 rev 180 1.0/170
Re: Android UCI engine test tournament
Posted: Wed May 18, 2011 9:14 am
by thorstenczub
many thanks aart.
Re: Android UCI engine test tournament
Posted: Sat May 21, 2011 1:18 pm
by Gargamel
Hello,
Another tournament :
Htc legend, chess for Android, 16MB hash, no tablebase.
Chess for Android Match
France, 2011.05.18 - 2011.05.21
--------------------------------------------- Score -- Stockfish----- RobboLito-----IvanHoe-Be---- Toga II ---- Chess for
-----------------------------------------------------------------------------------------------------
1: Stockfish 2.1 ------------------------- 30.5 / 40 XXXXXXXXXX =11==10=== 0===10=011 11=1111111 1111111111
2: RobboLito 0.085e4l ---------------- 28.5 / 40 =00==01=== XXXXXXXXXX =0011=01=1 11=11=1111 1111111111
3: IvanHoe-Beta version 999947c -- 27.0 / 40 1===01=100 =1100=10=0 XXXXXXXXXX 1==1=10111 1111111111
4: Toga II 1.4.1SE ----------------------- 14.0 / 40 00=0000000 00=00=0000 0==0=01000 XXXXXXXXXX 1111111111
5: Chess for Android ------------------- 0.0 / 40 0000000000 0000000000 0000000000 0000000000 XXXXXXXXXX
-----------------------------------------------------------------------------------------------------
100 games: +45 =19 -36