IPON Rating's List Bash Script

[kingliveson]

This program started out as a challenge to make a point, but developed into a very nifty tool. It was written in Bash using descriptive variables, functions, and commented for easier modification by others. Attached archive includes source, help file, sample game database, and stats. To re-iterate some of the features:

• it has 100 chess engines common to most ratings' lists
• head-to-head matches are supported
• supports gauntlet and round-robin style tournaments
• gauntlet can be any engine versus all, selected, or random group
• round-robin can be all 100 engines going at it, selected, or random group
• simulates and generates a game (pgn) database
• database preserving option -- newly simulated results can be appended
• displays progress and time elapsed

Chess Engine Rating List Simulated :

9/12/2011 4:08:30 PM :

    Program                          Elo    +   -   Games   Score   Av.Op.  Draws

  1 Houdini 2.0 STD                : 3052    7   7  9900    81.4 %   2796   21.6 %
  2 Houdini 1.5a                   : 3039    7   7  9900    80.2 %   2796   23.5 %
  3 Komodo64 3 SSE42               : 3000    6   6  9900    76.3 %   2797   27.5 %
  4 Deep Rybka 4.1 SSE42           : 2996    6   6  9900    75.9 %   2797   27.8 %
  5 Critter 1.2                    : 2996    6   6  9900    75.9 %   2797   28.1 %
  6 Houdini 1.03a                  : 2990    6   6  9900    75.3 %   2797   28.7 %
  7 Komodo 2.03 DC SSE42           : 2990    6   6  9900    75.2 %   2797   28.7 %
  8 IvanHoe 9.47b                  : 2985    6   6  9900    74.8 %   2797   28.2 %
  9 Stockfish 2.1.1 JA             : 2983    6   6  9900    74.5 %   2797   29.6 %
 10 Deep Rybka 4                   : 2981    6   6  9900    74.3 %   2797   29.0 %
 11 Stockfish 2.01 JA              : 2967    6   6  9900    72.7 %   2797   30.9 %
 12 Igorrit_0086v9_x64             : 2966    6   6  9900    72.6 %   2797   30.3 %
 13 Critter 1.01 SSE42             : 2965    6   6  9900    72.5 %   2797   30.4 %
 14 Houdini 1.0                    : 2963    6   6  9900    72.2 %   2797   31.2 %
 15 RobboLito 009                  : 2961    6   6  9900    71.9 %   2797   30.8 %
 16 IPPOLIT                        : 2951    6   6  9900    70.9 %   2797   31.1 %
 17 Rybka 3 mp                     : 2948    6   6  9900    70.5 %   2797   31.5 %
 18 Stockfish 1.9.1 JA             : 2947    6   6  9900    70.3 %   2797   31.6 %
 19 Critter 0.90 SSE42             : 2947    6   6  9900    70.3 %   2797   32.6 %
 20 Stockfish 1.7.1 JA             : 2936    6   6  9900    69.0 %   2797   33.1 %
 21 Rybka 3 32b                    : 2903    6   6  9900    64.7 %   2798   35.0 %
 22 Komodo64 1.3 JA                : 2896    6   6  9900    63.8 %   2798   36.2 %
 23 Stockfish 1.6.x JA             : 2894    6   6  9900    63.6 %   2798   35.7 %
 24 Naum 4.2                       : 2884    6   6  9900    62.1 %   2798   36.1 %
 25 Critter 0.80                   : 2876    5   5  9900    61.1 %   2798   37.4 %
 26 Komodo 1.2 JA                  : 2866    5   5  9900    59.7 %   2798   37.9 %
 27 Deep Shredder 12               : 2860    5   5  9900    58.8 %   2798   38.2 %
 28 Rybka 2.3.2a mp                : 2858    5   5  9900    58.6 %   2798   37.8 %
 29 Gull 1.2                       : 2854    5   5  9900    58.0 %   2798   37.8 %
 30 Critter 0.70                   : 2853    5   5  9900    57.9 %   2798   38.8 %
 31 Komodo 1.0 JA                  : 2846    5   5  9900    56.8 %   2798   39.3 %
 32 Gull 1.1                       : 2846    5   5  9900    56.8 %   2798   38.0 %
 33 Naum 4.1                       : 2845    5   5  9900    56.8 %   2798   39.2 %
 34 Spike 1.4 32b                  : 2844    5   5  9900    56.6 %   2798   39.5 %
 35 Deep Sjeng c't 2010 32b        : 2844    5   5  9900    56.6 %   2798   39.5 %
 36 Deep Fritz 12 32b              : 2838    5   5  9900    55.7 %   2798   38.7 %
 37 Naum 4                         : 2838    5   5  9900    55.7 %   2798   38.9 %
 38 Rybka 2.2n2 mp                 : 2833    5   5  9900    55.0 %   2798   39.2 %
 39 Gull 1.0a                      : 2828    5   5  9900    54.2 %   2798   39.7 %
 40 Rybka 1.2f                     : 2822    5   5  9900    53.4 %   2798   40.2 %
 41 Stockfish 1.5.1 JA             : 2821    5   5  9900    53.3 %   2798   39.0 %
 42 Protector 1.4.0 x64            : 2818    5   5  9900    52.8 %   2798   40.5 %
 43 Hannibal 1.1                   : 2816    5   5  9900    52.5 %   2798   39.7 %
 44 HIARCS 13.2 MP 32b             : 2815    5   5  9900    52.3 %   2798   39.5 %
 45 spark-1.0 SSE42                : 2813    5   5  9900    52.1 %   2798   40.2 %
 46 Fritz 12 32b                   : 2805    5   5  9900    50.9 %   2799   40.5 %
 47 HIARCS 13.1 MP 32b             : 2797    5   5  9900    49.8 %   2799   39.8 %
 48 Deep Junior 12.5               : 2790    5   5  9900    48.7 %   2799   41.5 %
 49 Deep Fritz 11 32b              : 2787    5   5  9900    48.3 %   2799   40.6 %
 50 Thinker 5.4d Inert x64         : 2781    5   5  9900    47.4 %   2799   41.4 %
 51 spark-0.4                      : 2776    5   5  9900    46.7 %   2799   40.6 %
 52 Stockfish 1.4 JA               : 2774    5   5  9900    46.4 %   2799   40.6 %
 53 Doch64 1.2 JA                  : 2773    5   5  9900    46.2 %   2799   40.6 %
 54 Zappa Mexico II                : 2771    5   5  9900    45.9 %   2799   41.4 %
 55 Shredder Bonn 32b              : 2769    5   5  9900    45.7 %   2799   41.1 %
 56 Critter 0.60                   : 2760    5   5  9900    44.4 %   2799   41.4 %
 57 Fruit 09_07_05 x64             : 2760    5   5  9900    44.4 %   2799   40.5 %
 58 Protector 1.3.2 JA             : 2759    5   5  9900    44.3 %   2799   41.1 %
 59 Booot 5.1.0                    : 2753    5   5  9900    43.4 %   2799   40.5 %
 60 Deep Shredder 11               : 2752    5   5  9900    43.2 %   2799   40.9 %
 61 Doch64 09.980 JA               : 2746    5   5  9900    42.4 %   2799   39.6 %
 62 Deep Junior 12                 : 2737    5   5  9900    41.1 %   2799   40.0 %
 63 Hannibal 1.0a                  : 2736    5   5  9900    41.0 %   2799   39.7 %
 64 Naum 3.1                       : 2735    5   5  9900    40.9 %   2799   41.1 %
 65 Deep Onno 1-2-70               : 2735    5   5  9900    40.9 %   2799   40.1 %
 66 Zappa Mexico I                 : 2733    5   5  9900    40.6 %   2799   40.2 %
 67 Onno-1-1-1                     : 2733    5   5  9900    40.6 %   2799   39.8 %
 68 Rybka 1.0 Beta                 : 2731    5   5  9900    40.3 %   2799   40.3 %
 69 Spark-0.3 VC(a)                : 2729    5   5  9900    40.0 %   2799   40.3 %
 70 Onno-1-0-0                     : 2729    5   5  9900    40.0 %   2799   39.5 %
 71 Deep Sjeng WC2008              : 2726    5   5  9900    39.6 %   2799   39.5 %
 72 Toga II 1.4 beta5c BB          : 2725    5   5  9900    39.5 %   2799   39.5 %
 73 Deep Junior 11.2               : 2722    5   5  9900    39.0 %   2799   39.6 %
 74 Strelka 2.0 B                  : 2721    5   5  9900    38.9 %   2799   40.3 %
 75 Hiarcs 12.1 MP 32b             : 2714    5   5  9900    38.0 %   2799   39.8 %
 76 Deep Sjeng 3.0                 : 2711    5   5  9900    37.6 %   2800   39.6 %
 77 Umko 1.2 SSE42                 : 2711    5   5  9900    37.5 %   2800   38.8 %
 78 Shredder Classic 4 32b         : 2705    5   5  9900    36.8 %   2800   38.1 %
 79 Critter 0.52b                  : 2703    5   5  9900    36.5 %   2800   39.0 %
 80 Bright 0.4a                    : 2700    5   5  9900    36.1 %   2800   37.8 %
 81 Deep Junior 11.1a              : 2689    6   6  9900    34.5 %   2800   37.1 %
 82 Naum 2.2 32b                   : 2687    6   6  9900    34.3 %   2800   36.8 %
 83 Deep Junior 2010               : 2684    6   6  9900    33.9 %   2800   36.8 %
 84 Glaurung 2.2 JA                : 2683    6   6  9900    33.7 %   2800   36.1 %
 85 Umko 1.1 SSE42                 : 2682    6   6  9900    33.7 %   2800   36.5 %
 86 Rybka 1.0 Beta 32b             : 2682    6   6  9900    33.6 %   2800   36.8 %
 87 Fruit 05/11/03 32b             : 2674    6   6  9900    32.7 %   2800   35.7 %
 88 HIARCS 11.2 32b                : 2673    6   6  9900    32.5 %   2800   35.4 %
 89 Crafty 23.4 JA x64             : 2665    6   6  9900    31.5 %   2800   34.8 %
 90 Loop 13.6/2007                 : 2663    6   6  9900    31.3 %   2800   33.8 %
 91 ListMP 11                      : 2658    6   6  9900    30.7 %   2800   34.2 %
 92 Toga II 1.2.1a                 : 2657    6   6  9900    30.5 %   2800   34.2 %
 93 Jonny 4.00 32b                 : 2656    6   6  9900    30.4 %   2800   34.5 %
 94 LoopMP 12 32b                  : 2648    6   6  9900    29.4 %   2800   32.5 %
 95 Deep Shredder 10               : 2645    6   6  9900    29.1 %   2800   32.3 %
 96 Twisted Logic 20100131x        : 2644    6   6  9900    28.9 %   2800   31.8 %
 97 Crafty 23.3 JA                 : 2635    6   6  9900    27.8 %   2800   31.3 %
 98 Spike 1.2 Turin 32b            : 2628    6   6  9900    27.0 %   2800   29.7 %
 99 Deep Sjeng 2.7 32b             : 2594    6   6  9900    23.4 %   2801   26.2 %
100 Crafty 23.1 JA                 : 2585    7   7  9900    22.4 %   2801   24.7 %

If an engine is out of place, it is an easy fix -- just change its weight relative to other programs. A newly released (or unlisted) program's weight can be determined by simply playing it against one or two programs listed. Once the weight is obtained, thousands and millions of games can then be simulated. Final release? I think so...

[kingliveson]

Simulation Formula

Engine's weights are calculated using the following:
1000/(1+10^((3025-E2)/(400))) 
Houdini 2 is used as reference.

E1K = engine's one weight
E2K = engine's two weight

E1RV = engine's one randomly generated value between 0 and E1K
E2RV = engine's two randomly generated value between 0 and E2K

RVD = (E1RV - E2RV) or (E2RV - E1RV) = random values difference
KV = E1K or E2K = constant lesser weight value of the two engines

BF = log[10](-1+1/.270) = 0.432 = black factor
WF = log[10](-1+1/.375) = 0.222 = white factor

BWV = KV(BF) = black winning value
WWV = KV(WF) = white winning value

RVD >= BWV = black wins
RVD >= WWV = white wins

Sample runs using Zappa Mexico II and Stockfish 2.1.1

Zappa = E1K = 143
Stockfish = E2K = 382

1st run:
Zappa is white and E1RV = 125
Stockfish is black and E2RV = 185
RVD = (E2RV-E1RV) = (185-125) = 60
KV = E1K = 143
BWV = KV(BF) = 143(.432) = 61
RVD < BWV = 1/2-1/2

2nd run:
Zappa is black and E1RV = 100
Stockfish is white and E2RV = 10
RVD = (E1RV-E2RV) = (100-10) = 90
KV = E1K = 143
BWV = KV(BF) = 143(.432) = 61
RVD >= BWV = 0-1

3rd run:
Zappa is white and E1RV = 110
Stockfish is black and E2RV = 81
RVD = (E1RV-E2RV) = (110-81) = 29
KV = E1K = 143
WWV = KV(WF) = 143(.222) = 31
RVD < WWV = 1/2-1/2

4th run:
Zappa is black and E1RV = 142
Stockfish is white and E2RV = 215
RVD = (E2RV-E1RV) = (215-142) = 73
KV = E1K = 143
WWV = KV(WF) = 143(.222) = 31
RVD >= WWV = 1-0

Stockfish's Elo relative to Houdini = (-400*ln(-1+1/(382/(1000))))*(1/ln(10))+3025 = 2941
Zappa's Elo relative to Houdini = (-400*ln(-1+1/(143/(1000))))*(1/ln(10))+3025 = 2713

[kingliveson]

Ok am sure this is the final release which re-adjusted all weights, and corrected progress display glitch.

[kingliveson]

Well well, it seems as if there will be at least one more release. Supposedly, there's roughly 35 Elo advantage to moving first, and to reflect existing databases' win/draw/loss ratio, a subtle change was made in reference to previous post:

black factor: log[10](-1+1/.262) = 0.44975507050329610828941338578068
white factor: log[10](-1+1/.360) = 0.24987747321659990626489977238775

Code: Select all

Games        : 495000 (finished)

White Wins   : 181800 (36.7 %)
Black Wins   : 134155 (27.1 %)
Draws        : 179045 (36.2 %)
Unfinished   :      0

White Perf.  : 54.8 %
Black Perf.  : 45.2 %

There were other cosmetic changes and minor corrections to facilitate input bugs. I can't believe am actually doing this. It looks good though and reliable. Some screen shots:


This simulates/generates database for Rybka 1.0's gauntlet performance against 99 other programs.


Simulating all 100 programs in a round-robin tournament can take a while, depending on the system. On one computer, the time (minutes) it takes to simulate all 100 programs is estimated with:
(.1940997046)(R^1.4303970905) where R is number of rounds


Tournament help.

In order to not abuse upload privileges, the file was uploaded here instead, and subsequent, if any other updates, will be stored there.

[orgfert]

I like this.

However the versions are curious. Before you had CRLS_1.0z but on the upload site the newest seems to be CRLS_0b.09.10.

[kingliveson]

I like this.

However the versions are curious. Before you had CRLS_1.0z but on the upload site the newest seems to be CRLS_0b.09.10.

My mistake; it should have been noted that all previous releases were re-named. Since this whole thing was not planned out, a scheme where version is the date is preferred. So yes, 0B.09.10 is the latest which is just today's date represented in hexadecimal. 11.09.16.

[Uly]

Any chance of getting an executable out of this?

[kingliveson]

Any chance of getting an executable out of this?

Under assumption you aren't willing to install Linux, the best approach at current will be to install VirtualBox for Windows hosts, and download openSUSE Live, preferably, KDE.

Live system allows you to run Linux without actually installing it. If burnt to a CD-ROM, you can boot from it. VirtualBox however, allows you to run the ISO inside your Windows without even burning it to a CD-ROM.

You can and should add a shared directory between the host OS (Windows) and guest OS (Linux). Run the script in this directory so the database is accessible to the host. If further assistance is needed, please post it here.

[orgfert]

Any chance of getting an executable out of this?

Under assumption you aren't willing to install Linux, the best approach at current will be to install VirtualBox for Windows hosts, and download openSUSE Live, preferably, KDE.

Live system allows you to run Linux without actually installing it. If burnt to a CD-ROM, you can boot from it. VirtualBox however, allows you to run the ISO inside your Windows without even burning it to a CD-ROM.

You can and should add a shared directory between the host OS (Windows) and guest OS (Linux). Run the script in this directory so the database is accessible to the host. If further assistance is needed, please post it here.

Installing CYGWIN is another easy option for Windows users.

[veritas]

Any chance of getting an executable out of this?

Under assumption you aren't willing to install Linux, the best approach at current will be to install VirtualBox for Windows hosts, and download openSUSE Live, preferably, KDE.

Live system allows you to run Linux without actually installing it. If burnt to a CD-ROM, you can boot from it. VirtualBox however, allows you to run the ISO inside your Windows without even burning it to a CD-ROM.

You can and should add a shared directory between the host OS (Windows) and guest OS (Linux). Run the script in this directory so the database is accessible to the host. If further assistance is needed, please post it here.

Installing CYGWIN is another easy option for Windows users.

not everyone is pc literate or a geek

Developers coming from a Windows background will be able to write console or GUI executables that rely on the Microsoft Win32 API instead of Cygwin using the -mno-cygwin option to GCC. The -shared option allows to write Windows Dynamically Linked Libraries (DLLs). The resource compiler windres is also provided


Rocket science to some , simple for others