Houdini Is Top Rated Chess Engine

[Odeus37]

http://sedatchess.110mb.com/files/scct_191210.txt

Code: Select all

                     SCCT Rating List - All Versions

                      Last Update:10:43 19.12.2010


Rank Name                        Elo    +   -  games score oppo. draws 

   1 Houdini 1.5 x64 T4         3340   51   48   204   83%  3063   22% 
   2 Rybka 4 Exp 61 x64 T4      3297   31   30   420   75%  3109   31% 
   3 Rybka 4 x64 T4             3287   17   17  1440   78%  3065   33% 
   4 Rybka 3 x64 T4             3234   18   18  1070   69%  3095   37% 
   5 Rybka 4 Exp 61 x64 T1      3230   31   30   560   83%  2938   19% 
   6 Stockfish 1.9 JA x64 T4    3222   16   16  1331   69%  3083   40% 
   7 Stockfish 1.8 JA x64 T4    3214   21   20   850   70%  3060   38% 
   8 Critter 0.90 x64 T4        3213   29   28   431   69%  3075   37% 
   9 Stockfish 1.7.1 JA x64 T4  3193   19   19   930   66%  3079   45% 
  10 Rybka 4 x64 T1 Ponder ON   3191   18   18  1110   74%  3000   40% 
  11 Rybka 4 x64 T1 Ponder OFF  3190   18   18  1090   69%  3043   39% 
  12 Naum 4.2 x64 T4            3176   15   15  1481   65%  3069   40% 
  13 Critter 0.80 x64 T4        3152   19   18  1020   65%  3044   37% 
  14 Rybka 4 w32 T1             3144   25   24   570   68%  3012   39% 
  15 Stockfish 1.8 JA x64 T1    3130   24   24   680   68%  2983   35% 
  16 Stockfish 1.7.1 JA x64 T1  3119   21   20   850   60%  3027   38% 
  17 Critter 0.70 x64 T4        3113   21   21   720   54%  3083   41% 
  18 Deep Shredder 12 x64 T4    3099   15   15  1440   54%  3068   40% 
  19 Deep Fritz 12 T4           3094   16   16  1304   53%  3069   44% 
  20 Spark 1.0 x64 T4           3078   29   29   371   46%  3108   44% 
  21 Spark 0.4 x64 T4           3058   16   16  1330   49%  3067   41% 
  22 Hiarcs 13.1 T4             3055   15   15  1473   50%  3055   41% 
  23 Zappa Mexico II x64 T4     3055   15   15  1430   48%  3069   43% 
  24 Naum 4.2 x64 T1            3054   19   18   970   53%  3021   44% 
  25 Protector 1.3.6 x64 T4     3042   17   17  1180   49%  3053   39% 
  26 Komodo 1.2 JA x64          3034   14   14  1641   47%  3051   41% 
  27 Critter 0.60 x64 T4        3022   24   24   550   58%  2963   40% 
  28 Protector 1.3.4 JA x64 T4  3016   24   24   570   56%  2975   40% 
  29 Thinker 5.4D x64 T4        3015   16   16  1300   47%  3036   37% 
  30 Komodo 1.1 JA x64          3015   24   24   560   52%  3003   43% 
  31 Komodo 1.0 JA x64          3015   24   24   560   53%  2989   40% 
  32 Deep Onno-1-2-70 x64 T4    3008   16   16  1300   44%  3050   40% 
  33 Gull 1.0 x64               3007   16   16  1448   49%  3010   37% 
  34 Deep Junior 11.2 x64 T4    3007   15   15  1451   45%  3041   36% 
  35 Spark 0.3a x64 T4          3006   25   25   540   55%  2967   37% 
  36 Deep Junior 11.1a x64 T4   3004   25   25   500   50%  3002   42% 
  37 Fritz 12                   2989   30   30   417   61%  2895   34% 
  38 Deep Sjeng WC2008 x64 T4   2988   16   16  1260   42%  3048   36% 
  39 Hiarcs 12.1 T4             2987   24   24   560   49%  2987   44% 
  40 Gull 0.12a x64             2976   24   24   590   49%  2980   37% 
  41 Jonny 4.00 T4              2943   19   19  1000   40%  3016   34% 
  42 Loop 2007 x64 T4           2938   17   17  1240   37%  3036   35% 
  43 Crafty 23.4 PS x64 T4      2938   29   29   413   35%  3041   35% 
  44 Equinox 0.87t x64 T4       2923   33   34   300   37%  3014   36% 
  45 Rybka 1.0 Beta             2914   38   37   319   71%  2761   22% 
  46 Umko 1.1 x64 T4            2909   32   32   330   50%  2909   33% 
  47 Hannibal 1.0a x64          2907   17   17  1320   33%  3030   31% 
  48 Umko 1.0 x64 T4            2898   18   18  1090   32%  3029   33% 
  49 Crafty 23.3 JA x64 T4      2890   18   18  1117   31%  3032   34% 
  50 Booot 5.0.0                2885   20   20   880   51%  2884   32% 
  51 Hannibal 021310            2875   26   26   540   35%  2988   32% 
  52 Scorpio 2.6 x64 JA T4      2874   23   23   660   50%  2878   29% 
  53 Tornado 4.4 x64 T4         2871   34   35   310   28%  3037   32% 
  54 Crafty 23.2 JA x64 T4      2865   26   26   530   39%  2946   32% 
  55 Fruit 2.3.1                2857   23   23   710   36%  2969   31% 
  56 Ktulu 9                    2851   25   26   600   47%  2879   25% 
  57 Bison 10.1 DC x64          2849   26   26   550   37%  2953   28% 
  58 The King 3.50 x64 T4       2845   24   25   670   35%  2961   26% 
  59 Scorpio 2.5 x64 JA T4      2834   27   27   510   36%  2944   32% 
  60 Spike 1.2 Turin            2828   27   27   520   34%  2954   33% 
  61 Umko 0.9 x64               2827   24   24   580   44%  2870   33% 
  62 Shredder 9                 2824   36   35   320   59%  2769   23% 
  63 SmarThink 1.20 x64         2817   25   25   620   41%  2898   27% 
  64 Delfi 5.4 T2               2813   26   27   560   33%  2949   30% 
  65 Tornado 4.1 x64 T4         2786   37   38   300   34%  2919   23% 
  66 Chess Tiger 2007.1         2777   24   24   650   39%  2875   29% 
  67 BugChess2 1.7 x64          2766   33   33   320   46%  2801   28% 
  68 Fritz in Bahrain           2757   34   34   331   40%  2837   26% 
  69 Daydreamer 1.75 JA x64     2753   29   29   430   48%  2780   28% 
  70 BugChess2 V1.64 x64        2744   35   35   270   48%  2755   33% 
  71 Frenzee may07 x64 T4       2737   34   34   310   51%  2739   25% 
  72 Hamsters 0.8.2 T4          2730   35   35   280   49%  2753   31% 
  73 Chess Tiger 14.0           2730   35   35   277   53%  2715   32% 
  74 Booot 4.15.1               2729   39   39   220   51%  2721   31% 
  75 Baron 2.22 x64 T4          2727   33   33   310   51%  2719   25% 
  76 Chronos 1.98 beta x64      2725   35   35   255   49%  2727   36% 
  77 Alaric 707                 2720   33   33   315   45%  2758   24% 
  78 Cyrano 0.6b17 JA           2717   33   33   320   52%  2704   28% 
  79 CM9000 Mapi                2716   32   32   320   52%  2705   27% 
  80 Deep Pharaon 3.5.1 T4      2711   37   37   270   43%  2758   24% 
  81 Colossus 2008b             2710   34   35   330   42%  2796   21% 
  82 Brutus v8.05 JA x64 T4     2708   34   35   300   42%  2771   28% 
  83 CM9000 Sedat               2703   26   26   520   51%  2699   25% 
  84 Zarkov 6.41                2702   34   35   320   35%  2820   28% 
  85 Tornado 3.50d x64 T4       2697   35   35   290   49%  2715   29% 
  86 E.T.Chess 13.01.2008       2693   34   34   275   50%  2693   32% 
  87 ProDeo 1.6                 2688   32   33   320   42%  2742   28% 
  88 Wildcat 8.0                2687   34   34   315   43%  2758   25% 
  89 Zarkov 6.33                2687   32   33   315   42%  2737   31% 
  90 Jonny 2.89                 2685   31   31   355   52%  2670   24% 
  91 CM9000 Default             2679   37   37   250   47%  2708   26% 
  92 SlowChess Blitz WV2.1      2678   39   40   220   44%  2728   30% 
  93 Alfil 8.1.1 Opt            2673   34   35   315   40%  2765   25% 
  94 Gandalf 6.01               2672   31   32   360   40%  2752   29% 
  95 Movei 0.08.438             2668   32   33   340   43%  2730   27% 
  96 Ruffian 2.1.0              2661   34   35   360   31%  2825   19% 
  97 SOS 5.1 for Arena          2660   33   33   340   40%  2746   27% 
  98 Arasan 11.7 x64 T4         2652   36   37   250   45%  2688   31% 
  99 GarboChess v2.20 x64       2647   32   32   345   43%  2707   26% 
 100 Junior 6.0                 2642   37   37   258   40%  2721   30% 
 101 Deuterium v09.01.26.492    2640   33   33   305   44%  2681   31% 
 102 Hiarcs 7.32                2636   35   36   280   38%  2725   33% 
 103 Arasan 12.1 x64 T4         2632   35   35   290   38%  2720   27% 
 104 Gaviota 0.80 x64 T4        2614   35   36   299   30%  2769   28% 
 105 Nimzo 8                    2600   37   38   260   34%  2726   26% 
 106 Fritz 6                    2582   38   39   260   31%  2729   25% 
 107 Francesca 0.15             2554   42   44   200   31%  2690   24% 
 108 Gaviota 0.75.7 x64 T4      2521   39   40   250   28%  2681   22% 

[BB+]

However, I am willing to wager that CCRL testers haven't had to dole out much, if any, money on the commercial chess engines they test,

Free samples are always one of the perks of testing. It makes me wonder what the word "independent" might really mean.

That's the fear of an empty, hobbyless void of a life.

There was some quotation, I think in September 2009 during the last successful TalkChess attempt to hush up IPPOLIT, when someone worried that "clones" (and particularly open-source ones) would "ruin our hobby" -- which seems almost more like paranoia (or fear of change) than anything else.

[notyetagm]

And slowly but surely, gear lust trumps "principled" objection.

Houdini is "Mainstream" enough for me! I like the work that Robert has done on this engine.

+1

[Jeremy Bernstein]

However, I am willing to wager that CCRL testers haven't had to dole out much, if any, money on the commercial chess engines they test,

Free samples are always one of the perks of testing. It makes me wonder what the word "independent" might really mean.

Well, I don't really think it's the free samples which are the problem. It's the lack of an objective, organized testing system which could trump the threat of revoked perks. CCRL is just a bunch of guys running chess engine matches on their basement PCs and maintaining meticulously organized archives of chess engine releases and rarities.

If Samsung decided to boycott Stiftung Warentest or Consumer Reports because they object to their testing of LG products, they'd a) be a laughingstock and b) be missing out on valuable press about their products. If Vasik decides not to send CCRL a free Rybka to test, it's leverage. My guess is, the rating lists are fairly insignificant to everyone except insiders, and thus rather subject to that kind of leverage. The competitions are probably more significant as far as sales are concerned (World Computer Chess Champion!), and the fact that HIARCS or Rybka representatives are able to influence competition entrance policy is scandalous.

jb

[Adam Hair]

Here is a true independent chess engine ratings list: SWCR by Frank Quisinsky. All engines are tested including Fruit derivative such as Rybka. He said he would be testing Houdini soon as he has the opportunity.

IPON chess engine ratings list is another independent website to check for top engines. Only issue I have with this list is that raw game data is not provided.

There was a time when CCRL was independent, but now they are Rybka beta testers who receive free copies of the final product; nothing wrong with that. Personal relationship with Rybka author has grown to the point where they feel as if part of the team, and there is also an emotional attachment -- so testing another engine that's much stronger/better than the one they helped produce, and publishing the results seem inappropriate. Who can blame them?!

I had better check with Graham. For some reason I have not received my free copy.

Let's not forget how one mod who also happens to be a Rybka beta tester almost destroyed that site until the new mod team got in. It's only a matter of time before the "right" mod gets in again. I haven't logged in there since June, perhaps I will again if there's something really interesting.

I never realized that Graham was a Rybka beta tester. Wow.

If anymore proof is needed that CCRL members have emotional attachment to Rybka, all you have to read is comedy on talkchess. Never-the-less, Houdini is now number 1 chess engine in the world wether they like it or not. It is also now clear to most people that these so-called rating lists are nothing but shills for Rybka. Am glad that there are now other lists who are independent.

Seriously Franklin, you are engaging in a fair amount of supposition. There is but one
Rybka beta tester in the CCRL, and he does not limit beta testing to Rybka. Graham is
not a Rybka beta tester. One look at the tournaments that he runs tells anyone that his
interest is in all chess engines that he considers legitimate, not just one. I am not
going to defend his actions as moderator; he acted how he acted due to his opinion
concerning Ippolit, rightly or wrongly. And I believe he would have reacted strongly
if he thought that Ippolit was a reverse engineered version of Shredder or, for that
matter, Critter ( if Critter had been at its present level 1 1/2 years ago).


This whole Fruit/Rybka/Ippolit/Houdini debate has been interesting in its own right.
Human society in minature. Much of the debate has been fueled by a clash of personalities,
not evidence for one side or another. The actions by some parties has been mystifying
on some levels, yet possibly enlightning over all. Quite interesting.

[Adam Hair]

Things aren't as bad as they once were. TalkChess is (at least currently) not censoring it.

There was a time when CCRL was independent, but now they are Rybka beta testers who receive free copies of the final product; nothing wrong with that. Personal relationship with Rybka author has grown to the point where they feel as if part of the team, and there is also an emotional attachment -- so testing another engine that's much stronger/better than the one they helped produce, and publishing the results seem in appropriate. Who can blame them?!

I much appreciate CCRL, but feel that its focus has been mis-oriented, or at least mis-understood. For some reason, others have pointed to it as the "gold standard" of ratings lists, but the thing I find useful about it is that it canvasses so many engines.

I would say that the idea of a ratings "league" is a bit of a bad idea. I would say you should either: have a "league" like Chess War, or TCEC, or many others; or, do a proper quasi-scientific ratings service -- but trying to have a "league" and act as if the ratings are anything more than just local to your "league" is where I find there to be a bit of unclear purpose. CCRL (and all of this applies to CEGT too) manages to be a "pooled league", which to me seems to end up with the worst of both worlds: the "league" aspect for individual testers is diminished (you don't get a "champion" as with TCEC), while the manner of the pooling such results exaggerates the value of the ratings. Maybe it should be called some like "Meta Rating List of Various Certified Local Computer Chess Leagues" or some other monstrosity. [Though at some point, I don't see why you don't go further and merge even results outside your "pooled league" into such a meta-list].

So to recapitulate: CCRL (or CEGT) is a pooled league with its own rating list. This could be re-phrased as saying it's a bunch of individual testers, each with his own "league", who then merge together their results to make a meta-rating list. So then, why stop there in doing this meta-operation? Why not include CEGT or SWCR in with CCRL results? If FIDE doesn't have separate ratings for "separate conditions", why should computer chess? [The CCRL statement "We thought that our hobby would be more meaningful if we combined our results by being part of a group" begs the question: why not pool your results more widely?]. As has been pointed out in other places the "standard conditions" of these leagues can often be rather lacking in uniformity across testers (such as benchmarking to find out what "40/40" on a given computer means). The current situation seems to be one where you are either in a CCRL/CEGT "circle" or you're not, and such decisions seem more based on social aspects (or even the politics of exclusion) rather than anything else.

As for any emotional attachment, I think (though not sure) that Rybka has been #1 for the entire history of CCRL (since 2006), which might play some type of rôle.

I am going to reply to this soon ( it is past my bed time right now), but you really could answer some of these questions yourself. There is more uniformity of testing conditions intra-league as opposed to inter-league.

[BB+]

I never realized that Graham was a Rybka beta tester. Wow.

I too was wondering about the reference here, as it seemed to me that GB was the TalkChess mod to whom he referred, while I too thought it unlikely he was a Rybka beta tester.

There is more uniformity of testing conditions intra-league as opposed to inter-league.

It is precisely this contention with which I disagree. Firstly, doing Crafty benchmarks to normalise 40/40 on a given machine is just not too precise. A given engine might be 5-10% slower or faster (relatively) due to setup issues, particularly with memory speed and/or caching. Furthermore, every tester can choose a different book, even on a match/tournament basis. These two aspects are fairly large sources of non-uniformity, even for one tester who has multiple non-identical machines and/or plays different tournaments with different books. More minor aspects of non-uniformity would be choice of TB usage and possibly GUI draw/resign rules (if applicable). In contrast, SSDF uniformised its conditions almost completely. I would like to see some actual evidence (or at least a compelling argument) that lumping together all "blitz" time controls from the alphabet of agencies is any worse than what is already extant from combining quasi-comparable data under conditions deemed equivalent.

[kingliveson]

I never realized that Graham was a Rybka beta tester. Wow.

Of course he's not a Rybka beta tester

Seriously Franklin, you are engaging in a fair amount of supposition. There is but one
Rybka beta tester in the CCRL, and he does not limit beta testing to Rybka. Graham is
not a Rybka beta tester. One look at the tournaments that he runs tells anyone that his
interest is in all chess engines that he considers legitimate, not just one. I am not
going to defend his actions as moderator; he acted how he acted due to his opinion
concerning Ippolit, rightly or wrongly. And I believe he would have reacted strongly
if he thought that Ippolit was a reverse engineered version of Shredder or, for that
matter, Critter ( if Critter had been at its present level 1 1/2 years ago).

You should see me right now eating a very delicious Peach Fruit as I type with the juice dripping wet all over my keyboard...hope it doesn't cause any future damages.

Look, If the most public character from a group behaves a certain way, and becomes an advocate in the name of that group, what do you suppose will be concluded about that entity -- is there an individual from any other testing group who goes out of his way trying to delegitimize a chess engine -- posting feces picture to depict the engine?! And at one point even quiting the forum because public discussion of the new engine was allowed. I see a love affair.

In the same token, he feels ok testing an engine that has been proven to more than likely violated the GPL and yet continues to bash another which merely took ideas (not code). LOl, this is why I normlay have popcorn handy when reading talkchess. The difference between he and I on this matter is the price we pay for our copy.

As for his behaviour as a mod, whew, that ship already sailed, and am pretty sure there's a general consensus on that so no need to go into it.

The general view on this matter tends to agree that a testing group claiming to be independent should test the strongest chess engine. See the picture of recent poll on talkchess. Anyways, they are private groups and can do as they will, but the point is you cant claim independece when you refuse to test an enigne because it threatens personal relationship.

This whole Fruit/Rybka/Ippolit/Houdini debate has been interesting in its own right.
Human society in minature. Much of the debate has been fueled by a clash of personalities,
not evidence for one side or another. The actions by some parties has been mystifying
on some levels, yet possibly enlightning over all. Quite interesting.

Oh the evidence is out there -- some just cannot or decided not to see it -- perhaps blinded by a free copy.

[kingliveson]

SCCT Rating List - TOP 20

Rank	Program				Author			Elo	
01	Houdini 1.5 x64 T4		Robert Houdart		3356	
02	Rybka 4 Exp 61 x64 T4		Vasik Rajlich		3298	
03	Stockfish 1.9 JA x64 T4		Tord Romstad		3220	
04	Critter 0.90 x64 T4		Richard Vida		3210	
05	Naum 4.2 x64 T4			Aleksandar Naumov	3175	
06	Deep Shredder 12 x64 T4		Stefan Mayer Kahlen	3100	
07	Deep Fritz 12 T4		F.Morsch & M.Feist	3095	
08	Spark 1.0 x64 T4		Allard Siemelink	3075	
09	Zappa Mexico II x64 T4		Anthony Cozzie		3055	
10	Hiarcs 13.1 T4			Mark Uniacke		3055	
11	Protector 1.3.6 x64 T4		Raimund Heid		3041	
12	Komodo 1.2 JA x64		Don Dailey		3036	
13	Thinker 5.4D x64 T4		Lance Perkins		3015	
14	Deep Onno 1.2.70 x64 T4		Onno Garms		3008	
15	Gull 1.0 x64			Vadim Demichev		3007	
16	Deep Junior 11.2 x64 T4		A.Ban & S.Bushinsky	3006	
17	Deep Sjeng WC2008 x64 T4	Gian-Carlo Pascutto	2987	
18	Jonny 4.00 T4			Johannes Zwanzger	2943	
19	Crafty 23.4 PS x64 T4		Robert Hyatt		2938	
20	Loop 2007 x64 T4		Fritz Reul		2938

[Adam Hair]

This whole Fruit/Rybka/Ippolit/Houdini debate has been interesting in its own right.
Human society in minature. Much of the debate has been fueled by a clash of personalities,
not evidence for one side or another. The actions by some parties has been mystifying
on some levels, yet possibly enlightning over all. Quite interesting.

Oh the evidence is out there -- some just cannot or decided not to see it -- perhaps blinded by a free copy.

I don't mean that there is no evidence. There is definitely evidence, although the exact amount of skew may be a matter of debate.
Most of the time the debate ignores the evidence, an error that is committed by both sides.

The general view on this matter tends to agree that a testing group claiming to be independent should test the strongest chess engine. See the picture of recent poll on talkchess. Anyways, they are private groups and can do as they will, but the point is you cant claim independece when you refuse to test an enigne because it threatens personal relationship.

The only bone of contention I have with the above statement is "the point is you cant claim independece when you refuse to test an enigne because it threatens personal relationship". That part is off the mark. In Graham's case, if you let go of trying to explain his
actions by assuming some relationship with Vas would threatened, then you might would see other factors at play.

Anyway, my role in life isn't to apologize for or justify Graham's thoughts and actions. He can do those things for himself. My reason
for speaking up is that I am tired of the suppositions that are thrown out by both sides, especially by people who I think are capable
of distinguishing between fact and opinion.