Page 1 of 6

Creating a new (and independent) rating list

Posted: Wed Jun 23, 2010 12:01 pm
by Rebel
It doesn't look like the ban on the IPPO's and family is going to be unleashed by the current 3 main rating lists (CCRL/CEGT/SSDF). Therefore I do hope some people here on this forum will take the initiative to create an own group of volunteers for an independent rating list.

It's quite possible some of the CCRL/CEGT folks are in doubt about the current state of affairs and can be persuaded to join such a new initiative.

Ed

Re: Creating a new (and independent) rating list

Posted: Wed Jun 23, 2010 12:07 pm
by Ristaaf
http://www.ccerl.net/

Norman Schmidts attempt to make a list, it didn't seem to became much more than you see now, but maybee it can with more volonters. I don't have the computer to run such tests (only an old 32bit non SSE one) otherwise (and probably when I get a new computer) I would had helped.

Re: Creating a new (and independent) rating list

Posted: Wed Jun 23, 2010 1:05 pm
by Robert Flesher
Rebel wrote:It doesn't look like the ban on the IPPO's and family is going to be unleashed by the current 3 main rating lists (CCRL/CEGT/SSDF). Therefore I do hope some people here on this forum will take the initiative to create an own group of volunteers for an independent rating list.

It's quite possible some of the CCRL/CEGT folks are in doubt about the current state of affairs and can be persuaded to join such a new initiative.

Ed

I may be willing to dedicate some time to this. I have most the current engines. My computer is moderately fast, Core i7 920.

Re: Creating a new (and independent) rating list

Posted: Wed Jun 23, 2010 3:55 pm
by JackStraw
Count me in if you can get somebody to organize it........

I've got several programs , and an 'old' i7-940 .........

Re: Creating a new (and independent) rating list

Posted: Wed Jun 23, 2010 8:36 pm
by xshat
There is nothing more pathetic and laughable in the chess community than the commercialized attempts to monopolize chess engines, with the subtle intention to slow down the rate at which chess is solved...

I can help.

Re: Creating a new (and independent) rating list

Posted: Wed Jun 23, 2010 9:40 pm
by Gino
Ristaaf wrote:http://www.ccerl.net/

Norman Schmidts attempt to make a list, it didn't seem to became much more than you see now, but maybee it can with more volonters. I don't have the computer to run such tests (only an old 32bit non SSE one) otherwise (and probably when I get a new computer) I would had helped.
Why is Houdini being excluded?

Re: Creating a new (and independent) rating list

Posted: Wed Jun 23, 2010 10:17 pm
by BTO7
Gino wrote:
Ristaaf wrote:http://www.ccerl.net/

Norman Schmidts attempt to make a list, it didn't seem to became much more than you see now, but maybee it can with more volonters. I don't have the computer to run such tests (only an old 32bit non SSE one) otherwise (and probably when I get a new computer) I would had helped.
Why is Houdini being excluded?
He actually started the site right before Houdini came out. Its been stuck in that state since about a week after it was made. Have not seen Norm around other then his new site www.CycloneChess.com He's gone kinda quiet actually since Houdini came onto the scene.

Regards
BT

Re: Creating a new (and independent) rating list

Posted: Wed Jun 23, 2010 11:26 pm
by thorstenczub
BTO7 wrote:
Gino wrote:
Ristaaf wrote:http://www.ccerl.net/

Norman Schmidts attempt to make a list, it didn't seem to became much more than you see now, but maybee it can with more volonters. I don't have the computer to run such tests (only an old 32bit non SSE one) otherwise (and probably when I get a new computer) I would had helped.
Why is Houdini being excluded?
He actually started the site right before Houdini came out. Its been stuck in that state since about a week after it was made. Have not seen Norm around other then his new site http://www.CycloneChess.com He's gone kinda quiet actually since Houdini came onto the scene.

Regards
BT
maybe the witch hunters visited him ...
:roll:

whatever.

Re: Creating a new (and independent) rating list

Posted: Thu Jun 24, 2010 5:14 am
by BB+
I'm not sure I like the methodology of any of the current groups, but then I have a very strong preference for science. For instance, some of the rating groups let the operator choose/create the book. I have no idea how much mischief this could entail, though I see anecdotes around that Engine X does (relatively) better than Engine Y with Book Z. If we want to make it a scientific venture, more discussion is needed. For instance, should a uniform platform be adopted, or is the "benchmark and adjust" procedure sufficient? What aspects of the engine are you trying to measure (for instance, is time management important)? What interference is allowed from the GUI (for instance, is N straight moves with both at 0.00 a draw, even if no repetition has been made)? Is the focus for top engines (top 10 or 20), or for a wide variety (200+) of amateur engines? If you want the latter, then you will likely have to sacrifice "science" to some degree, as to cover such a broad spectrum you will need many different testers involved.

Re: Creating a new (and independent) rating list

Posted: Thu Jun 24, 2010 6:28 am
by zwegner
BB+ wrote:I'm not sure I like the methodology of any of the current groups, but then I have a very strong preference for science. For instance, some of the rating groups let the operator choose/create the book. I have no idea how much mischief this could entail, though I see anecdotes around that Engine X does (relatively) better than Engine Y with Book Z. If we want to make it a scientific venture, more discussion is needed. For instance, should a uniform platform be adopted, or is the "benchmark and adjust" procedure sufficient? What aspects of the engine are you trying to measure (for instance, is time management important)? What interference is allowed from the GUI (for instance, is N straight moves with both at 0.00 a draw, even if no repetition has been made)? Is the focus for top engines (top 10 or 20), or for a wide variety (200+) of amateur engines? If you want the latter, then you will likely have to sacrifice "science" to some degree, as to cover such a broad spectrum you will need many different testers involved.
I would definitely agree here. Then again I am quite skeptical of my own testing methods.

I doubt anyways that the concept of Elo rating for chess has any real meaning. I don't think it's possible to even get any sort of unbiased sample--it's a game of perfect information after all, play shouldn't be random. Things like time control affect the rating, but different engines scale differently with time. In addition there are clearly cases where the rating system is not one dimensional (it doesn't obey the well-ordering principle, if we want to be mathematical about it :)).

Luckily, though, it seems that the Elo model roughly works in most cases.