Independant testers

As in chess tournaments and matches...
Post Reply
LucenaTheLucid
Posts: 160
Joined: Thu Jun 10, 2010 2:14 am
Real Name: Luis Smith

Independant testers

Post by LucenaTheLucid » Sun Jun 13, 2010 11:38 pm

To all those who post matches of Rybka vs. Houdini, Rybka vs. Fire etc etc...

Why not collaborate and make a new rating list which includes all engine from the Ippo family? I would consider purchasing another computer or two if we could all collaborate?

64x
Posts: 301
Joined: Thu Jun 10, 2010 2:47 am

Re: Independant testers

Post by 64x » Mon Jun 14, 2010 4:07 am

What do you feel should be the right framework for this? (i.e. Time Control, opening books/start positions, hardware baseline, minimum number of games per contribution, etc)

Also, how to address the multiple versions of IvanHoe (being that many).

Finally, I see that most of the IvanHoe, Fire, Houdini, etc matches are vs Rybka and/or Stockfish. More inclusive matches will be necessary to create a proper list.

LucenaTheLucid
Posts: 160
Joined: Thu Jun 10, 2010 2:14 am
Real Name: Luis Smith

Re: Independant testers

Post by LucenaTheLucid » Mon Jun 14, 2010 11:28 pm

I like either 60/0 or 40/40 for time control.

Opening books would go by this criteria:

1. Suggestion by Author
2. Book that comes with the engine
3. If no book then we assign a book through the use of Polyglot.

Tablebases would be 5 man if the engine can use them. Or bitbases when appropriate. Use the newest version of Crafty with a benchmark of 1 mnps (one of the testers hardware as the base) so those with faster computers can churn out more games.

Just a start.

64x
Posts: 301
Joined: Thu Jun 10, 2010 2:47 am

Re: Independant testers

Post by 64x » Tue Jun 15, 2010 12:12 am

Good start. I may be able to dedicate a few cores if enough interest builds. Still don't know how to handle that many versions of IvanHoe, perhaps only those directly submitted by the author with settings should be tested.

I for one would not be able to test daily compiles.

Post Reply