Page 1 of 1
Independant testers
Posted: Sun Jun 13, 2010 11:38 pm
by LucenaTheLucid
To all those who post matches of Rybka vs. Houdini, Rybka vs. Fire etc etc...
Why not collaborate and make a new rating list which includes all engine from the Ippo family? I would consider purchasing another computer or two if we could all collaborate?
Re: Independant testers
Posted: Mon Jun 14, 2010 4:07 am
by 64x
What do you feel should be the right framework for this? (i.e. Time Control, opening books/start positions, hardware baseline, minimum number of games per contribution, etc)
Also, how to address the multiple versions of IvanHoe (being that many).
Finally, I see that most of the IvanHoe, Fire, Houdini, etc matches are vs Rybka and/or Stockfish. More inclusive matches will be necessary to create a proper list.
Re: Independant testers
Posted: Mon Jun 14, 2010 11:28 pm
by LucenaTheLucid
I like either 60/0 or 40/40 for time control.
Opening books would go by this criteria:
1. Suggestion by Author
2. Book that comes with the engine
3. If no book then we assign a book through the use of Polyglot.
Tablebases would be 5 man if the engine can use them. Or bitbases when appropriate. Use the newest version of Crafty with a benchmark of 1 mnps (one of the testers hardware as the base) so those with faster computers can churn out more games.
Just a start.
Re: Independant testers
Posted: Tue Jun 15, 2010 12:12 am
by 64x
Good start. I may be able to dedicate a few cores if enough interest builds. Still don't know how to handle that many versions of IvanHoe, perhaps only those directly submitted by the author with settings should be tested.
I for one would not be able to test daily compiles.