To all those who post matches of Rybka vs. Houdini, Rybka vs. Fire etc etc...
Why not collaborate and make a new rating list which includes all engine from the Ippo family? I would consider purchasing another computer or two if we could all collaborate?
Independant testers
-
- Posts: 160
- Joined: Thu Jun 10, 2010 2:14 am
- Real Name: Luis Smith
Re: Independant testers
What do you feel should be the right framework for this? (i.e. Time Control, opening books/start positions, hardware baseline, minimum number of games per contribution, etc)
Also, how to address the multiple versions of IvanHoe (being that many).
Finally, I see that most of the IvanHoe, Fire, Houdini, etc matches are vs Rybka and/or Stockfish. More inclusive matches will be necessary to create a proper list.
Also, how to address the multiple versions of IvanHoe (being that many).
Finally, I see that most of the IvanHoe, Fire, Houdini, etc matches are vs Rybka and/or Stockfish. More inclusive matches will be necessary to create a proper list.
-
- Posts: 160
- Joined: Thu Jun 10, 2010 2:14 am
- Real Name: Luis Smith
Re: Independant testers
I like either 60/0 or 40/40 for time control.
Opening books would go by this criteria:
1. Suggestion by Author
2. Book that comes with the engine
3. If no book then we assign a book through the use of Polyglot.
Tablebases would be 5 man if the engine can use them. Or bitbases when appropriate. Use the newest version of Crafty with a benchmark of 1 mnps (one of the testers hardware as the base) so those with faster computers can churn out more games.
Just a start.
Opening books would go by this criteria:
1. Suggestion by Author
2. Book that comes with the engine
3. If no book then we assign a book through the use of Polyglot.
Tablebases would be 5 man if the engine can use them. Or bitbases when appropriate. Use the newest version of Crafty with a benchmark of 1 mnps (one of the testers hardware as the base) so those with faster computers can churn out more games.
Just a start.
Re: Independant testers
Good start. I may be able to dedicate a few cores if enough interest builds. Still don't know how to handle that many versions of IvanHoe, perhaps only those directly submitted by the author with settings should be tested.
I for one would not be able to test daily compiles.
I for one would not be able to test daily compiles.