Creating a new (and independent) rating list

[Andrew]

It largely impractical, I would even say impossible. You would need to have not only the same CPU, but the same memory (same size, same brand, same type, same latency), same motherboard, same GUI, same OS, even same settings of the OS (same services running in the background, same startup processes), because all of these things do impact performance.

Certainly not impossible. I'm fine with testing being conducted by a small number of people.

[kingliveson]

You don't have to go through all that trouble since we have Bob who was so generous to make (for quite some time) his 40k starting positions available on his website for anyone to use.
And I must say I do trust his choice of startup positions.

40k or 4k? Anyways, EPD positions are not best for ratings list because the games become useless for book making. PGN files are better.

rnbq1rk1/ppppppbp/5np1/8/2PP4/6P1/PP2PPBP/RNBQK1NR w KQ - 3 5



[Event "KLO 250 ECO (A00-E97) Variations"]
[Site "?"]
[Date "2009.10.05"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]
[ECO "E60"]
[WhiteElo "3000"]
[BlackElo "3000"]
[PlyCount "8"]
[EventDate "2009.??.??"]

1. d4 Nf6 2. c4 g6 3. g3 Bg7 4. Bg2 O-O *

[Sentinel]

40k or 4k? Anyways, EPD positions are not best for ratings list because the games become useless for book making. PGN files are better.

Ups. It's 4k. However, since all the positions are 22 plies deep, it should be quite easy with a big book to generate all the PGNs. I'm also sure that Bob has them somewhere.

[BB+]

Ups. It's 4k. However, since all the positions are 22 plies deep, it should be quite easy with a big book to generate all the PGNs. I'm also sure that Bob has them somewhere.

It should be a day's (or week's) programming exercise to write something to go through a large database of games (like JunkBase) and find something like: all positions that were reached 100 times, which have no descendant that was reached 100 times. This would match the popularity constraint. You would then have to have some sort of mechanism for tossing out positions which were too one-sided, or too easily led to draws (like a repetition right out of book). I see no real reason to have a "ply limit" -- in some cases, theory goes quite deep.

[Sentinel]

I see no real reason to have a "ply limit" -- in some cases, theory goes quite deep.

Engines should be tested in all the elements of gameplay, not only late middlegame and endgame, or more technically said, all game phases should have equal probability of occurrence. Starting positions should not:
a) change material from default (excluding pawns)
b) change pawn structure too much
c) change king safety too much

You achieve all these in a very simple way by limiting ply number.

On the other hand it is good that result variance from a starting position is as high as possible, and result mean as close to 50% as possible. And this is easy to test. Take a very strong engine, play the starting position many time against itself (MC), exclude full duplicates, if result is more than 55-58% (for white/black) drop out the position. If draw rate is higher than 50-67% drop out position.

[BB+]

Starting positions should not:
a) change material from default (excluding pawns)
b) change pawn structure too much
c) change king safety too much

Do you eliminate lines that sac a piece for pawns, also involving (b) and (c)? Many Semi-Slav lines would seem not to meet your criteria.

[Sentinel]

Starting positions should not:
a) change material from default (excluding pawns)
b) change pawn structure too much
c) change king safety too much

Do you eliminate lines that sac a piece for pawns, also involving (b) and (c)? Many Semi-Slav lines would seem not to meet your criteria.

This is a dilemma I had when making starting positions. Eventually, having material reduced for one minor might be acceptable. However, two would be already too much IMO.

[Hood]

When it will be at the end ?
Sth not extra is better then nth extra.

Better is enemy of good. Lets start with sth then improve. Experience is the best teacher.

[kingliveson]

I hope this idea is still alive. We need volunteers. Athlon 64 x2 4200+ will be dedicated by me.

[aiorla]

As I said before my i5-750 (x64) will be mainly dedicated to this list