Opening set for engine testing

[lucasart]

This set of openings is designed especially for engine testing:

• 29692 unique 5-ply openings
• extremely diverse, far away from known theory for the most part
• all tested and shown to be reasonably balanced

I post this for anyone interested:

• engine developpers who want to increase their testing resolution.
• engine testers who want to reduce draw rates, and see unorthodox games.

And also for my own record, as I am good at losing files, so uploading saved my quite a few times...

[User923005]

This is a really interesting experiment.
According to my analysis, there are 1134 positions with a score of +/- 50 centipawns or more, and 151 with a score of +/- 75 centipawns or more.

These are the positions with a score of at least one full pawn:
rnbqk1nr/pppp1ppp/8/2b1p3/1P2P2P/8/P1PP1PP1/RNBQKBNR b KQkq - acd 33; acs 380; bm Bxb4; ce 110; pm Bxb4; pv Bxb4 c3 Ba5 Na3 Nf6 Nc4 Bb6 Qa4 O-O Ba3 d6 Nxb6 cxb6 f3 Qc7 Bc4 Nc6 Ne2 a6 Qc2 b5 Bb3 Be6 Bxe6 fxe6 O-O Rac8 Rab1 Rfd8 Qd1 d5 Ng3 Na5 exd5 Nc4;
rnbqkb1r/pppp1ppp/4p2n/8/1P6/2N4P/P1PPPPP1/R1BQKBNR b KQkq - acd 33; acs 287; bm Bxb4; ce 110; pm Bxb4; pv Bxb4 e4 O-O Rb1 c5 Nf3 f5 e5 Nf7 Be2 Bxc3 dxc3 Nc6 Bf4 Qc7 Qd2 h6 Bh2 g5 Bb5 b6 Rd1 a6 Bxc6 dxc6 Qe3 Rd8 Rxd8+ Qxd8 c4 Qc7 O-O Bb7 Rd1 Rd8 Rxd8+ Qxd8 Qd3 Qc7;
r1bqkbnr/ppp1pppp/n7/3p4/6P1/3P4/PPPNPP1P/R1BQKBNR b KQkq - acd 32; acs 532; bm Bxg4; ce 106; pm Bxg4; pv Bxg4 Ngf3 e6 Rg1 Bh5 e3 Nf6 Be2 g6 b3 c5 Bb2 Bg7 a3 O-O c4 Nd7 Bxg7 Kxg7 cxd5 exd5 d4 Rc8 Kf1 Qf6 Bb5 Qe7 Kg2 cxd4 exd4 Nc7 Bxd7 Bxf3+ Qxf3;
rnbqk1nr/ppppppbp/6p1/8/4PPP1/8/PPPP3P/RNBQKBNR b KQkq - acd 33; acs 624; bm e5; ce 104; pm e5; pv e5 Qe2 Nc6 Nf3 exf4 Nc3 d6 h3 g5 Qg2 h6 Bc4 Bxc3 bxc3 Nf6 d3 Qe7 Kd1 Be6 Bb5 O-O c4 Ne5 Rb1 Nxf3 Qxf3 c6 Ba4 Rab8 Bb2 b5 cxb5 cxb5 Bxf6 Qxf6 Rxb5 Bxa2;
rnbqk1nr/ppppbppp/4p3/6P1/8/4P3/PPPP1P1P/RNBQKBNR b KQkq - acd 32; acs 276; bm Bxg5; ce 103; pm Bxg5; pv Bxg5 d4 d5 Bg2 Be7 Ne2 Nf6 c4 O-O O-O b6 b3 c6 Nd2 Bb7 e4 dxe4 Nxe4 Nxe4 Bxe4 Bd6 Qd3 Qh4 h3 Nd7 Bd2 a5 Bg2 Rfd8 Rae1 Nf6 f4 Bc5 Qe3 Bb4 Bc3;
rn1qkbnr/ppp1pppp/3p4/5b2/1P4P1/2N5/P1PPPP1P/R1BQKBNR b KQkq - acd 33; acs 307; bm Bxg4; ce 101; pm Bxg4; pv Bxg4 Nd5 e5 c4 Ne7 Ne3 Be6 Nf3 Nf5 d3 Be7 Nd5 O-O Rg1 Nd7 Bd2 Bf6 Nxf6+ Nxf6 Bc3 Re8 b5 h6 Nd2 Bd7 Bg2 d5 cxd5 Bxb5 a4 Ba6 Qb3 Nd4 Bxd4 exd4;
rn1qkbnr/ppp1pppp/3pb3/8/1P4P1/5N2/P1PPPP1P/RNBQKB1R b KQkq - acd 32; acs 509; bm Bxg4; ce 100; pm Bxg4; pv Bxg4 Bg2 Nf6 c4 e5 O-O Nbd7 d4 c6 Nc3 h6 h3 Be6 d5 cxd5 cxd5 Bf5 Nd2 Be7 e4 Bg6 Bf3 Rc8 Bb2 O-O Re1 Nb6 Re3 Nc4 Nxc4 Rxc4 a3 Qc8 Qf1 a6;

[lucasart]

According to my analysis, there are 1134 positions with a score of +/- 50 centipawns or more, and 151 with a score of +/- 75 centipawns or more.

Which engine ? What tc ?
I used Critter 1.6a and Komodo CCT, because I don't trust Stockfish evaluation.

[lucasart]

These are the positions with a score of at least one full pawn:
rnbqk1nr/pppp1ppp/8/2b1p3/1P2P2P/8/P1PP1PP1/RNBQKBNR b KQkq - acd 33; acs 380; bm Bxb4; ce 110; pm Bxb4; pv Bxb4 c3 Ba5 Na3 Nf6 Nc4 Bb6 Qa4 O-O Ba3 d6 Nxb6 cxb6 f3 Qc7 Bc4 Nc6 Ne2 a6 Qc2 b5 Bb3 Be6 Bxe6 fxe6 O-O Rac8 Rab1 Rfd8 Qd1 d5 Ng3 Na5 exd5 Nc4;

It's a matter of which engine you use. With Critter 1.6a, using 8 threads and 2048 MB Hash, even the worst position according to your analysis is playable:
rnbqk1nr/pppp1ppp/8/2b1p3/1P2P2P/8/P1PP1PP1/RNBQKBNR b KQkq - 0 1

Code: Select all

info multipv 1 depth 23 seldepth 57 nodes 1460703960 time 131717 nps 11089714 hashfull 1000 tbhits 0 score cp 50 pv c5b4 c2c3 b4e7 d2d4 d7d6 g1f3 g8f6 f1d3 e8g8 e1g1 b8c6 d4d5 c6b8 c3c4 b8d7 b1c3 f6g4 f3g5 d7c5 d3e2 h7h6 e2g4 h6g5 h4h5 c8g4 d1g4 d8d7

it's like a kind of Evan's gambit (with some weird moves added).

Edit: with more analysis, it seems that Critter is wrong, and this position is bad indeed.

[User923005]

Note that the depth I used was 33 plies for that position. Whenever I see something suspicious, I harden it with a deeper search.

[lucasart]

Somehow I couldn't open your 7z file. So I only removed the ones you posted above, with abs(ce) >= 100.

[User923005]

I will have another go with the current set