Perft test question

[y62wang]

Hi all,

I've been doing testing on my chess board move generation, and I am not sure how to resolve this particular issue:

My test board configuration is:
FEN r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1

at depth 5, my test result is 15833472 nodes but expected result is 15833292

As I track the problem, it seems every time the move b2a1Q is played, and white makes any valid move. For example:
FEN r3k2r/Pppp1ppp/1b3nbN/nPB5/B1P1P3/q4N2/P2P2PP/qQ3RK1 b kq -
My generator will provide 42 moves, but the testing software i use (roce) gives 36 moves. (no move generated for the queen at a1)
Then I checked this against qperft, which also generates 42 moves at depth 1 for this board. 42 moves definitely seem to be the right result, but if we check the original board "FEN r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1", both qperft and roce generated the correct result of 15833292 nodes. how can this happen? I suspect one of the two engines must be wrong, yet the both generate the correct number of nodes!

At this time, I am blocked to do any further testing because it seems roce is not being very helpful at this time because it obviously not generating valid moves... Any advice would be appreciated.

[User923005]

Code: Select all

F:\project\dcorbit\hqperft-master>hqperft.exe -f "r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq -"  -d 5  --div
HQPerft (c) Richard Delorme - 2017-2018
Bitboard move generation based on (H)yperbola (Q)uintessence & range attacks
Perft setting: no hashing; no bulk counting;
  a b c d e f g h
8 r . . . k . . r 8
7 P p p p . p p p 7
6 . b . . . n b N 6
5 n P . . . . . . 5
4 B B P . P . . . 4
3 q . . . . N . . 3
2 P p . P . . P P 2
1 R . . Q . R K . 1
  a b c d e f g h
w, kq
 c4c5          2145218
 d2d4          2816009
 f3d4          2928923
 b4c5          2027632
 f1f2          2703427
 g1h1          3212083
total   :        15833292 leaves in      5.281 s      2998162 leaves/s

[H.G.Muller]

I know little about Roce, but the normal way to hunt down perft discrepancies is to do a 'split perft', which does not only give the total number of counts for a given root position, but details this per move or per move sequence. Qperft allows splitting at any level; e.g.

Code: Select all

$ ./perft 4 -3 "8/4K3/4P3/8/4k3/8/8/8 w - - 0 1"
 - - - - - - - - - - - -
 - - - - - - - - - - - -
 - - . . . . . . . . - -
 - - . . . . K . . . - -
 - - . . . . P . . . - -
 - - . . . . . . . . - -
 - - . . . . k . . . - -
 - - . . . . . . . . - -
 - - . . . . . . . . - -
 - - . . . . . . . . - -
 - - - - - - - - - - - -
 - - - - - - - - - - - -

Quick Perft by H.G. Muller
Perft mode: No hashing, bulk counting in horizon nodes

perft( 1)=            7 ( 0.000 sec)
perft( 2)=           52 ( 0.000 sec)
2. e7f7 moves =         61 ( 0.000 sec)
2. e7f8 moves =         48 ( 0.000 sec)
2. e7e8 moves =         48 ( 0.000 sec)
2. e7d8 moves =         48 ( 0.000 sec)
2. e7d7 moves =         61 ( 0.000 sec)
2. e7d6 moves =         43 ( 0.000 sec)
2. e7f6 moves =         43 ( 0.000 sec)
perft( 3)=          352 ( 0.000 sec)
3. e7f7 e4f4 moves =         59 ( 0.000 sec)
3. e7f7 e4f5 moves =         41 ( 0.000 sec)
3. e7f7 e4e5 moves =         48 ( 0.000 sec)
3. e7f7 e4d5 moves =         59 ( 0.000 sec)
3. e7f7 e4d4 moves =         63 ( 0.000 sec)
3. e7f7 e4d3 moves =         64 ( 0.000 sec)
3. e7f7 e4e3 moves =         64 ( 0.000 sec)
3. e7f7 e4f3 moves =         64 ( 0.000 sec)
2. e7f7 moves =        462 ( 0.000 sec)
3. e7f8 e4f4 moves =         48 ( 0.000 sec)
3. e7f8 e4f5 moves =         41 ( 0.000 sec)
3. e7f8 e4e5 moves =         42 ( 0.000 sec)
3. e7f8 e4d5 moves =         45 ( 0.000 sec)
3. e7f8 e4d4 moves =         48 ( 0.000 sec)
3. e7f8 e4d3 moves =         48 ( 0.000 sec)
3. e7f8 e4e3 moves =         48 ( 0.000 sec)
3. e7f8 e4f3 moves =         48 ( 0.000 sec)
2. e7f8 moves =        368 ( 0.000 sec)
3. e7e8 e4f4 moves =         48 ( 0.000 sec)
3. e7e8 e4f5 moves =         42 ( 0.000 sec)
3. e7e8 e4e5 moves =         41 ( 0.000 sec)
3. e7e8 e4d5 moves =         42 ( 0.000 sec)
3. e7e8 e4d4 moves =         48 ( 0.000 sec)
3. e7e8 e4d3 moves =         48 ( 0.000 sec)
3. e7e8 e4e3 moves =         48 ( 0.000 sec)
3. e7e8 e4f3 moves =         48 ( 0.000 sec)
2. e7e8 moves =        365 ( 0.000 sec)
3. e7d8 e4f4 moves =         48 ( 0.000 sec)
3. e7d8 e4f5 moves =         45 ( 0.000 sec)
3. e7d8 e4e5 moves =         42 ( 0.000 sec)
3. e7d8 e4d5 moves =         41 ( 0.000 sec)
3. e7d8 e4d4 moves =         48 ( 0.000 sec)
3. e7d8 e4d3 moves =         48 ( 0.000 sec)
3. e7d8 e4e3 moves =         48 ( 0.000 sec)
3. e7d8 e4f3 moves =         48 ( 0.000 sec)
2. e7d8 moves =        368 ( 0.000 sec)
3. e7d7 e4f4 moves =         63 ( 0.000 sec)
3. e7d7 e4f5 moves =         59 ( 0.000 sec)
3. e7d7 e4e5 moves =         48 ( 0.000 sec)
3. e7d7 e4d5 moves =         41 ( 0.000 sec)
3. e7d7 e4d4 moves =         59 ( 0.000 sec)
3. e7d7 e4d3 moves =         64 ( 0.000 sec)
3. e7d7 e4e3 moves =         64 ( 0.000 sec)
3. e7d7 e4f3 moves =         64 ( 0.000 sec)
2. e7d7 moves =        462 ( 0.000 sec)
3. e7d6 e4f4 moves =         53 ( 0.000 sec)
3. e7d6 e4f5 moves =         48 ( 0.000 sec)
3. e7d6 e4d4 moves =         35 ( 0.000 sec)
3. e7d6 e4d3 moves =         57 ( 0.000 sec)
3. e7d6 e4e3 moves =         58 ( 0.000 sec)
3. e7d6 e4f3 moves =         61 ( 0.000 sec)
2. e7d6 moves =        312 ( 0.000 sec)
3. e7f6 e4f4 moves =         35 ( 0.000 sec)
3. e7f6 e4d5 moves =         48 ( 0.000 sec)
3. e7f6 e4d4 moves =         53 ( 0.000 sec)
3. e7f6 e4d3 moves =         61 ( 0.000 sec)
3. e7f6 e4e3 moves =         58 ( 0.000 sec)
3. e7f6 e4f3 moves =         57 ( 0.000 sec)
2. e7f6 moves =        312 ( 0.000 sec)
perft( 4)=         2649 ( 0.000 sec)

There are two ways to use this:
You can only split at the root for the programs you are comparing (add the '-2' argument in qperft), and if the total is different there must be at least one move for which the count is different. You can then repeat the process starting from the position after that move. This way you zoom in on a position that has a different perft(1), and for that you can count the correct value by hand. Often it is also obvious what makes the position special, and which moves are overlooked.

Sometimes this procedure fails, because a split value for a move at depth N does not reproduce as the total value for depth N-1. Such an inconsistency would of course tell you that the program that does that is in error, but if it happens for large N, and the value given for the root at depth N-1 is now correct, it prevent you from zooming in on the error further. In that case the solution is to keep perfting the position that produced the erroneous value at the lowest depth where this still occured, and do progressively deeper splits in order to follow the erroneous branch. This quickly gives unwieldly large lists of move sequences, but by writing those to file, and searching with an editor for the move sequence of interest, it should still be doable.