Houdini Is Top Rated Chess Engine

[ernest]

Also at move 70, with FEN: 4N3/6k1/p1p5/P1P3P1/8/5r2/3K4/8 b - - 3 70

70... Kg6? (which was played by Houdini) only draws, while 70... Kf7! is the only move to win.

Actually, that was precisely BB's analysis!
It is to be noted that with infinite analysis, 2 cores @3GHz, 512MB hash,
with Gaviota TB, 70... Kf7! replaces 70... Kg6? only after 2min 27s
without Gaviota TB, only after 10min 34s
Next, I will try to test that with 1 core only (to prevent multicore randomness).

infinite analysis, 2 cores @3GHz, 512MB hash, GTB
Analysis by Houdini 1.5 w32 gtb:
1...Kg6 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb4 Rxc5 5.Kd3 Kxg5 6.Kd4 Rc1 7.Ke5 Kg4 8.Nd3 Rf1 9.Ke4 Kg5 10.Ne5 Rf6 11.Kd4 Rd6+ 12.Kc4 Kf4 13.Nd3+ Ke3 14.Nc5 Rh6 15.Kc3 Rh4 16.Nd3 Rh1 17.Nc5 Rh6 18.Kc4 Rd6 19.Kc3 Rg6 20.Kc4 Rd6 21.Kc3
-+ (-1.52) Depth: 27/45 00:00:29 129mN, tb=1193
1...Kg6 2.Nc7
-+ (-1.43 --) Depth: 28/47 00:01:09 281mN, tb=3571
1...Kg6 2.Nc7
-/+ (-1.27 --) Depth: 28/47 00:01:50 422mN, tb=4803
1...Kf7
-+ (-1.60 !) Depth: 28/70 00:02:27 551mN, tb=6755
1...Kf7
-+ (-1.76 !) Depth: 28/70 00:02:40 603mN, tb=7650
1...Kf7
-+ (-2.14 !) Depth: 28/74 00:03:22 777mN, tb=9074
1...Kf7
-+ (-3.13 !) Depth: 28/74 00:05:50 1404mN, tb=15532
1...Kf7 2.Ke2 Ra3 3.Nd6+ Ke6 4.Nb7 Rc3 5.Kd2 Rg3 6.Nd8+ Kd5 7.Nf7 Kxc5 8.Ke2 Kd5 9.Kf2 Rg4 10.Kf3 Rg1 11.Kf2 Rd1 12.g6 Ke6 13.Ng5+ Kf6 14.Ne4+ Kxg6 15.Nc5 Rd5 16.Nxa6 Rxa5 17.Nb4 c5 18.Nd5 Kf5 19.Ke2 Ra3 20.Kd2 Ke5 21.Ne3 Rb3 22.Ke2 Kd4 23.Nf5+ Ke4 24.Nd6+ Ke5 25.Nf7+ Kd4 26.Kd2 c4 27.Ng5 c3+ 28.Kc1 Rb2
-+ (-3.24) Depth: 28/74 00:10:54 2660mN, tb=25914
1...Kf7 2.Ke2 Ra3 3.Nd6+ Ke6 4.Nb7 Rc3 5.Kd2 Rg3 6.Nd8+ Kd5 7.Nf7 Kxc5 8.Ke2 Kd5 9.Kf2 Rg4 10.Kf3 Rg1 11.Kf2 Rd1 12.g6 Ke6 13.Ng5+ Kf6 14.Ne4+ Kxg6 15.Nc5 Rd5 16.Nxa6 Rxa5 17.Nb8 c5 18.Nd7 Kf7 19.Ke3 Ke6 20.Nb6 Ra6 21.Nc4 Kd5 22.Kd3 Re6 23.Kc3 Rg6 24.Ne3+ Kd6 25.Nd1 Rg1 26.Ne3 Rg3 27.Kd2 Kd7 28.Nf5 Rh3 29.Ne3 Kc6 30.Nf5 c4 31.Kc2 Kd5 32.Ne7+ Kc5 33.Nf5 Kb4 34.Nd4 Rh2+ 35.Kc1 c3
-+ (-3.25) Depth: 29/74 00:11:55 2917mN, tb=28676
1...Kf7 2.Ke2 Ra3 3.Nd6+ Ke6 4.Nb7 Rc3 5.Kd2 Rg3 6.Nd8+ Kd5 7.Nf7 Kxc5 8.Ke2 Kd5 9.Kf2 Rg4 10.Kf3 Rg1 11.Kf2 Rd1 12.g6 Ke6 13.Ng5+ Kf6 14.Ne4+ Kxg6 15.Nc5 Rd5 16.Nxa6 Rxa5 17.Nb8 c5 18.Ke3 Kf6 19.Nd7+ Ke6 20.Nb6 Ra6 21.Nc4 Kd5 22.Kd3 Re6 23.Kc3 Rg6 24.Ne3+ Kd6 25.Nd1 Rg1 26.Ne3 Rg3 27.Kd2 Kd7 28.Nf5 Rh3 29.Ne3 Kc6 30.Nf5 c4 31.Kc2 Kd5 32.Ne7+ Kc5 33.Nf5 Kb4 34.Nd4 Rh2+ 35.Kc1
-+ (-3.25) Depth: 30/74 00:14:18 3508mN, tb=34165

infinite analysis, 2 cores @3GHz, 512MB hash, no GTB
1...Kg6 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb4 Rxc5 5.Kd3 Rc1 6.Na2 Rd1+ 7.Kc4 Ra1 8.Nb4 Rc1+ 9.Kb3 Kxg5 10.Na2 Rb1+ 11.Kc4 Kf6 12.Nc3 Rc1 13.Kd4 Ke6 14.Ne4 Rd1+ 15.Kc4 Rd5 16.Nf2 Rf5 17.Ne4 Ke5 18.Nc5 Rf4+ 19.Kd3 Kd6
-+ (-1.54) Depth: 28/45 00:00:52 230mN
1...Kg6 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb4 Rxc5 5.Kd3 Rc1 6.Na2 Rd1+ 7.Kc4 Ra1 8.Nb4 Rc1+ 9.Kb3 Kxg5 10.Na2 Rb1+ 11.Kc4 Kf6 12.Nc3 Rc1 13.Kd4 Ke6 14.Ne4 Rd1+ 15.Kc4 Rd5 16.Nf2 Ke5 17.Nh3 Kd6 18.Nf2 Rc5+ 19.Kd4 Re5 20.Ng4 Rd5+ 21.Kc3 Rc5+ 22.Kd4 Ra5 23.Nf6 Ra4+ 24.Kd3 c5 25.Ne4+ Kc6
-+ (-1.54) Depth: 29/46 00:01:16 333mN
1...Kg6 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb8 Rxc5 5.Kd3 Kxg5 6.Kd4 Rc1 7.Ke5 Kg4 8.Ke4 Re1+ 9.Kd4 Rd1+ 10.Kc5 Rc1+ 11.Kd6 c5 12.Kd5 Kf3 13.Nc6 Ke2 14.Ne5 Rc3 15.Ng4 Kd2 16.Ne5 Kc2 17.Nc4 Kb3 18.Na5+ Ka4 19.Nc6 c4 20.Kd4 Rc2
-+ (-1.51) Depth: 30/52 00:02:25 639mN
1...Kg6
-+ (-1.59 !) Depth: 31/52 00:03:14 847mN
1...Kg6 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb8 Rxc5 5.Kd3 Kxg5 6.Kd4 Rc1 7.Ke5 Kg4 8.Ke4 Re1+ 9.Kd4 Rd1+ 10.Kc5 Rc1+ 11.Kd6 c5 12.Kd5 Kf3 13.Nc6 Ke2 14.Ne5 Rc3 15.Kd6 Kd1 16.Kc6 Kd2 17.Kd6 Kc2 18.Kd5 Kb3 19.Nc6 c4 20.Na5+ Kb4 21.Nc6+ Ka4 22.Nd4 Ka5 23.Nc6+ Kb6 24.Kd4 Rg3
-+ (-1.60) Depth: 31/52 00:03:51 1018mN
1...Kg6
-+ (-1.69 !) Depth: 32/52 00:04:30 1190mN
1...Kg6
-+ (-1.87 !) Depth: 32/52 00:04:57 1310mN
1...Kg6 2.Nc7
-+ (-1.52 --) Depth: 32/75 00:06:58 1855mN
1...Kf7
-+ (-2.27 !) Depth: 32/75 00:10:34 2782mN
1...Kf7 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb8 Rxc5 5.Ke3 Ke7 6.g6 Kd6 7.Kf4 Rc2 8.Kf5 Rg2 9.Kf6 Rf2+ 10.Kg5 c5 11.Na6 c4 12.g7 Ke7 13.g8N+ Ke6 14.Nh6 c3 15.Nb4 Rb2 16.Nd3 c2 17.Ng4 Kd5 18.Kf4 Kd4 19.Nc1 Rb1 20.Ne2+ Kd3 21.Kf3 Kd2 22.Kf2 Rb8 23.Ne5 Rf8+ 24.Nf3+ Kd3 25.Nc1+ Ke4 26.Ke2 Rxf3 27.Kd2 Rf2+ 28.Kc3 Ke3 29.Nb3 Rh2 30.Nc1 Rg2 31.Nb3 Rf2 32.Kb2 Kd3 33.Nc5+ Kc4 34.Ne4 Re2 35.Nf6 Kd3
-+ (-3.25) Depth: 32/80 00:25:53 6853mN

[BB+]

4N3/6k1/p1p5/P1P3P1/8/5r2/3K4/8 b - - 3 70
IvanHoe, with all relevant "BlockedTripleBases", default parameters for "weak probing", and "BulkLoad" of 345Z into memory:

Code: Select all

info depth 7
info multipv 1 time 685 nodes 1739 nps 2000 tbhits 309 score cp 164 depth 7 pv g7g6 e8c7 f3f2 d2d3 f2a2 c7a6 a2a5 a6b4 a5c5
info multipv 1 time 826 nodes 2245 nps 2000 tbhits 346 score cp 172 lowerbound depth 7 pv g7f7 e8f6 f7g6
info multipv 1 time 1033 nodes 3020 nps 2000 tbhits 421 score cp 236 depth 7 pv g7f7 d2e2 f3a3 e8f6 a3a5 f6e4 f7g6
[...]
info multipv 1 time 2807 nodes 70453 nps 25000 tbhits 8358 score cp 303 depth 12 pv g7f7 e8c7 f3a3 d2c2 a3a5 c2b3 a5a1 b3c4 a6a5 c4b3 a1c1 c7a6 f7g6 b3b2
[...]
info multipv 1 time 3442 nodes 735260 nps 213000 tbhits 95098 score cp 872 depth 15 pv g7f7 e8c7 f3a3 d2c2 a3a5 c2b3 a5c5 g5g6 f7e7 b3a4 c5g5 c7a8 g5g6 a8b6 g6g4 a4a5 e7e6 b6c8 g4g5 a5b6 c6c5

So it gets it in a second, though with pathetic NPS (I think the first 5 ply are "strong probe"). With 4-piece only, I gave up after a minute, with Kf7 still seen as best.

ComStock rejected Kg6 in about 2s with the bases:

Code: Select all

info depth 8 seldepth 11 multipv 1 score cp 266 time 1410 nodes 6208 nps 4402 tbhits 1402 pv g7g6 e8c7 f3a3 c7a6 a3a2 d2d3 a2a5 a6b4 a5c5 d3d4 c5c1 
info depth 8 seldepth 9 multipv 1 score cp 408 time 1815 nodes 9984 nps 5500 tbhits 1763 pv g7f7 e8c7 f3a3 d2e2 a3a5 e2e3 a5c5 e3f4 a6a5
[...]
info depth 14 seldepth 19 multipv 1 score cp 606 time 3214 nodes 212524 nps 66124 tbhits 36462 pv g7f7 e8c7 f3a3 d2e2 a3a5 c7a8 a5c5 g5g6 f7g6 a8b6 a6a5 b6a4 c5e5 e2d3 e5g5 d3d2 g5g2 d2e3 g6f5 

But Stockfish already likes Kf7 in 0.25s w/o any bases:

Code: Select all

info depth 15 seldepth 10 multipv 1 score cp 307 upperbound time 126 nodes 229617 nps 1822357 tbhits 0 pv g7g6 e8c7 f3a3 c7e6 a3a5 d2c3 g6f5 e6d4 f5g5 c3b4 
info depth 15 seldepth 10 multipv 1 score cp 282 upperbound time 130 nodes 247793 nps 1906100 tbhits 0 pv g7g6 e8c7 f3a3 c7e6 a3a5 d2c3 g6f5 e6d4 f5g5 c3b4 
info depth 15 seldepth 21 multipv 1 score cp 278 time 150 nodes 356868 nps 2379120 tbhits 0 pv g7g6 e8c7 f3a3 c7e6 a3a5 d2c3 a5a3 c3c4 a3a4 c4c3 a4g4 e6d4 g4g5 c3c4 g5g4 c4c3 g6f6 d4c6 f6e6 c6b4 a6a5 
info depth 16
info depth 16 seldepth 26 multipv 1 score cp 311 time 210 nodes 609032 nps 2900152 tbhits 0 pv g7g6 e8c7 f3a3 c7e6 a3a5 d2c3 a5b5 c3c4 a6a5 e6d4 b5b4 c4d3 b4d4 d3d4 g6g5 d4d3 g5f4 d3c3 f4e5 c3b3 e5d5 b3a4 d5c5 a4b3 a5a4 b3a3 
info depth 16 seldepth 13 multipv 1 score cp 331 lowerbound time 252 nodes 678804 nps 2693666 tbhits 0 pv g7f7 e8c7 f3a3 c7a6 a3a5 a6b4 a5c5 d2d3 f7g6 d3d4 c5c1 b4d3 c1d1
[...]
info depth 30 seldepth 14 multipv 1 score cp 501 time 18030 nodes 106579171 nps 5911213 tbhits 0 pv g7f7 e8d6 f7g6 d6c8 g6g5 c8a7 f3f5 d2d3 f5c5 d3d4 c5d5 d4c4 d5a5 a7c6 

With bases, ComStock and IvanHoe both also see a draw (0.00) in maybe ~5s after Kg6.

[ernest]

4N3/6k1/p1p5/P1P3P1/8/5r2/3K4/8 b - - 3 70

Here is my test Houdini 1.5 with 1 core only (to prevent multicore randomness):

with Gaviota TB, 70... Kf7! replaces 70... Kg6? only after 11min
without Gaviota TB, only after a dismal 3hours

infinite analysis, 1 core @3GHz, 512MB hash, GTB
Analysis by Houdini 1.5 w32 gtb:
1...Kg6
-+ (-1.60 !) Depth: 28/54 00:02:33 358mN, tb=3240
1...Kg6 2.Nc7 Kxg5 3.Nxa6 Rb3 4.Kc2 Rb5 5.Kc3 Rxa5 6.Nb4 Rxc5+ 7.Kd4 Rc1 8.Ke5 Kg4 9.Nd3 Rf1 10.Ke4 Kg5 11.Kd4 Rd1 12.Ke4 Kf6 13.Nb2 Rd5 14.Nd3 Ke7 15.Ne5 Kd6 16.Nc4+ Ke6 17.Nb6 Re5+ 18.Kd3 Rb5 19.Nc4 Rh5 20.Nb6 Rh4 21.Nc4 c5 22.Nb6 Rd4+ 23.Kc3 Rg4 24.Kd3
-+ (-1.60) Depth: 28/54 00:02:35 363mN, tb=3282
1...Kg6 2.Nc7 Kxg5 3.Nxa6 Rb3 4.Nc7 Kf6 5.a6 Ra3 6.Kc2 Ke7 7.Kb2 Ra5 8.Kc3 Kd7 9.Na8 Rxa6 10.Nb6+ Ke6 11.Kb3 Ra7 12.Kc4 Ra1 13.Kd4 Rd1+ 14.Kc3 Ke5 15.Kc2 Ra1 16.Kd3 Rb1 17.Nd7+ Kd5 18.Kc3 Rd1 19.Nf6+ Kxc5 20.Ne4+ Kd5 21.Nf6+ Ke5 22.Ng4+ Kd6 23.Nf6 c5 24.Ne4+ Kc6 25.Ng3 Rd4 26.Nf5
-+ (-1.60) Depth: 29/54 00:02:56 413mN, tb=3936
1...Kg6 2.Nc7
-+ (-1.52 --) Depth: 30/54 00:04:10 582mN, tb=5557
1...Kg6 2.Nc7
-/+ (-1.34 --) Depth: 30/54 00:06:10 850mN, tb=7068
1...Kg6 2.Nc7
-/+ (-0.96 --) Depth: 30/71 00:09:50 1333mN, tb=8757
1...Kf7 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb8 Rxc5 5.Ke3 Ke7 6.g6 Kd6 7.Kf4 Rc2 8.Kf5 Rg2 9.Kf6 Rf2+ 10.Kg5 c5 11.Na6 c4 12.g7 Ke7 13.g8N+ Ke6 14.Nh6 c3 15.Nb4 Rb2 16.Nd3 Kd5 17.Kf4 Kd4 18.Nc1 c2 19.Nf5+ Kc3 20.Ke3 Rb1 21.Ne2+ Kb2 22.Nd6
-/+ (-1.30) Depth: 30/71 00:11:08 1507mN, tb=10856 (2256 kN/s)
1...Kf7
-/+ (-1.38 !) Depth: 31/71 00:11:53 1607mN, tb=11637
1...Kf7
-+ (-1.55 !) Depth: 31/71 00:12:19 1666mN, tb=12309
1...Kf7
-+ (-1.92 !) Depth: 31/75 00:14:45 1998mN, tb=14978
1...Kf7
-+ (-2.91 !) Depth: 31/81 00:22:08 3027mN, tb=29413
1...Kf7 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb8 Rxc5 5.Ke3 Ke7 6.g6 Kd6 7.Kf4 Rc2 8.Kf5 Rg2 9.Kf6 Rf2+ 10.Kg5 c5 11.Na6 c4 12.g7 Ke7 13.g8N+ Ke6 14.Nh6 c3 15.Nb4 Rb2 16.Nd3 c2 17.Ng4 Kd5 18.Kf4 Kd4 19.Nc1 Rb1 20.Ne2+ Kd3 21.Kf3 Kd2 22.Kf2 Rb8 23.Ne5 Rf8+ 24.Nf3+ Kd3 25.Nc1+ Ke4 26.Ke2 Rxf3 27.Kd2 Rf2+ 28.Kc3 Ke3 29.Nb3 Rh2 30.Nc1 Rd2
-+ (-3.42) Depth: 31/81 00:37:55 5231mN, tb=49529

infinite analysis, 1 core @3GHz, 512MB hash, no GTB
Analysis by Houdini 1.5 w32:
1...Kg6
-+ (-1.77 !) Depth: 34/57 00:15:27 2212mN
1...Kg6 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb4 Rxc5 5.Kd3 Rc1 6.Na2 Rd1+ 7.Kc4 Ra1 8.Nb4 Rc1+ 9.Kb3 Kxg5 10.Na2 Rb1+ 11.Kc2 Rb6 12.Kc3 Kf4 13.Kc4 Ke5 14.Nc3 Rb2 15.Nd1 Rb1 16.Nc3 Re1 17.Kd3 Kd6 18.Ne2 Rd1+ 19.Ke3 c5 20.Nc3 Rc1 21.Nb5+ Kc6
-+ (-1.77) Depth: 34/57 00:19:54 2879mN
1...Kg6 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb4 Rxc5 5.Kd3 Rc1 6.Na2 Rd1+ 7.Kc4 Ra1 8.Nb4 Rc1+ 9.Kb3 Kxg5 10.Na2 Rb1+ 11.Kc2 Rb6 12.Kc3 Kf4 13.Kc4 Ke5 14.Nc3 Rb2 15.Nd1 Rb1 16.Nc3 Rc1 17.Kd3 Kd6 18.Ne2 Rd1+ 19.Kc3 c5 20.Nf4 Ra1 21.Kb3 Kc6 22.Ne6 Ra5
-+ (-1.77) Depth: 35/58 00:32:46 4779mN
1...Kg6 2.Nc7 Ra3 3.Nxa6 Rxa5 4.Nb4 Rxc5 5.Kd3 Rc1 6.Na2 Rd1+ 7.Kc4 Ra1 8.Nb4 Rc1+ 9.Kb3 Kxg5 10.Na2 Rb1+ 11.Kc2 Rb6 12.Kc3 Kf4 13.Kc4 Ke5 14.Nc3 Rb2 15.Na4 Rg2 16.Nb6 Kd6 17.Nc8+ Kd7 18.Nb6+ Kc7 19.Na4 Kd6 20.Nb6 Rg4+ 21.Kc3 Rg3+ 22.Kd4 c5+ 23.Ke4 Rg4+ 24.Kd3 Rd4+ 25.Kc3 Rh4 26.Kd3 Rb4
-+ (-1.77) Depth: 36/61 00:51:41 7630mN
1...Kg6
-+ (-1.87 !) Depth: 37/62 01:10:03 10375mN
1...Kg6 2.Nc7
-+ (-1.68 --) Depth: 37/67 02:23:26 21252mN
1...Kf7
-+ (-2.05 !) Depth: 37/81 03:00:24 26661mN (2463 kN/s)
1...Kf7
-+ (-2.49 !) Depth: 37/81 03:51:41 34173mN
1...Kf7
-+ (-3.66 !) Depth: 37/81 06:26:54 57778mN

[Adam Hair]

Just an example so that you can realize it. You have a book with 10% of "biased" position where both engines win.
Lets suppose the result of the match was 40+/10-/50=, so 50% draw ratio. It's 65% or 108 Elo difference.
Now if you remove 10% of biased positions the real unbiased result would be 35+/5-/50= which is 66.7% or 121 Elo difference.
So you would get 13 Elo mean error!!!

Thank you very much for the example! I had never thought about that before!

Of course,I presume you meant it reduces the estimation of the Elo difference between the engines, seeing as
how the real Elo difference is unknown.

I, of course, was speaking about the bias of the individual positions. That bias does not go away, but by playing from
both sides the unknown effects of the biases on the estimation are mitigated, to some degree. In fact, the individual biases are transformed into systematic bias, as you noted above.

[BB+]

[quote[With 4-piece only [IvanHoe], I gave up after a minute, with Kf7 still seen as best.[/quote]That was a typo -- Kg6 was still seen as best, so IvanHoe only solved the position when using the right RobboBases.

Here is my test Houdini 1.5 with 1 core only (to prevent multicore randomness):
with Gaviota TB, 70... Kf7! replaces 70... Kg6? only after 11min
without Gaviota TB, only after a dismal 3hours

Yes, Stockfish seems to do much better here. My guess is that "in general" you want your King next to the opponent's passed g-pawn (so Kg6), but here you want to rush your King over to support the c-pawn (and/or ward off White's King), and can easily pick up the g-pawn with the Rook. Maybe it shows how difficult a "rule-based" approach to endgames can be.

[Prima]

BB+,

I downloaded the ComStock's source but no executive file. Where can we get this ComStock UCI from?

[BB+]

I downloaded the ComStock's source but no executive file. Where can we get this ComStock UCI from?

I don't know. I am using Linux and just typed "make", and there were no problems.