value of LMR and null-move

[Rebel]

Thanks for the efforts and figures Bob. Very interesting.
'
Ed

[hyatt]

Morning update, +63 at present for LMR over non-LMR. Going slowly, but going:

Code: Select all

Rank Name                  Elo    +    - games score oppo. draws
   1 Stockfish 1.8 64bit   2826   13   13  1991   80%  2586   32% 
   2 Stockfish 1.8a 64bit  2763   12   12  1933   69%  2610   35% 
   3 Toga2                 2572   23   23   609   19%  2795   28% 
   4 Glaurung 2.2          2543   24   24   631   16%  2795   27% 
   5 Fruit 2.1             2469   27   27   628   11%  2796   18% 
   6 Glaurung 1.1 SMP      2427   27   27   746    9%  2795   14% 

This poking along is a pain, but we can't power up all the cluster hardware due to A/C issues. We have seen the computer room spike way past 135F and it does not take very long, when the A/C fails.

[hyatt]

Big pain in the a**. This is slow.

now +64 after almost 2400 games for each...

Code: Select all

Rank Name                  Elo    +    - games score oppo. draws
   1 Stockfish 1.8 64bit   2827   12   12  2387   80%  2590   33% 
   2 Stockfish 1.8a 64bit  2763   11   11  2355   68%  2612   35% 
   3 Toga2                 2574   21   21   764   20%  2795   28% 
   4 Glaurung 2.2          2545   21   21   791   16%  2795   27% 
   5 Fruit 2.1             2463   25   25   782   11%  2796   17% 
   6 Glaurung 1.1 SMP      2429   26   26   803    9%  2795   14% 

[hyatt]

Going to be out of town tomorrow and Saturday. If I have internet access, I will post final results, otherwise here is the current status:

Code: Select all

Rank Name                  Elo    +    - games score oppo. draws
   1 Stockfish 1.8 64bit   2826   11   11  2785   79%  2590   33% 
   2 Stockfish 1.8a 64bit  2766   10   10  2780   69%  2610   35% 
   3 Toga2                 2579   19   19   912   20%  2796   29% 
   4 Glaurung 2.2          2538   20   20   919   16%  2796   26% 
   5 Fruit 2.1             2465   23   23   922   11%  2796   17% 
   6 Glaurung 1.1 SMP      2426   24   24   946    9%  2796   14% 

[tom]

So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..

Any theories on this?

- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?

It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily

Cheers, all.

Regards,
Tom

[Hood]

So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..

Any theories on this?

- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?

It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily

Cheers, all.

Regards,
Tom

Hi,

the green question suggestion

Shall be all the programms sensible in the same way to the part(element) of their functionality ?

I suppose that not. It shall depend on what role is playing that element in the whole construction.
The programm is the engineering construction and its strength could come from differrent parts.
If the constructions are similar the sensitivity to the element shall be similar if the designs are differrent then not.

Rgds
Hood

[tom]

So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..

Any theories on this?

- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?

It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily

Cheers, all.

Regards,
Tom

I'll try and run a test on my prog, so we get another view - value of LMR in a much weaker engine! It would be handy to have a cluster, you know..

[hyatt]

So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..

Any theories on this?

- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?

It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily

Cheers, all.

Regards,
Tom

I am not sure, at the present. I am going to try to go back and figure out which version I tested last year when I got +80, and re-run the test to sanity-check things. But in any case, current version is significantly more aggressive in LMR, using 2 almost everywhere. And the pruning stuff is more aggressive as well. How they "add up" and influence each other is unknown.

[tom]

So the big question is why LMR only gives 65 Elo for SF, but is worth closer to 200 elo to crafty..

Any theories on this?

- better lmr implementation in crafty?
- other fwd pruning in SF means lmr benefit is reduced?
- some unexpected, emergent behaviour, e.g. unexpectedly good interaction between crafty lmr and crafty's eval?

It would be interesting to know the "bounds" for lmr. Could a really good implementation give 300 elo? I guess I really bad implementation could lose 300 elo, easily

Cheers, all.

Regards,
Tom

I am not sure, at the present. I am going to try to go back and figure out which version I tested last year when I got +80, and re-run the test to sanity-check things. But in any case, current version is significantly more aggressive in LMR, using 2 almost everywhere. And the pruning stuff is more aggressive as well. How they "add up" and influence each other is unknown.

Well, a few test games (60 moves in 1 min) suggest that I'm getting somewhere around 50 Elo from my LMR implementation. Room for improvement, then? Wonder what I've missed..

Code: Select all

Rank Name               Elo    +    - games score oppo. draws
   1 Rybka 2.2 32 bit   241   34   32   320   86%   -81   14%
   2 Ruffian 1.0.5       44   28   27   320   57%   -15   18%
   3 fran015x          -115   27   28   320   33%    39   22%
   4 fran015x_nolmr    -170   28   29   320   24%    57   19%

[Uly]

I don't think it will measure correctly with engines so far apart in strength.