Designing an analysis friendly Stockfish?

[Jeremy Bernstein]

What about the PA_G and PA_I builds?

I retired them, but I can re-run them against the 2 winners of this tournament. In truth, it's pretty close all around. We're talking about just a few games, but I keep seeing this spread at different time controls. 64-ja (or my 64-bit build, doesn't really matter -- the two are out front) ahead, and the pa builds (doesn't matter which version) lagging a bit behind.

Now, the goal is to create an analysis-friendly Stockfish, not a tournament winner, so a slight dip in strength at fast time controls (3/1 is still pretty fast) isn't really a big deal, but I'm curious about what's causing the dip. It's interesting that the TBs don't make much of a difference so far, either.

Jeremy

[Uly]

Now, the goal is to create an analysis-friendly Stockfish, not a tournament winner

True, I'd like to continue that path on the strongest build though (there are still issues that can be resolved, like making Stockfish remember analysis when one moves to different positions, and to decide if learning would be easy to implement), for analyzing I'm liking the PA_G version (because, it felt clearly better than previous versions, and versions afterwards weren't clearly better than PA_G), but I haven't really used the others much, and I think the new code improvement could be tested if they should be better in theory.

fruity was the one that got me thinking that it's possible to have a build that is both analysis-friendly and a tournament winner

[Prima]

Unfortunately, I haven't been able to test all PA builds including the PA_MC latest build to see the difference.

It would seem that PA_G is stable for its purpose. I do agree with Uly that it would be interesting to see Stockfish's behaviour if the Learning function is implemented - something like Position Learning and Book Learning.

Of course that's easier said than done, on my part.

[Jeremy Bernstein]

Engine                     Score               St              St              St              St              St    S-B
Stockfish-201-64-ja        35,0/60 ··············· =1==101=1=10==0 =0=10====1=11== 1==1=11=0====== 11==1010=0===11  1004,2
Stockfish_201_x64          32,5/60 =0==010=0=01==1 ··············· ==1=0=01======0 =====1=11=11=== ==========1=1=1  939,00
Stockfish_201_64_pamc      29,0/60 =1=01====0=00== ==0=1=10======1 ··············· ==0=1=0=0==1=== =10=0=1==1==001  875,25
Stockfish_201_PAMC_GTB_x64 27,0/60 0==0=00=1====== =====0=00=00=== ==1=0=1=1==0=== ··············· =0=1=0=11======  816,50
Stockfish_201_popcnt_pamc  26,5/60 00==0101=1===00 ==========0=0=0 =01=1=0==0==110 =1=0=1=00====== ···············  811,50

150 of 500 games played

[Jeremy Bernstein]

It's interesting that the TBs don't make much of a difference so far, either.

I just found a position where the TBs should have helped exclude moves, but weren't used for this purpose, causing the engine to walk into a mate that it should have known about. I'm going to add some code for that, as well, and see if that makes some difference...

[Master Om]

Whats the best build so far for analysis.?
I have found something interesting.
scorpio bitbases support will be good.

[fruity]

Now, the goal is to create an analysis-friendly Stockfish, not a tournament winner

True, I'd like to continue that path on the strongest build though (there are still issues that can be resolved, like making Stockfish remember analysis when one moves to different positions, and to decide if learning would be easy to implement), for analyzing I'm liking the PA_G version (because, it felt clearly better than previous versions, and versions afterwards weren't clearly better than PA_G), but I haven't really used the others much, and I think the new code improvement could be tested if they should be better in theory.

fruity was the one that got me thinking that it's possible to have a build that is both analysis-friendly and a tournament winner

I'm currently running a 60,000 games match at 2'', SF default against my final proposal.
40,000 games have been already played and my proposal is +5 Elo ahead.
The only differences compared to what Marco Costalba had posted finally are

(1) I use TT hits for pruning also in qsearch() in PV case
(2) I accept mate(d) values in PV case (if tte->depth() < depth)
(3) I use "inline bool ok_to_use_TT<PvNode>(....)" for clean and fast code.

Code snippets and test result later.

[Jeremy Bernstein]

It's interesting that the TBs don't make much of a difference so far, either.

I just found a position where the TBs should have helped exclude moves, but weren't used for this purpose, causing the engine to walk into a mate that it should have known about. I'm going to add some code for that, as well, and see if that makes some difference...

New builds with proper sorting of tablebase hits. Enjoy.

[fruity]

I'm currently running a 60,000 games match at 2'', SF default against my final proposal.
40,000 games have been already played and my proposal is +5 Elo ahead.
The only differences compared to what Marco Costalba had posted finally are

(1) I use TT hits for pruning also in qsearch() in PV case
(2) I accept mate(d) values in PV case (if tte->depth() < depth)
(3) I use "inline bool ok_to_use_TT<PvNode>(....)" for clean and fast code.

Code snippets and test result later.

Result: +16426 -15522 =28052

function signature

Code: Select all

  template <NodeType PvNode>
  bool ok_to_use_TT(const TTEntry* tte, Depth depth, Value alpha, Value beta, int ply);

function def

Code: Select all

  template <NodeType PvNode>
  inline bool ok_to_use_TT(const TTEntry* tte, Depth depth, Value alpha, Value beta, int ply) {

    Value v = value_from_tt(tte->value(), ply);

    return
    PvNode ?    tte->type() == VALUE_TYPE_EXACT
             && (   tte->depth() >= depth
                 || v >= value_mate_in(PLY_MAX)
                 || v <= value_mated_in(PLY_MAX))

           :    (   tte->depth() >= depth
                 || v >= Max(value_mate_in(PLY_MAX), beta)
                 || v <= Min(value_mated_in(PLY_MAX), alpha))
             && (   ((tte->type() & VALUE_TYPE_LOWER) && v >= beta)
                 || ((tte->type() & VALUE_TYPE_UPPER) && v <= alpha));
  }

function call from search()

Code: Select all

    if (tte && ok_to_use_TT<PvNode>(tte, depth, alpha, beta, ply))
    {
        TT.refresh(tte);
        ss->bestMove = ttMove; // Can be MOVE_NONE
        return value_from_tt(tte->value(), ply);
    }

function call from qsearch()

Code: Select all

    if (tte && ok_to_use_TT<PvNode>(tte, ttDepth, alpha, beta, ply))
    {
        ss->bestMove = ttMove; // Can be MOVE_NONE
        return value_from_tt(tte->value(), ply);
    }

[mcostalba]

Sorry for posting only now, but I have limited net access these days.

Thanks for sharing your results. The most interesting thing is that 2 secs of TC seem to hold and give a realistic result. I would have tested 10K at 10 secs, but more for my prejudicies than for a real reason.

Your patch should be almost the same of what I posted because the impact in qsearch should be minimal, even lower then considering mate scores