Designing an analysis friendly Stockfish?

[gaard]

SF PA_G scores about 15 Elo below SF default at very fast games in my tests. If everything would have been correctly coded in SF PA_G that should not be the case. I think the ok_to_use_tt_PV() function is not quite right. The test for TT hits should rather look like

Code: Select all

bool ok_to_use_TT(const TTEntry* tte, Depth depth, Value alpha, Value beta, int ply) {

    Value v = value_from_tt(tte->value(), ply);

    return (   tte->depth() >= depth
            || v >= Max(value_mate_in(PLY_MAX), beta)
            || v <= Min(value_mated_in(PLY_MAX), alpha))

           && (
        		    ((tte->type() & VALUE_TYPE_LOWER) && v >= beta)
        		 || ((tte->type() & VALUE_TYPE_UPPER) && v <= alpha)
        		 || (tte->type() == VALUE_TYPE_EXACT && v < beta && v > alpha)
        	  );
  }

in both cases, PV and non-PV. Please correct me if I'm wrong. If I change the code appropriately, I get a slightly positive score after 6000 ultra fast games, like expected. Result was: +1921 -1814 =2265 from changed SF PA_G's pov.

I had not touched ok_to_use_TT but I will go ahead and fix this. Was the +1921 -1814 =2265 score from the 20" games?

http://dl.dropbox.com/u/11904592/stockfish_201_PA_I.zip

Here is a new build with the proposed corrections.

[gaard]

SF PA_G scores about 15 Elo below SF default at very fast games in my tests. If everything would have been correctly coded in SF PA_G that should not be the case. I think the ok_to_use_tt_PV() function is not quite right. The test for TT hits should rather look like

Code: Select all

bool ok_to_use_TT(const TTEntry* tte, Depth depth, Value alpha, Value beta, int ply) {

    Value v = value_from_tt(tte->value(), ply);

    return (   tte->depth() >= depth
            || v >= Max(value_mate_in(PLY_MAX), beta)
            || v <= Min(value_mated_in(PLY_MAX), alpha))

           && (
        		    ((tte->type() & VALUE_TYPE_LOWER) && v >= beta)
        		 || ((tte->type() & VALUE_TYPE_UPPER) && v <= alpha)
        		 || (tte->type() == VALUE_TYPE_EXACT && v < beta && v > alpha)
        	  );
  }

in both cases, PV and non-PV. Please correct me if I'm wrong. If I change the code appropriately, I get a slightly positive score after 6000 ultra fast games, like expected. Result was: +1921 -1814 =2265 from changed SF PA_G's pov.

I had not touched ok_to_use_TT but I will go ahead and fix this. Was the +1921 -1814 =2265 score from the 20" games?

http://dl.dropbox.com/u/11904592/stockfish_201_PA_I.zip

Here is a new build with the proposed corrections.

I only changed this for non-PV. Should I really change this for ok_to_use_TT_PV as well? I though the behavior should be fundamentally different for PV entries.

[Uly]

Thanks gaard and fruity, are the new changes of PA_I based on PA_G or PA_H? (e.g. are the changes of PA_H still in the latest build?)

[gaard]

Thanks gaard and fruity, are the new changes of PA_I based on PA_G or PA_H? (e.g. are the changes of PA_H still in the latest build?)

Based on H, so yes. Version I removes the option for turning preserve analysis on or off - it is always on. Also incorporated the code by fruity for ok_to_use_TT() for non-PV nodes. Everything else remains the same as in H.

[BB+]

According to the "Decembrists", the original Stockfish GTB code (and the latest version too it looks like) was erroneous in having "VALUE_NONE" in its TT.store() calls. They seem to have changed it to be "tbValue" in ComStock.
See http://ippolit.wikispaces.com/message/v ... k/33307456 (and some posts above that track the bug down).

re: Comstock Bugs Decembrist-57703
The error does not sit in StockFish, and too doesnot sit in ComStock. <smiley> It is when I copied StockFish GTB. We need to direct to PeterC this VALUE_NONE for TT.store doesnot consist valid. The new abridge upon RobboMacros.h desires this.

For instance, they changed

Code: Select all

        if (tbValue == VALUE_KNOWN_WIN)
            TT.store(pos.get_key(), tbValue, VALUE_TYPE_LOWER, depth, MOVE_NONE, VALUE_NONE, VALUE_NONE);

to

Code: Select all

        if (tbValue == VALUE_KNOWN_WIN)
          TT.store(pos.get_key(), tbValue, VALUE_TYPE_LOWER, depth, MOVE_NONE, tbValue, VALUE_ZERO); // Decembrist fix

and about 5 other changes of the same type.

[BB+]

# Added Dann Corbit's smooth scaling, configurable with UCI options (see below)

Incidentally, given that this is over a year old by now, has anyone actually verified whether it helps or not? It seems to have been introduced to much fanfare (150 Elo gain in limited testing, if I recall), but now it seems at best to be an option that doesn't hurt. Again, does anyone have actual data?

[Peter C]

According to the "Decembrists", the original Stockfish GTB code (and the latest version too it looks like) was erroneous in having "VALUE_NONE" in its TT.store() calls. They seem to have changed it to be "tbValue" in ComStock.
See http://ippolit.wikispaces.com/message/v ... k/33307456 (and some posts above that track the bug down).

re: Comstock Bugs Decembrist-57703
The error does not sit in StockFish, and too doesnot sit in ComStock. <smiley> It is when I copied StockFish GTB. We need to direct to PeterC this VALUE_NONE for TT.store doesnot consist valid. The new abridge upon RobboMacros.h desires this.

For instance, they changed

Code: Select all

        if (tbValue == VALUE_KNOWN_WIN)
            TT.store(pos.get_key(), tbValue, VALUE_TYPE_LOWER, depth, MOVE_NONE, VALUE_NONE, VALUE_NONE);

to

Code: Select all

        if (tbValue == VALUE_KNOWN_WIN)
          TT.store(pos.get_key(), tbValue, VALUE_TYPE_LOWER, depth, MOVE_NONE, tbValue, VALUE_ZERO); // Decembrist fix

and about 5 other changes of the same type.

Oh, thanks.
Actually, I think that code was from the Stockfish team, but it's possible I did it.

# Added Dann Corbit's smooth scaling, configurable with UCI options (see below)

Incidentally, given that this is over a year old by now, has anyone actually verified whether it helps or not? It seems to have been introduced to much fanfare (150 Elo gain in limited testing, if I recall), but now it seems at best to be an option that doesn't hurt. Again, does anyone have actual data?

Actually, this isn't the year-old version. This is a version he did for 1.9, but AFAIK it's the same algorithm.

Peter

[BB+]

The other thing you might want to do is change how the buffering is set in main.cpp -- see an exchange I had with Marco Costalba about it. I think the fix is to replace

Code: Select all

  cout.rdbuf()->pubsetbuf(NULL, 0);
  cin.rdbuf()->pubsetbuf(NULL, 0);

by (used by Fruit, for instance)

Code: Select all

  setvbuf(stdin,NULL,_IONBF,0);
  setvbuf(stdout,NULL,_IONBF,0);

The reason for this seems to be that "select" is used, and this is (presumably) a C function, and so the C methods to set buffers should also be used, rather than those of C++. But I really don't know the underlying specifics.

Marco also suggested that

Code: Select all

if (stdin->_cnt > 0)
    return 1;

could be added in the Windows part of data_available() in the misc.cpp file.

[Jeremy Bernstein]

The other thing you might want to do is change how the buffering is set in main.cpp -- see an exchange I had with Marco Costalba about it. I think the fix is to replace

Code: Select all

  cout.rdbuf()->pubsetbuf(NULL, 0);
  cin.rdbuf()->pubsetbuf(NULL, 0);

by (used by Fruit, for instance)

Code: Select all

  setvbuf(stdin,NULL,_IONBF,0);
  setvbuf(stdout,NULL,_IONBF,0);

The reason for this seems to be that "select" is used, and this is (presumably) a C function, and so the C methods to set buffers should also be used, rather than those of C++. But I really don't know the underlying specifics.

Marco also suggested that

Code: Select all

if (stdin->_cnt > 0)
    return 1;

could be added in the Windows part of data_available() in the misc.cpp file.

Is this why, if I type before the credits show and then hit enter, I get some error about not understanding " i S" (or similar)?

I am going to put some code (PA_I + GTB + whatever came down the pipe since then) up on github today. I am finding the current "oh yeah, make this change (and 6 others like it wherever appropriate)" a little too indefinite (and chaotic) for my tastes. I'll post details when it's there.

Jeremy

[Prima]

After reading your posts, specifically on the part where I had to click on the Stockfish_PA_GTB.sln, the VS2010 GUI automatically opened and I was able to make a successful and regular (but no optimization of any kind) compiled with it. Thanks Jeremy.

OK, that's a start. In my projects, optimization is turned off for the Debug build, but on for the Release. The PGO stuff is additional to the Release builds, and if you google "PGO Visual Studio" you will find some instructions about how to use it. Basically you build an instrumented version of your app, use it (I've been exercising the app with benchmark runs using different numbers of threads, hash sizes and pvs and 30s+0.5s engine matches (32 v 64 bit to exercise them both at once), and then build an optimized build on the basis of files which get saved automagically when you run the instrumented build. But see if you can't get the normal, non-PGO optimized builds going first.

This still leaves me vulnerable to future Stockfish engines with GTB ported to it with multiple sub-source-folders, or, any engine that has similar multiple sub-folders within its source folder. Perhaps there's a method to how I can accomplish similar end-results you did with the project files etc.

It's pretty simple: you add the .c/.cpp files that you want to compile to the project. You add the folders containing any .h files that the .c files reference (which aren't in the compiler header search path already) to the list of #include directories. That's actually it. This isn't a vulnerability -- it's just the way compilers work. When I converted my project from VS2008 to VS2010, VS flattened the file structure I had in the original project (which mirrored the file system's folders) for some reason, but internally they are referenced by path relative to the project file.

One more thing, the VS2010 Edition does not have settings/menu for "linking" to produce a non-dll-dependent executive file we talked about earlier. Are these settings hidden? I can't find it.

You need to examine the options in the property pages (in the Solution Explorer, right-click on the project file and choose Properties...). That's where all of the stuff you normally specify on the command line is set, such as preprocessor variables, linked libs, optimization settings, etc.

I tried compiling it using the command prompt method to compile it. I pointed it to the stockfish_pa_gtb.sln (as I would point it to .cpp & .c source files) but the command prompt gave errors and couldn't compile it.

The .sln/.vcproj thing is an alternative to using the command line. You do one or other other, usually.

Jeremy

Thanks for the enlightenment, Jeremy. I can always reflect on this tutorial for future references. This is helpful.
Here's my compile of Stockfish 2.0.1Preserve Analysis-G GTB, in 32 and 64-bits - possible through your project file configuration.