Stockfish 3 PA_GTB

[Jeremy Bernstein]

I am looking for some knowledgable testers with some experience using "learning" or "persistent hash" engine analysis features. If this is you, and you have a little time to spend playing with something that's potentially buggy, please drop me a PM.

Thanks
Jeremy

No one? I would have guessed that a few people might be interested in this.

I spent the weekend making some significant improvements to non-root tablebase searching (more or less completely rewriting the code used to determine when and how tablebases should be probed when there are more than 5 pieces on the board). I think the result is pretty good and robust, and is strong enough to push the engine in the right direction without taking too much time on tablebase probes. If the user waits long enough, though, a correct mate score will be provided. So I'm pretty happy with the results, and the code makes more sense to me now. This, combined with the fixed tbhits count, will probably be v004. The persistent hash work will be v005 or something later, but I would love some people with opinions and experience with similar features to bang the tires a bit before releasing to a wider public.

Thanks,
Jeremy

[endings]

I'll test for you but you need to tell me what you want me to do and I will do it

[BorgKing]

I would be glad to test it as I'm running some long test games on my corr games anyway. And a learning function for SF sounds like a good idea, if Houdini can have one, why not Stockfish!

And I would probably have some opinions and suggestions after looking thru the games, but since I'm more an engine user than a programmer or such, it's an open question how useful that will be to you...

You can PM here if that sounds alright to you.


Btw, SFish in my opinion is very strong already but it has a few weaknesses nearly all related to the evaluation function: 1) in the opening/early middlegame, given time, everything tends to end up "0.00" no matter which side is better, and usually one side is at least slightly better. I suppose this is related to 2) it tends to find a repetition deep into a line and call that equal, whether it's forced or not. A repetition should probably not be judged "= 0.00" until the third time, and if possible it should only repeat a position if it considers itself worse if it doesn't.
And 3) it's evaluations tend to jump around about 0.5 pawns up or down for every ply, which makes it susceptible to an inferior move jumping into the ML - though this may be mostly a concern for blitz games or if it's running out of time at the end of a game. i.e 5) better time management may also be desirable - in dynamic positions containing most pieces still on the board it tends to make its move a bit too soon, at ply 26-27 when a better option is usually found when it's forced to think just 1-2 ply deeper. While towards the end of a game, especially endings, it may be too willing to go deep, if 36ply gives a +2.03 verdict is there any reason to spend half it's remaining time on going to 38 ply?

[Jeremy Bernstein]

I would be glad to test it as I'm running some long test games on my corr games anyway. And a learning function for SF sounds like a good idea, if Houdini can have one, why not Stockfish!

And I would probably have some opinions and suggestions after looking thru the games, but since I'm more an engine user than a programmer or such, it's an open question how useful that will be to you...

You can PM here if that sounds alright to you.


Btw, SFish in my opinion is very strong already but it has a few weaknesses nearly all related to the evaluation function: 1) in the opening/early middlegame, given time, everything tends to end up "0.00" no matter which side is better, and usually one side is at least slightly better. I suppose this is related to 2) it tends to find a repetition deep into a line and call that equal, whether it's forced or not. A repetition should probably not be judged "= 0.00" until the third time, and if possible it should only repeat a position if it considers itself worse if it doesn't.
And 3) it's evaluations tend to jump around about 0.5 pawns up or down for every ply, which makes it susceptible to an inferior move jumping into the ML - though this may be mostly a concern for blitz games or if it's running out of time at the end of a game. i.e 5) better time management may also be desirable - in dynamic positions containing most pieces still on the board it tends to make its move a bit too soon, at ply 26-27 when a better option is usually found when it's forced to think just 1-2 ply deeper. While towards the end of a game, especially endings, it may be too willing to go deep, if 36ply gives a +2.03 verdict is there any reason to spend half it's remaining time on going to 38 ply?

BorgKing, please send me your email address + platform (Mac/PC 32/64) via PM and I'll get you something to test.

Once the low-hanging fruit that I'm currently working on is done, maybe there'll be some time for the other stuff -- I have a feeling that #2 is something to take seriously -- there have been a few reports about Stockfish's repetition handling. The other stuff, well, let's see. I've looked at where 'draw by repetition' is evaluated and it looks "right" for what it's trying to ascertain, but what you're describing is a potential side effect, if I read the code correctly.

All that with the caveat that I'm neither a Stockfish expert nor a "real" chess engine programmer, although I'm starting to feel pretty good about my general understanding of how Stockfish works. There are a lot of smart people working on the Github Stockfish project who may be better positioned to find and, as necessary, address the causes of these sorts of problems. The work I'm doing is non-trivial, but it's also mostly in the search domain and what I'd term "general programming" which any experienced programmer with some time on his/her hands could probably do just as well.

Jeremy

[Ace1]

@Jeremy Bernstein
Can we have a 3-fold rep=draw in Stockfish 3 PA_GTB?

While searching SF3 does detect 3-fold repetition and provides a draw score, which is used to score/sort PVs. That score isn't used directly at the root, but given the following line:

1.d4 d5 2.Nf3 Nf6 3.Ng1 Ng8 4.Nf3 {0.00: Nf6} Nf6 {+0.42: c4} 5.Ng1 {-0.55: c5} Ng8 {+0.31: c4} 6.Nf3 {0.00: Nf6}

You can see that something is going on after White's 4th move (1st repetition of the position). So Black is offered an eval of 0.00 if he plays Nf6. But surprise, after playing Nf6, the eval jumps to +0.42 if White plays c4 (strangely enough, this position is the 1st repetition of the position after 2...Nf6, so it should also be marked 0.00). Back and forth with real scores until 6.Nf3, when the draw recognition kicks in again.

So something isn't really working 100% there. I honestly don't know what the correct solution here is. Repetition detection doesn't come for free (this thread on approx the same topic might be of interest: https://groups.google.com/forum/?fromgr ... ZdxhyDZ9kJ), so that probably explains the half-hearted implementation here.

Instinctively, I think that a draw score only makes sense when the side to move has the option to make a 2nd repetition (and immediately end the game as a draw), and only if that's the best move, all other factors being equal. That's not how most engines do it, though, and I'm not well-versed in engine development to know why. Maybe Marco or Bob or someone equally knowledgable will chime in and explain it to us.

Jeremy

this could help you!

Code: Select all

https://github.com/mcostalba/Stockfish/commit/58c9fbacc787cc3b654244c071169057e6895c91

I don't no programming to correct this, but you can!

[Jeremy Bernstein]

this could help you!

Code: Select all

https://github.com/mcostalba/Stockfish/commit/58c9fbacc787cc3b654244c071169057e6895c91

I don't no programming to correct this, but you can!

Indeed. Nothing like work that's already been done. It's a shame that the Stockfish team is so performance-centric that obviously right code is removed in favor of obviously wrong code. But then again...

My version of reverted code fixes the behavior I described above. I'll see about getting this into an upcoming version.

Thanks, Jeremy

[Ace1]

Thanks Jeremy Bernstein!
Waiting

[snino64]

Thanks Jeremy Bernstein!

To compile with gcc should be added in the makefile in general linked options:

ifeq ($(comp),gcc)
LDFLAGS +=-Legtb -lgtb
endif

Benigno

[Jeremy Bernstein]

Thanks Jeremy Bernstein!

To compile with gcc should be added in the makefile in general linked options:

ifeq ($(comp),gcc)
LDFLAGS +=-Legtb -lgtb
endif

Benigno

Should be there -- I compile with GCC/MinGW on Windows... *rummage rummage* ah I see that I only added it for mingw. I'll fix this for the next one.

Thanks -
Jeremy

[Jeremy Bernstein]

v004 is released!

- Simplification and cleanup of non-root EGTB probing
- Fix for tbhits reporting
- Added confirmation message for changing UCI EGTB settings
- Added 'Soft Probe Depth' and 'Hard Probe Depth' for manual control of non-root EGTB probing. The defaults are probably best for most situations
- 3-fold repetition is reinstated
- Engine output was performing unnecessary locking. Replaced with a cleaner solution
- Makefile update (note that OSX builds made with the Makefile are 10.6+ due to a C++ incompatibility. Xcode builds based on the 10.6sdk run fine on older OSs [tested to 10.5])
- Update to the tip of the upstream Stockfish master branch

Enjoy, thanks for any feedback.