Stockfish 3 PA_GTB

Discussion about chess-playing software (engines, hosts, opening books, platforms, etc...)
Post Reply
Jeremy Bernstein
Site Admin
Posts: 1226
Joined: Wed Jun 09, 2010 7:49 am
Real Name: Jeremy Bernstein
Location: Berlin, Germany
Contact:
Contact Jeremy Bernstein

Re: Stockfish 3 PA_GTB

Post by Jeremy Bernstein » Mon May 27, 2013 7:52 am

Jeremy Bernstein wrote:I am looking for some knowledgable testers with some experience using "learning" or "persistent hash" engine analysis features. If this is you, and you have a little time to spend playing with something that's potentially buggy, please drop me a PM.

Thanks
Jeremy
No one? I would have guessed that a few people might be interested in this.

I spent the weekend making some significant improvements to non-root tablebase searching (more or less completely rewriting the code used to determine when and how tablebases should be probed when there are more than 5 pieces on the board). I think the result is pretty good and robust, and is strong enough to push the engine in the right direction without taking too much time on tablebase probes. If the user waits long enough, though, a correct mate score will be provided. So I'm pretty happy with the results, and the code makes more sense to me now. This, combined with the fixed tbhits count, will probably be v004. The persistent hash work will be v005 or something later, but I would love some people with opinions and experience with similar features to bang the tires a bit before releasing to a wider public.

Thanks,
Jeremy
Top
endings
Posts: 9
Joined: Wed May 22, 2013 12:07 am
Real Name: Robert

Re: Stockfish 3 PA_GTB

Post by endings » Mon May 27, 2013 8:27 am

I'll test for you but you need to tell me what you want me to do and I will do it
Top
BorgKing
Posts: 19
Joined: Mon May 20, 2013 9:47 am

Re: Stockfish 3 PA_GTB

Post by BorgKing » Mon May 27, 2013 11:38 am

I would be glad to test it as I'm running some long test games on my corr games anyway. And a learning function for SF sounds like a good idea, if Houdini can have one, why not Stockfish! :)

And I would probably have some opinions and suggestions after looking thru the games, but since I'm more an engine user than a programmer or such, it's an open question how useful that will be to you...

You can PM here if that sounds alright to you.


Btw, SFish in my opinion is very strong already but it has a few weaknesses nearly all related to the evaluation function: 1) in the opening/early middlegame, given time, everything tends to end up "0.00" no matter which side is better, and usually one side is at least slightly better. I suppose this is related to 2) it tends to find a repetition deep into a line and call that equal, whether it's forced or not. A repetition should probably not be judged "= 0.00" until the third time, and if possible it should only repeat a position if it considers itself worse if it doesn't.
And 3) it's evaluations tend to jump around about 0.5 pawns up or down for every ply, which makes it susceptible to an inferior move jumping into the ML - though this may be mostly a concern for blitz games or if it's running out of time at the end of a game. i.e 5) better time management may also be desirable - in dynamic positions containing most pieces still on the board it tends to make its move a bit too soon, at ply 26-27 when a better option is usually found when it's forced to think just 1-2 ply deeper. While towards the end of a game, especially endings, it may be too willing to go deep, if 36ply gives a +2.03 verdict is there any reason to spend half it's remaining time on going to 38 ply?
Top
Jeremy Bernstein
Site Admin
Posts: 1226
Joined: Wed Jun 09, 2010 7:49 am
Real Name: Jeremy Bernstein
Location: Berlin, Germany
Contact:
Contact Jeremy Bernstein

Re: Stockfish 3 PA_GTB

Post by Jeremy Bernstein » Mon May 27, 2013 1:19 pm

BorgKing wrote:I would be glad to test it as I'm running some long test games on my corr games anyway. And a learning function for SF sounds like a good idea, if Houdini can have one, why not Stockfish! :)

And I would probably have some opinions and suggestions after looking thru the games, but since I'm more an engine user than a programmer or such, it's an open question how useful that will be to you...

You can PM here if that sounds alright to you.


Btw, SFish in my opinion is very strong already but it has a few weaknesses nearly all related to the evaluation function: 1) in the opening/early middlegame, given time, everything tends to end up "0.00" no matter which side is better, and usually one side is at least slightly better. I suppose this is related to 2) it tends to find a repetition deep into a line and call that equal, whether it's forced or not. A repetition should probably not be judged "= 0.00" until the third time, and if possible it should only repeat a position if it considers itself worse if it doesn't.
And 3) it's evaluations tend to jump around about 0.5 pawns up or down for every ply, which makes it susceptible to an inferior move jumping into the ML - though this may be mostly a concern for blitz games or if it's running out of time at the end of a game. i.e 5) better time management may also be desirable - in dynamic positions containing most pieces still on the board it tends to make its move a bit too soon, at ply 26-27 when a better option is usually found when it's forced to think just 1-2 ply deeper. While towards the end of a game, especially endings, it may be too willing to go deep, if 36ply gives a +2.03 verdict is there any reason to spend half it's remaining time on going to 38 ply?
BorgKing, please send me your email address + platform (Mac/PC 32/64) via PM and I'll get you something to test.

Once the low-hanging fruit that I'm currently working on is done, maybe there'll be some time for the other stuff -- I have a feeling that #2 is something to take seriously -- there have been a few reports about Stockfish's repetition handling. The other stuff, well, let's see. I've looked at where 'draw by repetition' is evaluated and it looks "right" for what it's trying to ascertain, but what you're describing is a potential side effect, if I read the code correctly.

All that with the caveat that I'm neither a Stockfish expert nor a "real" chess engine programmer, although I'm starting to feel pretty good about my general understanding of how Stockfish works. There are a lot of smart people working on the Github Stockfish project who may be better positioned to find and, as necessary, address the causes of these sorts of problems. The work I'm doing is non-trivial, but it's also mostly in the search domain and what I'd term "general programming" which any experienced programmer with some time on his/her hands could probably do just as well.

Jeremy
Top
Ace1
Posts: 12
Joined: Thu Dec 20, 2012 4:40 am
Real Name: AceAce

Re: Stockfish 3 PA_GTB

Post by Ace1 » Mon May 27, 2013 4:03 pm

Jeremy Bernstein wrote:
Ace1 wrote:@Jeremy Bernstein
Can we have a 3-fold rep=draw in Stockfish 3 PA_GTB? :)
While searching SF3 does detect 3-fold repetition and provides a draw score, which is used to score/sort PVs. That score isn't used directly at the root, but given the following line: