Page 1 of 2
Save The Date: Sharpie 1.0
Posted: Tue Jun 15, 2010 1:52 pm
by Jeremy Bernstein
http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=17636
From
Graviton:
I'm pleased to announce that a new competitive engine will be available very soon. (Website link will follow in a few days)
Sharpie 1.0 will be the first complete version of the first 64-bit engine my team has ever managed to bring to a competitive level of strength based heavily on genetic algorithms. The engine is unique in that not only does it adjust the weights used in the evaluation function, but it comes up with new parameters to include in the weight matrices on its own. (each engine profile will have multiple matrices that are affected at different stages of the game (opening, opening-middle transition, middle, middle-end transition, end, and defusing and complicating transitions). The determination of the appropriate point for the matrix shift (which is dynamic in search as well as in evaluation) is determined again genetically by the specific engine profile.
Some examples of parameters it "evolved" into the matrix without any hint/coding on our part:
Doubled rooks
Mobility
Some quirky parameters that can best be described as a type of 'initiative' factor
Options (how many legal moves in the given position) <- this was a really interesting one. Since this is genetically generated, it's a factor that correlates strongly (but not perfectly) with this attribute of a position. And since it fell into the local maxima it clearly has an impact on the performance.
The engine will be open source.
Unfortunately it is not UCI compatible, but we will distribute our own open source GUI.
You can modify and extend (add new parameters) to the evaluation matrix.
You can run your own genetic simulations to test new ideas (and set your own heritability and mutation levels)
Though it's not distributed as a UCI engine, when you are ready to solidify the evaluation parameters (and some other aspects of its search method) you can simply click the 'compile to UCI' button in the GUI and it will compile the current state to a fully UCI compatible engine.
Naturally we will distribute the engine with several options for default settings ranging from tactical styles to positional styles and what we've found so far to be optimal. We haven't done too much testing in this department so we fully expect users to find better settings over time. We hope better or simply interesting matrices will be shared on the forums.
Website will be launched on the same day as the engine.
Rough strength (based on 250 games at 40/4) is +32 elo vs. Zappa Mexico 2 64-bit on a quad, -212 elo vs rybka 4.
based on 100 games @ 40/20, +65 elo vs Zappa, -111 elo vs rybka 4
based on 50 games @ 40/120, +112 elo vs Zappa, -36 vs rybka 4
Sample sizes are small and we're running similar tests with Naum 4 and possibly Stockfish if we can get another tester with decent hardware.
These were completed with the current 'optimal' profile and so far it seems that our engine benefits from longer time controls.
The evaluation isn't the only genetically derived aspect. Some critical aspects of the search (move ordering, pruning, extensions etc.) are also genetically weighted.
-These will 'evolve' as you run your simulations, along with the evaluation matrix, but unfortunately the first GUI we release won't have support for manual adjustment of these parameters. It will display the search parameters and highlight changes per generation, but won't allow manual editing as most arbitrary settings here will cause a crash on any multi-core cpu.
This is a totally free and open source project that has been worked on privately for six years since we decided purely non-genetic approaches were too static.
We don't claim to offer you a new strongest engine but hopefully a new way of looking at computer chess and hopefully an interesting playing partner as well.
Release date:June 18th. Along with website with fully functional forum and integrated engine-profile-sharing and distributing.
Re: Save The Date: Sharpie 1.0
Posted: Tue Jun 15, 2010 9:10 pm
by BrandonSi
This is really something. Very impressive work, and even more so that they're publishing the source code. I have a feeling this is really going to have an impact on engine development.
I'm curious to see their GUI as well!
Re: Save The Date: Sharpie 1.0
Posted: Tue Jun 15, 2010 9:29 pm
by zwegner
Do I have the honor of being the first to say that this looks like BS?
Re: Save The Date: Sharpie 1.0
Posted: Wed Jun 16, 2010 1:07 am
by LucenaTheLucid
Zach is it so hard to believe? I mean I suppose its easy to be skeptical but...what about "Zillions of games"? That is an amazing feat. This I don't think is so far fetched.
Re: Save The Date: Sharpie 1.0
Posted: Wed Jun 16, 2010 1:40 am
by Karger
It is entirely possible that this is a legit engine. It was done with a chess engine called blondie 25, which went from a 1600 engine to a 2500 engine in 6 weeks of play. read the following " Reno, NV – Dr. David Fogel, CEO of Natural Selection, Inc., presented a paper with new results on evolving chess programs at the 2006 IEEE Symposium on Computational Intelligence and Games today. The symposium is being held at the University of Nevada, Reno and is sponsored by the IEEE Computational Intelligence Society. The paper, titled "“The Blondie25 Chess Program Competes Against Fritz 8.0 and a Human Chess Master,” co-authored by Tim Hays, Sarah Hahn, and James Quon, indicated results of playing a self-learning computer chess program called Blondie25 against the program Fritz 8.0 and also against James Quon, who is a nationally ranked chess master.
Blondie25 is the result of over 8000 generations of variation and selection, simulated on a computer, in which a computer chess-playing program plays games against variations of itself to learn how to improve its play. Blondie25 includes mechanisms for learning the values of the pieces, their locations on the chessboard, and also uses neural networks to assess the formation of pieces in different areas of the board.
In 24 games against Fritz 8.0, one of the top chess programs in the world, rated about 2750 using the standard chess rating system, Blondie25 earned 3 wins, 10 losses, and 11 draws. These results mark the first time that an evolved chess program has earned wins over a world-class chess program. Furthermore, in four games against James Quon, Blondie25 won three and lost one. These results mark the first time that an evolved chess program has defeated a human master. Mr. Quon is rated at 2301. "Our previous research with self-learning chess programs indicated that the computer had taught itself to be a strong player, even without the proper use of allocating time per move," said Dr. Fogel. "With the addition of reasonable heuristics for time management, as we describe in the paper, the Blondie25 program can now begin to play a strong game against a top program and present a significant challenge to human masters."
Re: Save The Date: Sharpie 1.0
Posted: Wed Jun 16, 2010 2:28 am
by Hagen
Well, the descriptions about this engine is certainly very interesting...but as the saying goes.."the proof is in the pudding". We'll need to see test results of "Sharpie" against Rybka to see how it "adapts" it's playing style. But I'm more curious about the fact this is not using the UCI protocol. Nor does it say it's using Winboard either. The lack of information about the GUI to this engine leaves me skeptical. I'm curious to see if the Ridderkerk site is going to announce this engine on their update page when Sharpie comes out.
Re: Save The Date: Sharpie 1.0
Posted: Wed Jun 16, 2010 6:40 am
by LetoAtreides82
"Rough strength (based on 250 games at 40/4) is +32 elo vs. Zappa Mexico 2 64-bit on a quad, -212 elo vs rybka 4.
based on 100 games @ 40/20, +65 elo vs Zappa, -111 elo vs rybka 4
based on 50 games @ 40/120, +112 elo vs Zappa, -36 vs rybka 4"
Are those results from a UCI version? If so I hope they make that version available for download so I can test it for CEGT Blitz. It's unfortunate though that it appears it can't learn without it's GUI.
Re: Save The Date: Sharpie 1.0
Posted: Wed Jun 16, 2010 8:51 am
by Vael Jean-Paul
More from Graviton:
The GUI will compile your engine profile state to a UCI engine.
The only aspect that won't be really active when you compile it to UCI is the whole genetic factor.
When you compile to UCI it will be a static engine with the only variable parameters, the evaluation matrices (albeit several).
We used several different computers running anything in the range of quad to octal.
Games were played against both itself, (dynamic and static, meaning we might evolve it vs. a static (unevolving) version of itself, or simply against itself), AND against several other top (and not so top) engines. Rybka and Zappa were the main ones used but we also used Naum and Stockfish a bit.
We have a slight variant on the typical genetic algorithm that makes the tuning process work better when executed on several computers of hardware X, rather than one cluster of hardware several*X. The results are then permuted with each other via group theoretic operations to create several personalities, ranging from the weakest combinations of all the results to the strongest. Determining the ideal out of this small set is fairly easy since there is usually wild variance.
It will ship with it's own GUI. But the GUI can compile the engine to a UCI. I don't think Winboard supports that protocal.
Unfortunately, for now we will only be supporting the 64-bit version.
Re: Save The Date: Sharpie 1.0
Posted: Wed Jun 16, 2010 3:31 pm
by Damir Desevac
zwegner wrote:Do I have the honor of being the first to say that this looks like BS?
Hi Zack
I completely agree with you. This sounds like another Ben Lau BS as usual...
Re: Save The Date: Sharpie 1.0
Posted: Wed Jun 16, 2010 4:30 pm
by zwegner
LucenaTheLucid wrote:Zach is it so hard to believe? I mean I suppose its easy to be skeptical but...what about "Zillions of games"? That is an amazing feat. This I don't think is so far fetched.
Well first, Zillions of Games was never very strong, but it was more impressive IMO because it could play any game.
I'm not saying that such a feat is impossible, but when somebody says they can get close to Rybka 4 level with an entirely evolved evaluation, I become pretty skeptical.
But even beyond that, everything this guy says just sounds fishy. There's pretty subtle hints that he has no idea what he's talking about. I guess it would be pretty hard to notice this if you're not a chess programmer. Examples:
Describing both "mobility" and what he calls "options" as if they are somehow different
Having a "slight variant" on the genetic algorithm, this has nothing to do with what machines you run on, and it looks like he doesn't know what a cluster is
The engine isn't UCI, but the GUI can compile one
The search parameters can be tuned by the GUI, but manual changes cause crashes on "any multi-core cpu"
The different matrices for different stages of the game
Saying 32-bit OSes can only address 3.2 GB of RAM
...and so on. I will bet in a couple days someone will ask him about it and he will have some reason there is a delay. Or he comes out with another Ippolit clone.
