Hello,
I'd like to present you a couple of meta-engines that use
backpropagation neural networks for evaluation. This is a slightly
refined result of an introductory neural networks course assignment
that I did 2 yeas ago (this is why the first target was Grapefruit).
I slightly modified Grapefruit and Stockfish recently (hey, a big
thank you to all authors of open source engines!), swapping their
original evaluation functions with neural networks that are
trained... by the original functions. Changes are unobtrusive and the
engines have a few different modes of play (including original
engines' behaviour). I've not yet experimented much as I was eyeing
for bugs (recently found one that made all my previous learning
efforts moot), refactoring, some thread support, a little
documentation and a bit of benchmarking. There is room for
experimentation with different sized networks, network parameters and
of course - learning (also learning over eval logs of different
engine). It's not supposed to be strong but it's fun. Especially the
online learning modes that allow learning while playing. If anyone's
interested, here are links to the sources with more info:
neuroStock https://github.com/m00natic/neuroStock
neuroGrape https://github.com/m00natic/neuroGrape
neuroChessTrainer https://github.com/m00natic/neuroChessTrainer
neuroStock is a bit slow (on my laptop at least) compared to
neuroGrape, I'll have to get more familiar with it's inside to see
whether something can be done about this. neuroChessTrainer is a tool
that allows more controlled training/testing/creation of neural
networks over log files created by the engines.
Now, I'm using exclusively GNU/Linux but both target engines run on
more platforms. The only thing that would need attention for porting
is the rudimentary use of pthreads in the neural network part of the
code, I guess.
Cheers
Neural Networks and some modified engines
Re: Neural Networks and some modified engines
can u tell us a bit about your network and thoughts behind it? i am actually very much interested in using neural networks of various kinds for evaluation, but my background in this area is a bit slim. what did you do, which test-set, which settings, why backpropagation, what layout, etc?
Re: Neural Networks and some modified engines
Despite my interest in AI, I'm not much competent in this area either
for the moment. It was done from curiosity, it seemed to me that this
was not a tried approach. In recent days I saw some discussions in
TalkChess (btw I'm still waiting for confirmation of my registration
there) and understood that I'm wrong. My attempt is quite lame but
straightforward. It's not a self-learner, it learns from target
engine, there are quite good engines nowadays, why not use them
It's quite a basic backpropagation network. It tries to map almost
full description of a position (only 50-move rule info is discarded)
directly to an evaluation. 262 bits were needed for input. Why
backpropagation? I was more familiar with it, it looked better suited
for the problem at hand than the few other models I saw at the time.
I made some simplifying assumptions about the network, all neurons in
a layer have same output function, also either all have bias or none.
It just allows easier description of networks and thus easier
(de)serialisation to/from files. The reason is that I really have no
idea how big network might suffice, how many layers, how many neurons.
This could be a good target for a genetic algorithm. Chess is very
complicated, I don't know if a backpropagation neural network could at
all manage it. My little attempts for learning (with the default
about randomly chosen network layout) showed that after learning and
getting good error results over a set of about 1 000 000 positions and
then doing the same over another corpus of this size - error over the
first corpus got (substantially) worse. I guess it shows that this
network layout is too small. Take it with a grain of salt though,
this was done with a subtle bug that might have somewhat screwed
results.
The online learning is a recent addition that really made my day. It
has the nice property that the network is trained over immediately
relevant positions which allows for a more reasonable play. It's fun
to watch two such engines play against each other, it's like novices
that learn the game move after move.
As mentioned, I haven't experimented much yet. I don't have access to
good hardware (but I'm on Gentoo, so no need to overclock
) so at
least this made me try to squeeze what I can for performance from the
network and compilation (just for the curious, adding -march=core2 to
the gcc arguments, gave around x4 speedup for training and propagation
on my 2 x 2.16 GHz processor, whoa)... unfortunately I'm not that
masterful with C++ and worse, I don't have much interest in these ugly
imperative languages any more. If you care about performance, here's
what my little tests show on my computer: training at about 1720
backpropagations per second, running at about 80 000 propagations per
second. Also propagation/training on single core is just slightly
faster than on 2 cores on my laptop. It's not that surprising and I
have a thing in mind to tune further.
for the moment. It was done from curiosity, it seemed to me that this
was not a tried approach. In recent days I saw some discussions in
TalkChess (btw I'm still waiting for confirmation of my registration
there) and understood that I'm wrong. My attempt is quite lame but
straightforward. It's not a self-learner, it learns from target
engine, there are quite good engines nowadays, why not use them
It's quite a basic backpropagation network. It tries to map almost
full description of a position (only 50-move rule info is discarded)
directly to an evaluation. 262 bits were needed for input. Why
backpropagation? I was more familiar with it, it looked better suited
for the problem at hand than the few other models I saw at the time.
I made some simplifying assumptions about the network, all neurons in
a layer have same output function, also either all have bias or none.
It just allows easier description of networks and thus easier
(de)serialisation to/from files. The reason is that I really have no
idea how big network might suffice, how many layers, how many neurons.
This could be a good target for a genetic algorithm. Chess is very
complicated, I don't know if a backpropagation neural network could at
all manage it. My little attempts for learning (with the default
about randomly chosen network layout) showed that after learning and
getting good error results over a set of about 1 000 000 positions and
then doing the same over another corpus of this size - error over the
first corpus got (substantially) worse. I guess it shows that this
network layout is too small. Take it with a grain of salt though,
this was done with a subtle bug that might have somewhat screwed
results.
The online learning is a recent addition that really made my day. It
has the nice property that the network is trained over immediately
relevant positions which allows for a more reasonable play. It's fun
to watch two such engines play against each other, it's like novices
that learn the game move after move.
As mentioned, I haven't experimented much yet. I don't have access to
good hardware (but I'm on Gentoo, so no need to overclock
) so atleast this made me try to squeeze what I can for performance from the
network and compilation (just for the curious, adding -march=core2 to
the gcc arguments, gave around x4 speedup for training and propagation
on my 2 x 2.16 GHz processor, whoa)... unfortunately I'm not that
masterful with C++ and worse, I don't have much interest in these ugly
imperative languages any more. If you care about performance, here's
what my little tests show on my computer: training at about 1720
backpropagations per second, running at about 80 000 propagations per
second. Also propagation/training on single core is just slightly
faster than on 2 cores on my laptop. It's not that surprising and I
have a thing in mind to tune further.
Re: Neural Networks and some modified engines
some random thoughts concerning your experiment:
- i think it could be quite smth if done correctly. the parameters are, by definition, always pretty good. and if the trainingset is correctly chosen the resulting optimization could by mindblowing
- your lack of convergence over the two testsets pretty much proofs that your efforts are futile atm. you did not find a global minimum in the chosen function, and every local minima can only yield a random quality of response.
- did you think of using svm's? especially for performance-critical tasks they should perform better.
- split the task! to analyze a complete chess situation can hardly be achieved with manageable network-sizes. split the task and train several networks, eg: one for pawnstructure, kingsafety, etc... the function you are looking for is way simpler and can already be modeled with code in most eval-functions, hence you could achieve good convergence here
- online-learning is funny, but if you really want to achieve strength, you should leave it aside.
im very much interested in this, so keep up the good work
- i think it could be quite smth if done correctly. the parameters are, by definition, always pretty good. and if the trainingset is correctly chosen the resulting optimization could by mindblowing
- your lack of convergence over the two testsets pretty much proofs that your efforts are futile atm. you did not find a global minimum in the chosen function, and every local minima can only yield a random quality of response.
- did you think of using svm's? especially for performance-critical tasks they should perform better.
- split the task! to analyze a complete chess situation can hardly be achieved with manageable network-sizes. split the task and train several networks, eg: one for pawnstructure, kingsafety, etc... the function you are looking for is way simpler and can already be modeled with code in most eval-functions, hence you could achieve good convergence here
- online-learning is funny, but if you really want to achieve strength, you should leave it aside.
im very much interested in this, so keep up the good work
Re: Neural Networks and some modified engines
Thanks a lot for your interest and suggestions. Splitting the task
goes in my TODO (tothink) list. SVM stands for support vector
machine? I'll have to look it up. Reading recommendations are
welcomed
Meanwhile I pushed some changes that speedup training (hopefully even
more so on more cores), so training is about x4 slower than
propagation (I put one too many zeroes for propagation speed in my
previous post). Right now I have some heat+graphics issues (good that
this forum is accessible with terminal Emacs+w3m) with my laptop, so
I'm forced to suspend working on this for some time.
goes in my TODO (tothink) list. SVM stands for support vector
machine? I'll have to look it up. Reading recommendations are
welcomed
Meanwhile I pushed some changes that speedup training (hopefully even
more so on more cores), so training is about x4 slower than
propagation (I put one too many zeroes for propagation speed in my
previous post). Right now I have some heat+graphics issues (good that
this forum is accessible with terminal Emacs+w3m) with my laptop, so
I'm forced to suspend working on this for some time.