I am busy testing Houdini 1.5a versus the new Houdini Pro 2.0 on 32 bit machines, with ponder on, but
the rest of the settings I kept on default.
I have three different PC's, ranging from 6 years, 4 years and 1 year old.
On each one I am testing Houdini 1.5a versus Houdini 2.0, with tournaments of 30 minutes, that then lasts
about an hour per game.
What I have found so far is that with my oldest PC, which has the least amount of RAM and of course the slowest CPU,
Houdini 1.5a beats Houdini 2.0 standard (after 40 games) with about a 10% difference.
However, with my two "newer" PC's, the situation is as expected with the newer Houdini 2.0. There Houdini 2.0
is currently (after 40 games) ahead on both PC's, with a difference of about 10% and 20% respectively.
I know it is still early days and I cannot make any definately conclusions yet, but it seems to me that Houdini 2.0
needs more configuration/calibration/fine tuning on older machines to run at its optimum, than on newer machines with
more RAM and faster CPU's?
If one thus wants Houdini 2.0 to run at its absolute most powerful, one has to fiddle around with the software configuration,
as suggested by Robert Houdart, especially on older machines. So do yourself a favour and spend some 5 - 10 minutes fine tuning
Houdini 2.0 and then give it a go and see what happens.
My next project will be to test Houdini 2.0 against itself, with settings on one PC on default and the other PC on maximum tuned,
and then see what the outcome will be...
Houdini Pro 2.0 configuration very important
Re: Houdini Pro 2.0 configuration very important
So what is the score, after 40 games? 22-18?Pieterhb wrote:What I have found so far is that with my oldest PC, which has the least amount of RAM and of course the slowest CPU, Houdini 1.5a beats Houdini 2.0 standard (after 40 games) with about a 10% difference.
With 40 games and around 50% draws, the 95% error bar is +-4.5
Which means that your 22-18 could still be the opposite: 17.5-22.5
......besides, I just read your viewtopic.php?p=14321#p14321 post, so you are in complete agreement with me!
Re: Houdini Pro 2.0 configuration very important
Hi Ernest
See my new post.
What I found very interesting is the fact that up to 100 games, Houdini 1.5a and Houdini 2.0 were still neck and neck, with
the same amount of wins. Only thereafter did Houdini 2.0 slowly pull away.
Thus with games of 20, 30, 40 or even more, with chess engines that closely matched in terms of ELO, it becomes
very very difficult to see with a limited amount of games, which one is actually the stronger and by how much.
What I also discovered is that with shorter games, the difference is more, than with the longer games. For example,
Houdini 2.0 will win more games against Houdini 1.5a, with say 100, 5 minutes games than with 100, 20 minutes games...
That said, with Houdini 2.0 compared with other chess engines like say Critter, Stockfish, etc. Houdini 2.0 win about
all the games and draw only a couple of games, meaning that one can get away with way less games, to determine which
chess engine is stronger and by how much.
I think we as testers (myself included of course) should just be very careful with posting chess engine conclusion stats as
hard facts, if we did not run a sufficient amount of games between any two chess engines out there today...
See my new post.
What I found very interesting is the fact that up to 100 games, Houdini 1.5a and Houdini 2.0 were still neck and neck, with
the same amount of wins. Only thereafter did Houdini 2.0 slowly pull away.
Thus with games of 20, 30, 40 or even more, with chess engines that closely matched in terms of ELO, it becomes
very very difficult to see with a limited amount of games, which one is actually the stronger and by how much.
What I also discovered is that with shorter games, the difference is more, than with the longer games. For example,
Houdini 2.0 will win more games against Houdini 1.5a, with say 100, 5 minutes games than with 100, 20 minutes games...
That said, with Houdini 2.0 compared with other chess engines like say Critter, Stockfish, etc. Houdini 2.0 win about
all the games and draw only a couple of games, meaning that one can get away with way less games, to determine which
chess engine is stronger and by how much.
I think we as testers (myself included of course) should just be very careful with posting chess engine conclusion stats as
hard facts, if we did not run a sufficient amount of games between any two chess engines out there today...