Houdin 1.5a stronger than Houdini 2.0 on long games (32bit)

Discussion about chess-playing software (engines, hosts, opening books, platforms, etc...)
User avatar
Uly
Posts: 838
Joined: Thu Jun 10, 2010 5:33 am

Re: Houdin 1.5a stronger than Houdini 2.0 on long games (32b

Post by Uly » Fri Sep 16, 2011 3:50 am

What you say also applies to chess engines, e.g. you need 500 games between Houdini 2.0 and Houdini 1.5a only because they're very close in strength, you don't need as many games to be statistically confident that Houdini is stronger than Micro-Max.

So, again, chess statistics aren't really different from other statistics.

joraum
Posts: 3
Joined: Mon Jul 16, 2012 3:20 am
Real Name: Mark James

Re: Houdin 1.5a stronger than Houdini 2.0 on long games (32b

Post by joraum » Mon Jul 16, 2012 3:23 am

Ponder on ruins your results on a 1 PC match as the engines fight each other for CPU power with threading priority and OS multitasking leading to unequal resource distribution. Engine vs Engine games should always be Pondering off, unless the engines are on their own dedicated PCs, then you should have it on.

ernest
Posts: 247
Joined: Thu Sep 02, 2010 10:33 am

Re: Houdin 1.5a stronger than Houdini 2.0 on long games (32b

Post by ernest » Tue Jul 17, 2012 1:06 am

joraum wrote:Ponder on ruins your results on a 1 PC match as the engines fight each other for CPU power with threading priority
??? :shock:
any proof?
(of course you need at least 2 cores...)
Last edited by ernest on Tue Jul 17, 2012 1:09 am, edited 1 time in total.

User923005
Posts: 616
Joined: Thu May 19, 2011 1:35 am

Re: Houdin 1.5a stronger than Houdini 2.0 on long games (32b

Post by User923005 » Tue Jul 17, 2012 1:08 am

He probably has a machine with 1 CPU. Then, the statement makes sense.

joraum
Posts: 3
Joined: Mon Jul 16, 2012 3:20 am
Real Name: Mark James

Re: Houdin 1.5a stronger than Houdini 2.0 on long games (32b

Post by joraum » Tue Jul 17, 2012 5:57 am

It would make since with any number of CPUs, as long as those CPUS belonged to a single PC.

The dueling engines will spend the entire match fighting to control 100% of the available CPU power. Obviously, they can't both control 100%, so they will have to share it, but it's unlikely they will share it at a perfect 50/50 just due to all the random variables that go into how threading is distributed and handled, so one of the engines will have a CPU advantage. The only way to have a fair engine vs engine match on the same machine is to disable pondering. I suppose if you limited each engine to a specific CPU or core - that could work with pondering, although you would need to make sure the dedicated core/cpu is only being used by their respective engines and not anything else on the machine.

ernest
Posts: 247
Joined: Thu Sep 02, 2010 10:33 am

Re: Houdin 1.5a stronger than Houdini 2.0 on long games (32b

Post by ernest » Wed Jul 18, 2012 12:02 am

joraum wrote:It would make since with any number of CPUs, as long as those CPUS belonged to a single PC.
The dueling engines will spend the entire match fighting to control 100% of the available CPU power.
Go back to school... :roll:

joraum
Posts: 3
Joined: Mon Jul 16, 2012 3:20 am
Real Name: Mark James

Re: Houdin 1.5a stronger than Houdini 2.0 on long games (32b

Post by joraum » Wed Jul 18, 2012 5:57 am

Yes. The magical computer. Two processes say "give me all the CPU power" and the magic computer, like King Solomon, divides the CPU perfectly in half so not a single extra cycle goes to one of the engines. Brilliant insight

Edit: You don't have to take my word for it though you can search for advice and just about everyone says you need to set pondering off for engine-vs-engine on a single PC match

example thread: http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=23832

And a computer chess website also recommends setting pondering off: http://www.computerchess.org.uk/ccrl/4040/about.html

User923005
Posts: 616
Joined: Thu May 19, 2011 1:35 am

Re: Houdin 1.5a stronger than Houdini 2.0 on long games (32b

Post by User923005 » Wed Jul 18, 2012 7:42 pm

If I have a machine with 4 physical cores (hyperthreads ignored) then I can have a ponder on test with 2 engines each getting 2 threads with no penalty of any kind.
If I have a machine with 2 physical cores (hyperthreads ignored) then I can have a ponder on test with 2 engines each getting 1 thread with no penalty of any kind.
If I have a machine with 1 physical core then I cannot run a tournament with pondering set on.
It is as simple as that.

Simplified:
A single machine with k cores can allocate u=floor[k/n] cores each to a simultaneous engines=n tournament as long as u is greater than or equal to 1 with no performance loss whatsoever.

El Kitch
Posts: 5
Joined: Fri Aug 03, 2012 11:56 pm

Re: Houdin 1.5a stronger than Houdini 2.0 on long games (32b

Post by El Kitch » Sat Aug 04, 2012 7:27 am

Interesting thread and well expressed opinions. Wouldn't it be an idea to archive those runs somewhere? People like Pieterhb could enter the results from their runs, then eventually you'll have many many runs at different time controls and hardware.

Also a bit ironic btw: chess, which is known for its close to 0% luck, seems to become a game of (weighted) chance when it comes to a large number of games

Post Reply