Houdin 1.5a stronger than Houdini 2.0 on long games (32bit)

[Uly]

What you say also applies to chess engines, e.g. you need 500 games between Houdini 2.0 and Houdini 1.5a only because they're very close in strength, you don't need as many games to be statistically confident that Houdini is stronger than Micro-Max.

So, again, chess statistics aren't really different from other statistics.

[joraum]

Ponder on ruins your results on a 1 PC match as the engines fight each other for CPU power with threading priority and OS multitasking leading to unequal resource distribution. Engine vs Engine games should always be Pondering off, unless the engines are on their own dedicated PCs, then you should have it on.

[ernest]

Ponder on ruins your results on a 1 PC match as the engines fight each other for CPU power with threading priority

???
any proof?
(of course you need at least 2 cores...)

[User923005]

He probably has a machine with 1 CPU. Then, the statement makes sense.

[joraum]

It would make since with any number of CPUs, as long as those CPUS belonged to a single PC.

The dueling engines will spend the entire match fighting to control 100% of the available CPU power. Obviously, they can't both control 100%, so they will have to share it, but it's unlikely they will share it at a perfect 50/50 just due to all the random variables that go into how threading is distributed and handled, so one of the engines will have a CPU advantage. The only way to have a fair engine vs engine match on the same machine is to disable pondering. I suppose if you limited each engine to a specific CPU or core - that could work with pondering, although you would need to make sure the dedicated core/cpu is only being used by their respective engines and not anything else on the machine.

[ernest]

It would make since with any number of CPUs, as long as those CPUS belonged to a single PC.
The dueling engines will spend the entire match fighting to control 100% of the available CPU power.

Go back to school...

[joraum]

Yes. The magical computer. Two processes say "give me all the CPU power" and the magic computer, like King Solomon, divides the CPU perfectly in half so not a single extra cycle goes to one of the engines. Brilliant insight

Edit: You don't have to take my word for it though you can search for advice and just about everyone says you need to set pondering off for engine-vs-engine on a single PC match

example thread: http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=23832

And a computer chess website also recommends setting pondering off: http://www.computerchess.org.uk/ccrl/4040/about.html

[User923005]

If I have a machine with 4 physical cores (hyperthreads ignored) then I can have a ponder on test with 2 engines each getting 2 threads with no penalty of any kind.
If I have a machine with 2 physical cores (hyperthreads ignored) then I can have a ponder on test with 2 engines each getting 1 thread with no penalty of any kind.
If I have a machine with 1 physical core then I cannot run a tournament with pondering set on.
It is as simple as that.

Simplified:
A single machine with k cores can allocate u=floor[k/n] cores each to a simultaneous engines=n tournament as long as u is greater than or equal to 1 with no performance loss whatsoever.

[El Kitch]

Interesting thread and well expressed opinions. Wouldn't it be an idea to archive those runs somewhere? People like Pieterhb could enter the results from their runs, then eventually you'll have many many runs at different time controls and hardware.

Also a bit ironic btw: chess, which is known for its close to 0% luck, seems to become a game of (weighted) chance when it comes to a large number of games