Testing Houidini 1.03

[sirabc]

Googled: rybka hyperthreading

Most recent thread. http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=17836

Googled: houdini hyperthreading

http://www.open-chess.org/viewtopic.php ... t=10#p3992

http://www.open-chess.org/viewtopic.php ... t=10#p3994

I'm not saying this is definitive. Just the opinion of the programmers.

[Vael Jean-Paul]

Hi Tom,

Like Robbolito said..when you disable HT to OFF you win around 10°C when you use a very good cpu cooler!
The positive thing is,because you say your system run stable with the 8 threads running at 100% with Chess..
well when HT is OFF you can overclock your system even higher till you reach the same temps as before.
But you running now at a faster clockspeed what result in better game play then using these 8threads!

Is just a tip

JP.

[Tomcass]

FINAL RESULTS

TESTING HOUDINI 1.03a – GAMES IN 10 MINUTES

Global: 720 games(3 legs x 240 games each).

3rd. leg: 240 games.

TEST 1-Leg3-w32

Quad 2.33
Gui: Fritz 12
Ponder: Off
No Triple or Robbo Bases
Book: HS Masterbook 2.0
Time Control: 10 min+ 0 sec.
4 cores
120 games

1 Deep Rybka 4 x64 9.5 - 10.5 11.5 - 8.5 11.0 - 9.0 32.0/60
2 Houdini 1.03a x64 10.5 - 9.5 9.5 - 10.5 10.5 - 9.5 30.5/60
3 Ivanhoe-B57d_whm08 8.5 - 11.5 10.5-9.5 11.5 - 8.5 30.5/60
4 IvanHoe 55mU-x64Vlad0 9.0 - 11.0 9.5-10.5 8.5-11.5 27.0/60




CUMULATIVE RESULTS TEST 1 (w32) AFTER 360 GAMES.

1 Deep Rybka 4 w32 30,5-29.5 33.0-27.0 31.5-28.5 95.0/180 52,78%
2 IvanHoe 55mU-x32Vlad0 29.5-30.5 29.5-30.5 33.5-26.5 92.5/180 51.39%
3 Houdini 1.03a w32 27.0-33.0 30.5-29.5 30.0-30.0 87,5/180 48.61%
4 Ivanhoe-B57d_whm08 28.5-31.5 26.5-33.5 30.0-30.0 85.0/180 47.22%





TEST 2.- Leg3.-w64

I7 975
Gui: Fritz 12
Ponder: Off
No Triple or Robbo Bases
Book: HS Masterbook 2
Time Control: 10 min+ 0 sec.
4 cores and 8 threads
120 games

1 Deep Rybka 4 x64 9.5 - 10.5 11.5 - 8.5 11.0 - 9.0 32.0/60
2 Houdini 1.03a x64 10.5 - 9.5 9.5 - 10.5 10.5 - 9.5 30.5/60
3 Ivanhoe-B57d_whm08 8.5 - 11.5 10.5-9.5 11.5 - 8.5 30.5/60
4 IvanHoe 55mU-x64Vlad0 9.0 - 11.0 9.5-10.5 8.5-11.5 27.0/60


CUMULATIVE RESULTS TEST 2 (w64) AFTER 360 GAMES.

1 Deep Rybka 4 x64 32.0–28.0 32.0-28.0 34.5-25.5 98.5/180 54.72%
2 Houdini 1.03a x64 28.0-32.0 30.5-29.5 31.5-28.5 90.0/180 50,00%
3 Ivanhoe-B57d_whm08 28.0-32.0 29.5-30.5 31.5-28.5 89.0/180 49,44%
4 IvanHoe 55mU-x64 25.5-34.6 28.5-31.5 28.5-31.5 82.5/180 45,83%




GLOBAL SCORE AFTER 720 GAMES TIME CONTROL 10' 0.

1 Deep Rybka 4 65.0-55.0 65.0-55.0 63.5-56.5 193.5/360 53,75%
2 Houdini 1.03a 55.0-65.0 62.0-58.0 60.5-59.5 177.5/360 49.31%
3 IvanHoe 55mU-Vlad0 55.0-65.0 58.0-62.0 62.0-58.0 175.0/360 48.61%
4 Ivanhoe-B57d_whm08 56.5-63.5 59.5.-60.5 58.0-62.0 174.0/360 48.33%


PREVIOUS TEST SUMMARY. 480 GAMES. SAME GUI AND BOOKs. TIME CONTROL 5’ 0.

Houdini 1.03a 81,0-79,0 86,5-73,5 167,5/320 52.34%
Ivanhoe-B 57d_whm 79,0-81,0 79,5-80,5 158,5/320 49,53%
Deep Rybka4 73,5-86,5 80,5-79,5 154,0/320 48.13%


COMMENTS

After more than 300 hours testing and 1.200 games overall, this is my brief summary:

Whilst in my first test at 5’ 0 Houdini 1.3a was clearly stronger than Deep Rybka 4, with a win of 86,5-73,5 in its bilateral match, doubling the time control to 10’ 0 totally changed the landscape, with a very comfortable win for Deep Rybka 4. DR4 got a very convincing 65.0-55.0 against Houdini 1.3a. All Ivans (Bill’s, Vlad0’s and Ahmed’s) performed very well, but below the two leaders.

The inflection point where the weighted average strength of Houdini 1.3a and Deep Rybka 4 is roughly even in my computers, under the mentioned conditions, will be found at a time control around 7’ 0.

Regards from Barcelona,

Tom.

[Tomcass]

Googled: rybka hyperthreading

Most recent thread. http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=17836

Googled: houdini hyperthreading

http://www.open-chess.org/viewtopic.php ... t=10#p3992

http://www.open-chess.org/viewtopic.php ... t=10#p3994

I'm not saying this is definitive. Just the opinion of the programmers.

Thank you very much Sirabc. I will read these threads. I am sure this info will be useful for me.

Tom.

[Tomcass]

Hi Tom,

Like Robbolito said..when you disable HT to OFF you win around 10°C when you use a very good cpu cooler!
The positive thing is,because you say your system run stable with the 8 threads running at 100% with Chess..
well when HT is OFF you can overclock your system even higher till you reach the same temps as before.
But you running now at a faster clockspeed what result in better game play then using these 8threads!

Is just a tip

JP.

Good point, Jean-Paul!. In fact when I bought my last computer I was considering the possibility of overclocking it. But finally I decided not to do it. In fact I don't use my computers to play in the net. Only to test engines (and for other professional reasons, obviously). In this context an additional speed of a 15/20% is not that relevant for me.

Anyway, thanks for your suggestion!.

Tom.

[Charles]

COMMENTS

After more than 300 hours testing and 1.200 games overall, this is my brief summary:

Whilst in my first test at 5’ 0 Houdini 1.3a was clearly stronger than Deep Rybka 4, with a win of 86,5-73,5 in its bilateral match, doubling the time control to 10’ 0 totally changed the landscape, with a very comfortable win for Deep Rybka 4. DR4 got a very convincing 65.0-55.0 against Houdini 1.3a. All Ivans (Bill’s, Vlad0’s and Ahmed’s) performed very well, but below the two leaders.

The inflection point where the weighted average strength of Houdini 1.3a and Deep Rybka 4 is roughly even in my computers, under the mentioned conditions, will be found at a time control around 7’ 0.

Regards from Barcelona,

Tom.

This is very interesting and seems to imply that Rybka 4 is still #1 at longer time controls than 7'0!

Thus NO ONE can improve these engines beyond Rybka at longer time controls. Ippolit is not as great as it seems.

[Tomcass]

COMMENTS

After more than 300 hours testing and 1.200 games overall, this is my brief summary:

Whilst in my first test at 5’ 0 Houdini 1.3a was clearly stronger than Deep Rybka 4, with a win of 86,5-73,5 in its bilateral match, doubling the time control to 10’ 0 totally changed the landscape, with a very comfortable win for Deep Rybka 4. DR4 got a very convincing 65.0-55.0 against Houdini 1.3a. All Ivans (Bill’s, Vlad0’s and Ahmed’s) performed very well, but below the two leaders.

The inflection point where the weighted average strength of Houdini 1.3a and Deep Rybka 4 is roughly even in my computers, under the mentioned conditions, will be found at a time control around 7’ 0.

Regards from Barcelona,

Tom.

This is very interesting and seems to imply that Rybka 4 is still #1 at longer time controls than 7'0!

Thus NO ONE can improve these engines beyond Rybka at longer time controls. Ippolit is not as great as it seems.

The first part of your statement is true, in my opinion, Charles. UNDER MY TESTING CONDITIONS and below 5/6 minutes Time Control Houdini 1.3a is the clear number one. Above 8/9 minutes Time Control DR4 is the clear number one under the same conditions.

The second part of your statement implies a subjective judgement: 'No one can improve these engines beyond Rybka at longer time controls'. As an objective tester I am not able to know if anyone will be able to do this. But I think the probabilities are relatively high. The reason: From now until next Rybka issue we will have to wait more than one year, if last issuing rythm aplies. (As a Rybka customer, I would be delighted to get a new version sooner, of course!). In this time Ippo will issue between 50 and 100 new IPPO engines and/or compiles. Perhaps one of them will be stronger than Rybka at long time controls. And also Robert Houdart is doing a fantastic task.

This is an extremely exciting time for computer chess lovers, isn't it?.

Regards from Barcelona,

Tom.

[kingliveson]

This is very interesting and seems to imply that Rybka 4 is still #1 at longer time controls than 7'0!

Thus NO ONE can improve these engines beyond Rybka at longer time controls. Ippolit is not as great as it seems.

Bold but unsubstantiated statement! There is no data supporting Rybka 4 being #1 at longer time control. And exactly what does longer time control mean -- faster hardware?

If one plays 40/40 repeating time control on 2000 MHz single core CPU, does that count? The last tournament I ran on such hardware, though only 30 games between IvanHoe and Rybka, it was even.

Another such tournament with latest IvanHoe is underway, and after 19 games, result is even at +6/=7/-6. IvanHoe (±.66) just blundered and Rybka (±.88) might win the 20th round. The point is, there is not enough data to say whether Houdini, IvanHoe, or Rybka is #1 at long time control. Until such data is presented, it's all speculation.

I do plan on doing some 40/40 tournament with Houdini, IvanHoe, Naum, Rybka, Shredder, and Stockfish when the time is right.

[robbolito]

Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz4x 3500 MHz with 4,096 MB Memory
Microsoft Windows XP 64 Bit Professional Service Pack 2 (Build 3790)
Fritz Benchmark:
Speed: 20.90
KNS: 10032
GUI: CB Rybka 3
Book: Perfect 2009 10 moves
Hash: 256
RB and TB: ON
Ponder: OFF


Blitz:10' 0

Code: Select all

1   Houdini 1.03a x64 4_CPU  3195  +5/-4/=21 51.67%   15.5/30
2   Deep Rybka 4 x64         3190  +4/-5/=21 48.33%   14.5/30

[robbolito]

AMD Phenom(tm) II X6 1090T Processor6x @ 3.2 GHz 4,096 MB Memory
Windows 7 Home Premium Edition (Build 7600)
Fritz Benchmark:
Speed: 23.33
KNS: 11196
GUI: CB Rybka 3
Book: Perfect 2009 10 moves
Hash: 256
RB and TB: ON
Ponder: OFF

Blitz:10' 0

Code: Select all

1   Houdini 1.03a x64 8_CPU  3190  +25/-3/=12 77.50%   31.0/40
2   Deep Shredder 12         3130  +3/-25/=12 22.50%    9.0/40

Deep Shredder 12 about 120$.
Houdini 1.03a free.