Page 1 of 2
Fritz (12) and the limit strength feature
Posted: Sun Feb 17, 2013 4:16 pm
by Alexander Schmidt
Does anyone know how to use UCI Limit strength features under Fritz? In the engineoption menu the feature is not available, so it must be somewhere else.
Under Training -> Rated games Fritz loads the UCI engine, sends the correct ELO value to the engine and then calculates itself. The UCI engine sleeps.
I really don't like that I implement some nice ELO feature, and the user thinks the stupid blunder moves by Fritz are done by my engine...
Re: Fritz (12) and the limit strength feature
Posted: Mon Feb 18, 2013 1:07 am
by kinderchocolate
As far as I can remember, only Arena properly supports the UCI strength limit option. By the way, I've been studying how you implemented limit strength features and might include one of yours in my new GUI application.
Re: Fritz (12) and the limit strength feature
Posted: Mon Feb 18, 2013 7:49 am
by Alexander Schmidt
If a GUI has no special implementation of the limit strength feature the options UCI_LimitStrength and UCI_Elo should at least appear in the engineoptions menu.
Shredder has also a good implementation.
Re: Fritz (12) and the limit strength feature
Posted: Tue Feb 19, 2013 1:24 am
by kinderchocolate
How do you limit the strength? You slow the search, limit the number of nodes etc. But is this enough? And how do you tune and verify the parameters are correct?
Re: Fritz (12) and the limit strength feature
Posted: Tue Feb 19, 2013 7:14 pm
by Alexander Schmidt
The strength reduction is done only by limitation of the nodes per second. It works for SlowChess, with modern engines it is tricky because of presearch functions.
The tweaking is don by games against Emulations of dedicated machines in the range between 1450 and 2450 SSDF ELO. This games where played with the 2.96LS, the formula was simplyfied in the last versions, but the difference is tiny.
Code: Select all
1464 ELO Mephisto III*
1824 ELO Rebel 5 Emulation
1904 ELO MM IV Emulation**
1971 ELO Dallas Emulation
1974 ELO MM V Emulation**
2030 ELO Roma 32Bit Emulation
2445 ELO Resurrection Fruit 2.1***
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mephisto III 1464 27,5 / 50 11011111011=10=11001=100=00=10011=1=0=10010001=011 1500 +25
2: Mysticum SlowChess ELO 1464 1464 22,5 / 50 00100000100=01=00110=011=11=01100=0=1=01101110=100 1428 -25
----------------------------------------------------------------------------------------------------------------
50 games: +24 =9 -17
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mysticum SlowChess ELO 1824 1824 32,5 / 50 =01=10110111101=0110110=110=1=101111=1011011100110 1934 +75
2: Mephisto Rebel 5 1824 17,5 / 50 =10=01001000010=1001001=001=0=010000=0100100011001 1714 -75
----------------------------------------------------------------------------------------------------------------
50 games: +24 =7 -19
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mysticum SlowChess ELO 1769 1769 25,5 / 50 100=11=11001====101=010=10=1010=0011110001010101=0 1831 +5
2: Mephisto Rebel 5 1824 24,5 / 50 011=00=00110====010=101=01=0101=1100001110101010=1 1817 -5
----------------------------------------------------------------------------------------------------------------
50 games: +16 =11 -23
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mephisto MM IV 1904 26,5 / 50 0010===10011=011011011=011101==01001001=1011=01==0 1925 +15
2: Mysticum SlowChess ELO 1904 1904 23,5 / 50 1101===01100=100100100=100010==10110110=0100=10==1 1883 -15
----------------------------------------------------------------------------------------------------------------
50 games: +22 =11 -17
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mysticum SlowChess ELO 1927 1927 26,5 / 50 0011100=01=11100001=0=11==1===1==11=001=1====01001 1948 +15
2: Mephisto Amsterdam 1927 23,5 / 50 1100011=10=00011110=1=00==0===0==00=110=0====10110 1906 -15
----------------------------------------------------------------------------------------------------------------
50 games: +22 =17 -11
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mephisto Dallas 16Bit 1971 25,0 / 50 ===11==110=1=1111=01==0=100100000011=0=011=0000110 1971 +0
2: Mysticum SlowChess ELO 1971 1971 25,0 / 50 ===00==001=0=0000=10==1=011011111100=1=100=1111001 1971 +0
----------------------------------------------------------------------------------------------------------------
50 games: +25 =14 -11
Rtng Score 1234567890123456789012345678901234567890123 Perf Chg
---------------------------------------------------------------------------------------------------------
1: Mephisto MM V 1974 22,0 / 43 1=1011101011=001010011=0110=0=000==11=00011 1981 +4
2: Mysticum SlowChess ELO 1974 1974 21,0 / 43 0=0100010100=110101100=1001=1=111==00=11100 1967 -4
---------------------------------------------------------------------------------------------------------
43 games: +18 =8 -17
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mysticum SlowChess ELO 2030 2030 29,0 / 50 ==10011=1011=0=01=1=101001=1111=1=1=1=0=11=0001010 2087 +40
2: Mephisto Roma 32Bit 2030 21,0 / 50 ==01100=0100=1=10=0=010110=0000=0=0=0=1=00=1110101 1973 -40
----------------------------------------------------------------------------------------------------------------
50 games: +16 =14 -20
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mysticum SlowChess ELO 2011 2011 25,5 / 50 0110==010===1=11010=010==1100101010100110=10=10110 2037 +5
2: Mephisto Roma 32Bit 2030 24,5 / 50 1001==101===0=00101=101==0011010101011001=01=01001 2023 -5
----------------------------------------------------------------------------------------------------------------
50 games: +22 =11 -17
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
-------------------------------------------------------------------------------------------------------------------
1: Mysticum SlowChess ELO 2445 2445 26,0 / 50 1001==01010=10=00110==0011=1=1=110==101=0=10=11==0 2459 +10
2: Resurrection Fruit 2.1 203 MHz 2445 24,0 / 50 0110==10101=01=11001==1100=0=0=001==010=1=01=00==1 2431 -10
-------------------------------------------------------------------------------------------------------------------
50 games: +22 =16 -12
Here some fast games:
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mephisto III 1464 27,5 / 50 0011101011===001011011101011101=1010000=1111100=0= 1500 +25
2: Mysticum SlowChess ELO 1464 1464 22,5 / 50 1100010100===110100100010100010=0101111=0000011=1= 1428 -25
----------------------------------------------------------------------------------------------------------------
50 games: +28 =7 -15
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mysticum SlowChess ELO 1824 1824 28,5 / 50 0=1111=1=11==0=1011=0010=0=111110110==110010000101 1874 +35
2: Mephisto Rebel 5 1824 21,5 / 50 1=0000=0=00==1=0100=1101=1=000001001==001101111010 1774 -35
----------------------------------------------------------------------------------------------------------------
50 games: +21 =11 -18
Rtng Score 12345678901234567890123456789012345678901234567890 Perf Chg
----------------------------------------------------------------------------------------------------------------
1: Mysticum SlowChess ELO 2030 2030 26,0 / 50 1111000000111001010==1111101010=0101=0==1101000101 2044 +10
2: Mephisto Roma 32Bit 2030 24,0 / 50 0000111111000110101==0000010101=1010=1==0010111010 2016 -10
----------------------------------------------------------------------------------------------------------------
50 games: +31 =6 -13
A quick test of SlowChess 1000 ELO against Microchess 1976, just to be sure that SlowChess 1000 ELO is able to lose against this weakest thing available:
Rtng Score 12345678901234567890123 Perf Chg
-------------------------------------------------------------------------------------
1: Mysticum SlowChess ELO 1000 1000 18,0 / 23 =1=1=10111=111011111110 1310 +94
2: Microchess 1976 ???? 5,0 / 23 =0=0=01000=000100000001 780 -94
-------------------------------------------------------------------------------------
23 games: +11 =4 -8
It is hard to classify ELO values of 1000, so maybe at this level the values are not exact.
*Mephisto III = The Glasgow Emulation at 3,05 MHz. (Mephisto III runs at 6,1 MHz, the Glasgow Emulation is 2x faster because it is 16Bit instead of 8Bit)
**The MM Emulations where not able to play all games because they hang at some positions. Thats why there are less than 50 games.
***Resurrectiion Fruit 2.1 = Fruit 2.1 at the speed of a Resurrection 203 MHz. In the middlegame it plays like the Resurrection, in the endgame it is 5-6% faster which should not have much influence on the strength. The Resurrection Fruit `05 is listed in the SSDF list with 2395 ELO. Fruit 2.1 is around 50 ELO weaker in all ratinglists. Because the SSDF once reduced all ELO values by 100 I have to add this 100 ELO again. So this version should have quite exactly 2445 SSDF(old) ELO.
Re: Fritz (12) and the limit strength feature
Posted: Wed Feb 20, 2013 3:02 am
by kinderchocolate
I see your idea. Just curiosity, the point of reducing strength is to allow a human beat the engine. If the engine isn't calibrated against humans on a real human playing scale, how accurate would that be? For example, if I was a 1400 rated and I played a LS engine 1400 rated. If I don't score around 50%, would that cause any problems?
Re: Fritz (12) and the limit strength feature
Posted: Wed Feb 20, 2013 7:14 am
by Alexander Schmidt
It is of course dificult to compare. One human of1400 ELO maybe beats a computer of 1400 ELO easily, another human will lose. One will try some anti computer chess strategy, another is not aware of such things.
I just can try to get as close as possible to the human values. The ssdf values from the past where quite good calibrated to the human rating because the "new chess playing machines " where allowed to play in human tournaments. Of course I am interested in feedback about the strength of the ELO levels
Re: Fritz (12) and the limit strength feature
Posted: Thu Feb 21, 2013 4:34 am
by kinderchocolate
I've played your LS engines. In fact I downloaded your engines every time you announced on talkchess. I think the reduced difficulty achieved isn't as consistent and smooth as I would have thought. For example, the LS feature in Shredder is better. I felt a bit artificial, I don't know how to express it but it's like Carlsen deliberately make very stupid mistakes because you have a gun pointing at him.
Would randomly selecting inferior PV lines in multipv mode work better?
Re: Fritz (12) and the limit strength feature
Posted: Thu Feb 21, 2013 7:14 am
by Alexander Schmidt
Thanks for the feedback. At what strength and timecontrol did you play?
I didn't observe silly mistakes, probably mainly because I tested with long timecontrol.
Re: Fritz (12) and the limit strength feature
Posted: Fri Feb 22, 2013 6:18 am
by kinderchocolate
I tested your engines with a human (myself). I wouldn't play it with a long control, every game was played at 5 minutes or something like two seconds per move. Maybe you should try a bit more in shorter control. Nobody would bother to sit and play a computer for a standard control. I also don't find much improvement in your LS settings against the default settings in Stockfish.