STS [1-10] TJChess 0121R

[Swaminathan]

http://sites.google.com/site/strategict ... st-results


1000 Positions
10 seconds per position
Hardware: Q6600, 32 bits, 2 GB RAM, 2.4 GHZ. Arena 2.01 GUI.

[Tony H]

Thanks for testing TJchess! I hope to release an updated version this summer.

[BTO7]

Swam ..I'm really interested in this suit idea you have here. A couple of things. First i installed it and ran it. How do you get it into a nice graph like the one above? I have a log file but when you open it ....it has all the detail in a big list. How do i consolidate it like you have for comparison. Second and most important. I see potential here in that ....is it possible to tune your engine have you found? Such as ....have you played with piece values and then gotten better scores? If you have i could see this being able to develop into a very useful engine tuning tool. Or does changing values have no effect and the search is mainly in the code? It would be neat to be able to develop it into a tool that could tune the bishop, knight , rook values to optimum levels and possibly find the increase in tuning much faster like a computer tuning a car racing chip for fuel air delivery ect. Would love to hear your thoughts.

Regards
BT

[Swaminathan]

Hi!

Swam ..I'm really interested in this suit idea you have here. A couple of things. First i installed it and ran it. How do you get it into a nice graph like the one above? I have a log file but when you open it ....it has all the detail in a big list. How do i consolidate it like you have for comparison.

Please refer to this page:
http://sites.google.com/site/strategict ... e/sts-stat

Download STS Stat, Set EPD's from STS Stat folder in Arena "Automatic analysis mode" and open Arena log file using STS Stat once the testing is done.

You will get results in STS Stat - Open as HTML.
Use Image capture program (SnagIt, for instance) to capture the specific region and upload it to the site.

For testing the engine in Arena:
Refer to this
http://sites.google.com/site/strategict ... pd-testing

Make sure that you test one or two engines at a time.

Second and most important. I see potential here in that ....is it possible to tune your engine have you found? Such as ....have you played with piece values and then gotten better scores? If you have i could see this being able to develop into a very useful engine tuning tool. Or does changing values have no effect and the search is mainly in the code? It would be neat to be able to develop it into a tool that could tune the bishop, knight , rook values to optimum levels and possibly find the increase in tuning much faster like a computer tuning a car racing chip for fuel air delivery ect. Would love to hear your thoughts.

Regards
BT

This is very interesting question. But unfortunately I haven't gotten around to making parameter changes/value changes in engine to fine tune. I had thought of doing it using Chessmaster personalities but it's still in my "to-do" list. I've been busy testing all the engines, and at the same time designing new test suites. (Currently I'm working on STS 12)

However, you might like to hear opinions of engine authors, they surely have done this experiment.

Richard Vida, the author of Critter,
Vlad Stamate, the author of Plisk,
Mincho Georgiev the author of Pawny
Gailhac Phillipe, the author of Philou
and, Aaron Becker the author of Daydreamer

....may give you some valuable input. I think they used STS for their tests. They have definitely improved their engines with more updates over the past few months.

[Swaminathan]

Thanks for testing TJchess! I hope to release an updated version this summer.

I'm gladly looking forward to new versions!

Keep up the good work.

[BTO7]

Thank you for your answer Swam. I tried to get the sts stat program to work but nothing shows up in it. Must i complete all the epd's in order for it to work? Im using are arena and tried opening the analysis log but nothing appeared. Anyways as soon as I get it figured out I will play with the idea and if i find out anything will let you know. Im curios if changing piece values only just for the most part gives better eval info but nothing really in strength or if it gives both better eval and strength through better selective moves based on the adjustment of piece values. Great project with great potential

Regards
BT

[Swaminathan]

Thank you for your answer Swam. I tried to get the sts stat program to work but nothing shows up in it. Must i complete all the epd's in order for it to work?

Ofcourse, you should complete all the 10 epds before you can load log files in STS Stat. Make sure you have loaded epd files from the STS stat Folder in Arena automatic analysis mode.

Im using are arena and tried opening the analysis log but nothing appeared. Anyways as soon as I get it figured out I will play with the idea and if i find out anything will let you know.

In Arena, under the "Automatic analysis mode", Go to output and view analysis log. Put the log file in STS stat once the testing is finished.

Im curios if changing piece values only just for the most part gives better eval info but nothing really in strength or if it gives both better eval and strength through better selective moves based on the adjustment of piece values. Great project with great potential

Regards
BT

Thanks, I'm also equally curious if changing the piece values in chessmaster to get the better score in STS would equate to much better playing strength. I'd still have to try that experiment. But it would make sense to wait until I complete atleast 15 STS's to make way for the diversity.

[BTO7]

Ok Swam ...what i have found and a few thoughts I hope can help. First of its clearly YES ...changing piece values can influence the score and not just the avail. Ok now my thoughts on the best way to tune up a engine. So far the main epd I have used is #3 Knight outposts. I figured this was the best place to start since all moves only involved the knights. I played with the piece values till i found a sweet spot. On PeterPans new build 57c i found a adjustment that gave 3 better moves over all or 3%. Standard setting scored 87 ...my setting scored 90. Anyways i think the key thing to tuning a engine is three things. Pawn play, Knight play and bishop play. They are the most complicated pieces to coordinate properly for a sharp middle game. I think queen and rook are different animal but anyways. I think if you had a power epd for the knights ....like #3 but maybe 3-5 hundreds positions and the same for the bishop. Then mix that with a big epd bishop vs knight like epd 5 same thing 3-5 hundred positions and one could really tune these two pieces to the max of a engine. Then increasing your pawn epd #8 and 9 and the rest would fall into place. Anyways i know thats a lot of work but I'm just tossing out ideas because i can see this being a engine super tuning tool as it develops and only gets better. Will save a lot of time of weather or not certain adjustments are actually better and thus we can narrow in on the art of fine tuning these engines thanks to great ideas like STS !

Regards
BT

[Swaminathan]

Ok Swam ...what i have found and a few thoughts I hope can help. First of its clearly YES ...changing piece values can influence the score and not just the avail. Ok now my thoughts on the best way to tune up a engine. So far the main epd I have used is #3 Knight outposts. I figured this was the best place to start since all moves only involved the knights. I played with the piece values till i found a sweet spot. On PeterPans new build 57c i found a adjustment that gave 3 better moves over all or 3%. Standard setting scored 87 ...my setting scored 90. Anyways i think the key thing to tuning a engine is three things. Pawn play, Knight play and bishop play. They are the most complicated pieces to coordinate properly for a sharp middle game. I think queen and rook are different animal but anyways. I think if you had a power epd for the knights ....like #3 but maybe 3-5 hundreds positions and the same for the bishop. Then mix that with a big epd bishop vs knight like epd 5 same thing 3-5 hundred positions and one could really tune these two pieces to the max of a engine. Then increasing your pawn epd #8 and 9 and the rest would fall into place. Anyways i know thats a lot of work but I'm just tossing out ideas because i can see this being a engine super tuning tool as it develops and only gets better. Will save a lot of time of weather or not certain adjustments are actually better and thus we can narrow in on the art of fine tuning these engines thanks to great ideas like STS !

Regards
BT

Yes you're right that more than 200 positions may be needed for the single theme, but I'd like to cover all the remaining ideas before I update the already exisiting suite with many new positions.

It's great to hear that you've been doing a lot of experimenting with the values to fine tune the engine. I'd like to hear more about the results after you're done.
I'm glad to hear your settings is doing better than Houdini in one on one match. Tweak to get more optimised value and report results here!

[BTO7]

My new idea .... POWER TUNING with STS ! Ok folks this is looking promising !! I used STS to power tune PeterPans new 57c Knights and Bishops. The over all gain i got overall was close to 4% overall gain in STS.

Knight back and white..default 300 STS=298 was 3% gain

Bishop pair Black and White default 45 STS=35 WOW!? was 3-5% gain


Then i played 25 games 4+2 ....No ponder ...No books....No TBs....1024 hash ...pawn hash 128 ...2.4Ghz core2duo 4 gig of ram

AFTER TUNING

-----------------Houdini_102w32_2CPU-----------------
Houdini_102w32_2CPU - Ivan57cx32SSE3 : 10.5/25 5-9-11 (===10=010=====010110==000) 42% -56
-----------------Ivan57cx32SSE3-----------------
Ivan57cx32SSE3 - Houdini_102w32_2CPU : 14.5/25 9-5-11 (===01=101=====101001==111) 58% +56


I'm really excited. I have not even tuned the pawns, queen or rooks and this is great results ...the best i have ever had against Houdini no less. Great Job PeterPan !!

Regards
BT