On "clone testing"

General discussion about computer chess...
User avatar
kingliveson
Posts: 1388
Joined: Thu Jun 10, 2010 1:22 am
Real Name: Franklin Titus
Location: 28°32'1"N 81°22'33"W

Re: On "clone testing"

Post by kingliveson » Tue Dec 28, 2010 7:11 pm

Sentinel wrote:
Edit: In case you don't know how to do it, just add another engine in you test called Fruit 2.1_identical_copy and see how many identical moves you would get with Fruit 2.1. ;)
kingliveson wrote:The data plotted is actual representation of the output produced by the "similarity tool."
No it's not. You have 24 data points in your plot that you've just invented.
You can be entertaining at times seriously. You are given a set of data based on a scale of 100% but you cannot see that. You want to introduce more data that will change that scale, again which is another subject. lol, might I remind you that we've had the probability of repeated positions discussion before in which you participated and I carried out the tests and posted the results -- see talkchess 31170.

Edit: May be you can tell me how to plot a relationship tree of the sample posted without introducing more samples.
PAWN : Knight >> Bishop >> Rook >>Queen

Sentinel
Posts: 122
Joined: Thu Jun 10, 2010 12:49 am
Real Name: Milos Stanisavljevic

Re: On "clone testing"

Post by Sentinel » Tue Dec 28, 2010 7:33 pm

kingliveson wrote:You can be entertaining at times seriously. You are given a set of data based on a scale of 100% but you cannot see that. You want to introduce more data that will change that scale, again which is another subject. lol, might I remind you that we've had the probability of repeated positions discussion before in which you participated and I carried out the tests and posted the results -- see talkchess 31170.

Edit: May be you can tell me how to plot a relationship tree of the sample posted without introducing more samples.
It's not the matter of scale or introducing new data. You've introduced new data when you've plotted 24 non-existing points.
It's fairly simple, this is why I don't understand why you have a need to plot those points, simply omit 24 points on the edge of the circle (http://open-chess.org/viewtopic.php?p=7640#p7640) and you would get a correct plot and an accurate representation.

You might think of it as nitpicking, but it's important coz with your original plot you convey a totally different message.

User avatar
kingliveson
Posts: 1388
Joined: Thu Jun 10, 2010 1:22 am
Real Name: Franklin Titus
Location: 28°32'1"N 81°22'33"W

Re: On "clone testing"

Post by kingliveson » Tue Dec 28, 2010 9:23 pm

Sentinel wrote:
kingliveson wrote:You can be entertaining at times seriously. You are given a set of data based on a scale of 100% but you cannot see that. You want to introduce more data that will change that scale, again which is another subject. lol, might I remind you that we've had the probability of repeated positions discussion before in which you participated and I carried out the tests and posted the results -- see talkchess 31170.

Edit: May be you can tell me how to plot a relationship tree of the sample posted without introducing more samples.
It's not the matter of scale or introducing new data. You've introduced new data when you've plotted 24 non-existing points.
It's fairly simple, this is why I don't understand why you have a need to plot those points, simply omit 24 points on the edge of the circle (http://open-chess.org/viewtopic.php?p=7640#p7640) and you would get a correct plot and an accurate representation.

You might think of it as nitpicking, but it's important coz with your original plot you convey a totally different message.
Believe when I tell you that the second your post was read and you mentioned 100 and self-test, I understood the difference. Let me try to explain the 100 percent; engines A and B are given 8 positions, and B plays the exact 6 moves as does engine A each successive run, which means there is 75% agreement. It does not mean that A will play the same move each time thereby agreeing with itself 100%. It comes down to how many moves picked by B are the same ones picked by A. It is simply a matter of ratio between the pairs.

Take a look at your suggestion regarding removing the assumed introduced values -- smaller data sets used for readability. You will notice the graphs are the same with one exception; a reference point. The reference point is 100 in Dailey's "similarity" tool.
Attachments
20101228_similarity_data_plot.png
PAWN : Knight >> Bishop >> Rook >>Queen

User avatar
kingliveson
Posts: 1388
Joined: Thu Jun 10, 2010 1:22 am
Real Name: Franklin Titus
Location: 28°32'1"N 81°22'33"W

Re: On "Similarity Testing"

Post by kingliveson » Thu Dec 30, 2010 2:31 am

I did the test over with time increased to 1/4 th of a second for all engines except critter because it had the advantage of 4 cores. Naum 4.2 and Strelka 2.0 seem to have very similar style getting a little over 70% agreement. Ippolit was also tested, but the numbers were weired so it's not included.
********        Cr	Fr	Ho	Iv	Ko	Ko	Na	Ro	Ry	Ry	Ry	Ry	St	St
Critter 0.90	100	57.5	66.2	66.45	57.95	59.4	61.25	65.8	59.35	60.75	62.95	61.3	60.3	61.25
Fruit 2.1	57.5	100	57.5	58.5	60.2	57.95	61.4	57.75	63.7	64.45	59.2	58.85	58.95	64.3
Houdini 1.0	66.2	57.5	100	75	59.8	61.05	63.3	75.85	62.4	63.9	68.35	65.5	61.9	62.85
IvanHoe 9.81b	66.45	58.5	75	100	60.6	60.85	64.1	77.1	62.05	64.7	69.4	64.4	61.75	63.55
Komodo 1.0	57.95	60.2	59.8	60.6	100	65.45	61.35	59.45	62	63.05	63.7	61.2	58.15	63.6
Komodo 1.3	59.4	57.95	61.05	60.85	65.45	100	60.35	61	60.1	61.95	62	60.35	57.55	60.75
Naum 4.2	61.25	61.4	63.3	64.1	61.35	60.35	100	63.75	69.65	71.9	66.55	65	61.4	71.35
RobboLito 0.90	65.8	57.75	75.85	77.1	59.45	61	63.75	100	62.95	64.85	68.15	65.25	61.05	63.35
Rybka 1.0 	59.35	63.7	62.4	62.05	62	60.1	69.65	62.95	100	69.9	64.65	63.4	60.3	74.15
Rybka 2.3.2a	60.75	64.45	63.9	64.7	63.05	61.95	71.9	64.85	69.9	100	67.95	66.25	61.55	73.5
Rybka 3	        62.95	59.2	68.35	69.4	63.7	62	66.55	68.15	64.65	67.95	100	67.5	60.85	65.15
Rybka 4	        61.3	58.85	65.5	64.4	61.2	60.35	65	65.25	63.4	66.25	67.5	100	60.05	63.6
Stockfish 1.9.1	60.3	58.95	61.9	61.75	58.15	57.55	61.4	61.05	60.3	61.55	60.85	60.05	100	60.7
Strelka 2.0	61.25	64.3	62.85	63.55	63.6	60.75	71.35	63.35	74.15	73.5	65.15	63.6	60.7	100

Image
Attachments
20101229_similarity_test.7z
similarity test data
(341.97 KiB) Downloaded 338 times
PAWN : Knight >> Bishop >> Rook >>Queen

supersharp77
Posts: 12
Joined: Thu Jul 24, 2014 12:35 am

Re: On "clone testing" ie "Toby Tal" info

Post by supersharp77 » Thu Jul 24, 2014 1:11 am

Concerning the so called "Toby Tal" chess engine.........was it actually a chess engine .exe file or was it a engine hidden within the toby chess gui? Few seem to know as few copies of this so called chess engine were sold to the public at large.....AAR

Post Reply