ChessBase: A Gross Miscarriage of Justice in Computer Chess

[wgarvin]

(1) Most of the panel members were experienced programmers active in computer chess, who had written at least one chess engine. A small number (including myself) were experienced programmers with a long-standing interest in computer chess, but who had not written a complete chess engine. There were a few non-programmers who signed up, but they did not participate much in the discussion of the strength of the evidence.

(2) there was no rule that everyone would have to vote. Some people signed up just to see what was going on, or decided after signing up not to participate much. Users who tried to sign up were vetted by the Secretariat to make sure the discussions would be productive and civil (i.e. to avoid the noise you see in talkchess General, where everybody has opinions whether they are qualified to have them or not).

(3) The purpose of the panel was NOT to find Vas guilty or innocent. It was the Panel's job to collect the evidence together, make a technical assessment of the evidence, and advise the ICGA Board (a 5-person body led by Dr. Levy) about whatever the panel found. If you want to compare it to a courtroom, the Panel was NOT like a judge or jury--it was more like an "expert witness for the prosecution".

(4) Because no Rybka source was available, semantic comparisons had to be done by reverse-engineering the executable code and matching up the parts of an assembly listing with Fruit source, etc. Thus, not all programmers are qualified to judge the strength of that evidence (and its above the heads of laypersons). You need years of experience with x86 assembly code, a decent grasp of what optimizing compilers do when they compile source code into x86 machine code, AND the chess-programming experience to judge how significant a particular snippet of code is for the originality of the whole. Anyways, the pool of such people in the whole world is not that big (a few hundred? a few thousand?). A few people did most of the work, but (1) it took them months of effort, (2) they were emminently qualified to do it, and (3) very few people had both the skills and the interest to carry out the work of investigating, comparing, and documenting what they found. The rest of the Panel then studied the evidence, debated its significance, and tried to poke holes in it. When the panel voted, everyone who participated in the vote was convinced that significant amounts of Fruit 2.1 had been copied by Rybka 1.0 Beta and that Vas had broken rule 2. (there was a lot of copying of Crafty code into earlier versions, too... Not all of the evidence was even discussed, since the Fruit-like eval was already enough).

(5) as Dr. Levy has explained several times, Vas was given LOTS of chances (at least 4 or 5) to present any evidence or arguments he wanted. We would all have welcomed some sort of response from him. But he declined tp join the panel. He did not provide any source code to help make comparisons. He was sent a preliminary set of evidence, and again ignored it. The panel voted, a report was prepared by the Secretariat to give to the Board. That report was sent to Vas and he was given time to respond, and again he ignored it. He decided not to engage with the process in any way, in effect, he decided not to defend himself at all. This probably contributed to the harshness of the ICGA Board's decision (the lifetime ban).

[lmader]

I appreciate that you are trying to wend your way through this muddle. I think it is hard for people to sift through.

I have seen NO clear definition or guidelines presented by ICGA in their examination of Rybka.

Personally, the guidelines used by the ICGA to make their determination aren't that interesting to me because I think the more important issue is simply the issue of "did Vas copy code from Fruit". This question is simpler and more important to the general concern of whether there was wrong doing on the part of Vas. Why is it simpler? Because, and this is important, the Fruit code was covered by a license. It is "open-source" code, but it is NOT public domain. This means that copying the code is subject to the terms of its license. If Vas broke these rules, he is offending the law, as well as programmers in any domain (i.e. not just chess programmers).

The Closest I have seen is: from chessvibes.com

Watkins: "Furthermore, [Riis] is similarly lacking in any knowledge of the relevant aspects of copyright law, particularly the Abstraction-Filtration-Comparison Test that formed a basis for one part of the ICGA Panel analysis. He appears to apply a minimalist copy/paste standard to what “copying” might mean, ignoring any other creative aspects."

emphasis is mine.... what are the other parts? WHY add other parts? the test was created to determine guilt (

I think the implication here is that copyright law can extend to code that isn't _exactly_ copy/paste, but close enough due to some level of minimal modification. As an example, perhaps the code isn't exact because Vas took Fruit and changed the board representation to bitboards, but otherwise didn't change things. This would still be considered a violation of copyright law because there are still extremely significant elements of the Fruit code that were carried over. I suppose this is an area that can be difficult to assess without some technical knowledge of the specific situation.

Essentially ICGA made a ruling and they can do what they want since it is a private organization. they need no evidence or even proof just a vote. In this case a simple we dont like you and we THINK you did something wrong would be enough evidence for the ICGA to throw Rybka out of their system. Again ICGA can do what they want.
the problem they tried to create an illusion of a legal process to indicate credibility.

I'm not sure that the ICGA can be blamed for any perception of legal process. Their ruling was simply very visible. On the other hand, I think the report given to the board contains evidence that stands on its own, and that is the more relevant aspect here. That is, does that evidence stand up? I don't see the point in worrying about how fair the ICGA's decision was. In the end it's about that fundamental question - "did Vas copy code from Fruit."

it absolutelylooks likecode was copied and the GPL was therefore violated.

Looks like , similar, vaguely ,.... understand the base code isnt available but this begs doubt

I use this language because I think that absolute certainty is always difficult. But this I feel 100% certain about - there is enough evidence there that it demands a real refutation from Vas. He could have played this much more openly. There were members of that panel that have no vested interest in computer chess anymore, such as Ken Thompson - who is absolutely luminary and beyond reproach by the way - with whom he could have negotiated an agreement to view his source code or whatever.

Also, it's important to keep in mind that some people simply cannot be convinced regardless of the amount of evidence. There are people who will deny the holocaust ever happened. Can you prove it 100%? You can cite eye-witnesses, journals, logs from the prison camps, etc., and yet some people can still find a way to doubt.

A couple of people have attempted to introduce FUD (fear uncertainty and doubt) into the interpretation of the investigation's code comparison, but none of that FUD is valid. Sadly I guess one has to have some programming experience to be able to evaluate that statement

.
And how do you consider the evaluation of those people that are expert 3rd party programmers that have expressed doubt?

To my knowledge, there are really only a few chess programmers expressing doubt. Ed Shroder, Chris Whittington, and Miguel Ballorca (apologies for misspelling their names, i know I didn't get them exactly right) being the ones with which I am familiar. Ballorca has, to my knowledge, been reasonable, but the only thing he has argued is that the Piece Square Table evidence isn't convincing. I agree that by itself the PSTs are not compelling, so no big deal. I do think that given the rest of the evidence it fits. As far as Shroder and Whittington, their form of argument has been steeped with distortion and manipulation of the facts. I cannot explain what has happened to Shroder, who once had a decent chess engine and thus should be accorded some respect. Most of the people on this forum will agree that regardless of the VIG vs VII thing, he has lost a lot of respect and credibility. So to answer your question, I don't think Shroder or Whittington are credible. I don't have an explanation for their behavior though, except to comment that it really has sounded like the ravings of madmen.

The Fruit code was protected by a software license that doesn't allow code to be copied without adhering to the terms of that license. The answer is that there cannot be _any_ code copied. Just because some parts of the program were dramatically changed, does not move it along some continuum towards being ok. If there is code left in Rybka that was copied, it's a violation. No gray area.

Maybe in an academic world yes but read the article http://en.wikipedia.org/wiki/Abstractio ... rison_test
"Eliminating elements dictated by external factors is an application of the scènes à faire doctrine to computer programs. The doctrine holds that elements necessary for, or standard to, expression in some particular theme can not be protected by copyright.[8] Elements dictated by external factors may include hardware specifications, interoperability and compatibility requirements, design standards, demands of the market being served, and standard programming techniques.[9]
Finally, material that exists in the public domain can not be copyrighted and is also removed from the analysis.[2]"
So code can be copied... again the modification of things are covered as well in the article so the area is grey .

I am a layperson in this issue. I have actually zero interest in Vas as a person or anyone else.
My questions are based on a sense of justice. When I look at what happened I see some serious gaps in what I would consider a fair, open and just process.

Certainly a fair question. The first part of the above statement "... elements necessary for, or standard to, expression in some particular theme can not be protected..." could in principle possibly apply to something like a UCI interface in a chess engine. Probably not, but that's ok. In any case, much of the code in question is unique to the implementation of the engine and doesn't fall into this category.

The "public domain" section also doesn't apply - Fruit was not public domain, but instead covered under the Gnu Public License (GPL).

The [ICGA] board formed the panel by asking Mark, Harvey and myself to "chair" the panel and to enlist the aid of others as we saw fit. -Hyatt

Shouldnt the main requirement been to have programmers on the panel? Not SOME not partially but entirely composed of experts and programmers of equal level not 1 or 2 that could sway the group. The problem with 1 expert and 10 amateurs would be that the 1 experts opinion could sway the others. Hyatt formed and led the panel even with this the panel returned a 14-0 out of 34 members .
. with 34 members a 14-0 result in guilt so 20 members of the panel abstained?! if they chose not to vote or participate or anything else why were they asked to be on the panel? That amount indicates a clear problem in 1) the evidence was not convincing enough to sway enough members to get at 50% to vote guilty. 2) or that the committee was flawed in its formation with members on the panel who shouldn't have been there in the first place.

I guess I'm just not that interested in the process used by the ICGA. Zach W, Mark W, Bob Hyatt, etc spent a lot of time researching this and produced a report with their evidence. Regardless of the ICGA, the evidence has been produced, and it is this evidence that needs to be considered. It stands on its own; Whether or not one considers it convincing is the only real question.

The ONLY evidence presented was by the opposition. Vas was asked to offer a defense but well providing code to a group that was formed and led by your critics? Who would jump on that wagon.

I don't agree, as mentioned above. Vas was accused of code copying and some very serious evidence was presented. If he was really innocent he could have negotiated a way to defend his code. There were some excellent people available.

The program examined was NOT the same version that competed in the ICGA championships.

This is an example of the misinformation out there - we know that one of the Rybka 2.3.2 variations competed in the ICGA. They examined one of the 2.3.2 versions. Was it the exact same one? It really doesn't matter because they were all insubstantially different. (i.e. there were a bunch of very minor releases of 2.3.2, labelled with a suffix like 2.3.2a, 2.3.2n, etc., with very little difference between them). It really is crazy to imagine that the version of 2.3.2 that competed was somehow completely different from the 2.3.2 examined.

[Prima]

The program examined was NOT the same version that competed in the ICGA championships.

Judoka, it does really help to be cognizant in issues BEFORE jumping in & making assertions that turns out to be non factual, or at best, contradictory.

Here is proof from Lukas Cinmiotti affirming that Rybka 2.3.2a mp x64 did in fact play in WCCC 2007 (ICGA). This announcement directly contradicts your assertion that the Rybka 2.3.2a version examined was not the same Rybka 2.3.2a version that competed in ICGA [and/or WCCC] in 2007:



Uploaded with ImageShack.us

Also here, mjlef expresses the same concern of possible differences in Rybka 2.3.2a versions that played/ or not played in WCCC 2007. Vas confirms, AGAIN, as of today January 11 2012, that the Rybka 2.3.2a that was examined (in ICGA investigation) CAN be used for the Rybka 2.3.2(a) version that competed in WCCC 2007:



Uploaded with ImageShack.us

In reiteration, YOUR assertion(s) directly contradicts Lukas Cinmiotti's & Vasik Rajlich's statements. The rest of your current posts, including previous ones, have already been stated ad nauseam by those who hate adherence to licenses/rules/law/ethics/morals, only to be disproved with evidences & simple moral lessons.


@ K I Hyam: I agree with you. I didn't notice you already researched Luka Cinmiottie's public admittance of using Rybka 2.3.2a mp x64 in WCCC 2007. In any event, the link you provided is correct but broken or unable to link to the specific Rybka forum page of said 'confession'. I pulled it up and captured it, just in case the post somehow 'mysteriously vanishes' from Rybka forum, seeing it contradicts them in a major way.

@ Imader: I also agree with you. Apparently there are lots of misinformation circulating out there. In general, it clearly shows lots of Rybka zealots DON'T know nor understand exactly WHAT they're arguing for, or against.

[hyatt]

The program examined was NOT the same version that competed in the ICGA championships.

- Convicting someone for an activity that didn't take place in their tournament seems to be grasping for straws. If this was the case it would have been much easier to determine guilt by using version that was used that had clear cases of copying from Crafty that participated in events?
The panel did not present a clear guideline and criteria to follow to prove guilt or innocence."The Abstraction-Filtration-Comparison test (AFC) is a method of identifying substantial similarity for the purposes of applying copyright law." This IS the method to use to add steps makes the process questionable.The AFC process was only 1 step? What were the other steps used?
While evidence is very critical so is the manner in which it is examined. There have been people who have been tried and convicted by juries that were 100% sure of their decision based on the evidence presented only to be later proved innocent by additional evidence. The DNA in this case is the original code but after 5 years of accusations drama I dont blame Vas for basically giving the group the silent treatment when they asked him for it.

The first statement above is simply wrong, and has been proven to be wrong. Rybka 2.3.2a competed in an ICGA event. The operator / team member Lukas Ciamotti CLEARLY stated this in a Rybka Forum discussion that was going on DURING the tournament. He used two versions, 2.3.2 and 2.3.2a. We have provided links to this specific post several times, so there is absolutely no point in making a statement that is easily refuted.

The panel used the "abstraction - filtration - comparison" methodology. We HAD to, because we had to eliminate the differences caused by different board representations. It is quite obvious you don't know what has happened, so in that light, what is the point in making comments that are completely pointless??? Comparing binary to C is NOT complicated. Just time-consuming. It is 100% reliable. It is done every day. To imply otherwise shows a lack of understanding, not a lack of "fair process" within the ICGA panel investigation.

[Judoka]

More nonsense. This was not a jury trying a legal case, a "beyond reasonable doubt" verdict was not required. This was simply an internal affair; a panel deciding on whether someone had broken the rules of a private club. The composition of the panel and the way in which they reached a decision was therefore a private matter. There is no obligation for a member of a private club to be present at a disciplinary hearing and there is not even an obligation on those who are conducting the hearing to invite him.

The version examined was 1.3.x correct? saying they are basically the same does NOT mean they are the same and would get thrown out in court so fast it would make a first year law student laugh for trying to compare the two.

Am I the ONLY one that sees the contradiction in saying the data needed to be presented in the modified format to make it easier to understand but the panel was formed by experts? Its either one or the other.


The ICGA proceedings were only superficially similar to a court system that could be considered fair and just. The Panel formation, and the decision making process just superficially similar to a legal proceeding to give an illusion of a proven system so that resulting decision fair and just would gain greater public support.
Any talk of guilt or innocence outside of the ICGA is a moot , irrelevant and nonsense. Your right it was internal decision and investigation so the ICGA could have based their decision on any criteria they wished; the fact they didnt like how Vas smelled, his accent or any other arbitrary term they wanted to use. (being intentionally absurd to point out that either you make a serious attempt or you are just making the motions.
Thank you for clearing up the details this is exactly what i was trying to find out.
Conclusion:
ICGA decision was based on an internal investigation.
It used its own internal methodology to come to its decision.
The entire discussion of guilt or innocence is moot because it has no relevance outside of the ICGA.

[Harvey Williamson]

The version examined was 1.3.x correct? .

Lots of versions were looked at in the R1 and R2 series including the 1 Lukas confirms played.

[wgarvin]

Judoka seems to have reached his desired conclusions already, and I suspect no facts or argumentation will sway him.

If he finds the replies to his posts a bit shrill/indignant, I hope he understands that it is because these objections have been raised already several other times over the last 8 months, and the people who understand the facts are getting tired of explaining them over and over to casual onlookers who seem more interested in pushing their chosen agenda than actually knowing the truth about what happened.

The facts are: Vas copied code verbatim from Crafty (one of the strongest open-source programs of the time) into Rybka 1.6.1. Then Fruit 2.1 was released, and it was even stronger... and Vas went through it backwards and forwards and "took many things". This included some UCI/time control code (replacing the UCI code he had already written), some parts of the search code, and the part that the panel focused on: virtually the entire eval, which he translated to bitboards. This became Rybka 1.0 Beta. Between this version and the later version Rybka 2.3, the eval code remained almost completely unchanged. Rybka 2.3.2a had some small changes to that code, but it was still clearly derived from Fruit 2.1.

Now, the point of the ICGA rule 2 is to create a level playing field. The WCCC is a competition between human programmers, to see who can write the best engine. To make it fair, each team has to write their engine by themselves. Any third-party code they use, has to be acknowledged on the entry form, and they need the permission of the author(s) of that code, and those authors are treated as "part of the team" (in other words, an author can only contribute to one entry in the tournament). Vas did not tell anyone that his engine contained code copied from Fruit. He claimed it was entirely his own original work. He did this in 5 successive years at the WCCC and a few other tournaments.

Now, try to understand the magnitude of his transgression: To write from scratch, an eval as good as Fruit 2.1's, would have taken at least several months, perhaps a year--and there is no guarantee he would even succeed and make one that good. (the eval he copied was one of the strongest in the world, at that time). Instead, he copied and translated it, which probably took just a few days, and got a "guaranteed strong" eval. Fabien Letouzey's hard work was stolen by Vas, who pretended it was his own work. Rybka won world championships and became famous, and took commercial opportunities away from Fruit. In every tournament, Vas had a significant unfair advantage over his honest competitors who had taken the time and effort to write their own evals from scratch.

In summary: He cheated, he lied about it, claimed as his own work the things he had plagarized from others. Eventually he got found out, and a lot of evidence was collected to show how extensive the copying was, and he ignored the ICGA, whose tournament rules he had broken. Did not fess up, did not provide any reasonable explanation. So that's why they banned him for life from their "private club".

Now Vas did a lot of his own work on Rybka, obviously. But if you're looking for the real "miscarriage of justice" here, its that he copied lots of code from Bob Hyatt's Crafty and Fabien Letouzey's Fruit into his own programs. He violated their licenses (and thus infringed their copyright), he pretended Rybka was all his own work, and won world championships with it. The real "miscarriage of justice" is that the truth remained hidden for so long. No one is to blame for these things except Vas.

[K I Hyams]

The version examined was 1.3.x correct?

No.

Thank you for clearing up the details this is exactly what i was trying to find out.

We might have told you what you needed to know, the trouble is that you only heard what you wanted to hear.

[hyatt]

More nonsense. This was not a jury trying a legal case, a "beyond reasonable doubt" verdict was not required. This was simply an internal affair; a panel deciding on whether someone had broken the rules of a private club. The composition of the panel and the way in which they reached a decision was therefore a private matter. There is no obligation for a member of a private club to be present at a disciplinary hearing and there is not even an obligation on those who are conducting the hearing to invite him.

The version examined was 1.3.x correct? saying they are basically the same does NOT mean they are the same and would get thrown out in court so fast it would make a first year law student laugh for trying to compare the two.

Incorrect. Why not read the ICGA report and evidence FIRST?

We examined Rybka 1.6.1, Rybka 1.0 beta, and Rybka 2.3.2a.

ALL three.

This has been stated SEVERAL times. In this thread. But more importantly, in the ICGA report which you obviously have NOT read. But you are still telling us what we did wrong without even knowing what we did? Read the report FIRST. Then your comments won't look so foolish and uninformed...

Am I the ONLY one that sees the contradiction in saying the data needed to be presented in the modified format to make it easier to understand but the panel was formed by experts? Its either one or the other.


The ICGA proceedings were only superficially similar to a court system that could be considered fair and just. The Panel formation, and the decision making process just superficially similar to a legal proceeding to give an illusion of a proven system so that resulting decision fair and just would gain greater public support.
Any talk of guilt or innocence outside of the ICGA is a moot , irrelevant and nonsense. Your right it was internal decision and investigation so the ICGA could have based their decision on any criteria they wished; the fact they didnt like how Vas smelled, his accent or any other arbitrary term they wanted to use. (being intentionally absurd to point out that either you make a serious attempt or you are just making the motions.
Thank you for clearing up the details this is exactly what i was trying to find out.
Conclusion:
ICGA decision was based on an internal investigation.
It used its own internal methodology to come to its decision.
The entire discussion of guilt or innocence is moot because it has no relevance outside of the ICGA.

It apparently has a LOT of relevance outside the ICGA based on the length of the discussion.

[lmader]

The version examined was 1.3.x correct? saying they are basically the same does NOT mean they are the same and would get thrown out in court so fast it would make a first year law student laugh for trying to compare the two.

What the heck is wrong with you? Have you even read the replies before yours?

Am I the ONLY one that sees the contradiction in saying the data needed to be presented in the modified format to make it easier to understand but the panel was formed by experts? Its either one or the other.

My previous replies were based on my sense that you were really trying to understand this. I was wrong. You don't get it because you don't want to.

The ICGA proceedings were only superficially similar to a court system that could be considered fair and just. The Panel formation, and the decision making process just superficially similar to a legal proceeding to give an illusion of a proven system so that resulting decision fair and just would gain greater public support.

Crikey, so all of these people had it in for Vas? This is quite a conspiracy theory.

Any talk of guilt or innocence outside of the ICGA is a moot , irrelevant and nonsense.

Actually, you have it backwards.The ICGA's decision is simply their decision. Vas' guilt will live on forever.

Your right it was internal decision and investigation so the ICGA could have based their decision on any criteria they wished; the fact they didnt like how Vas smelled, his accent or any other arbitrary term they wanted to use. (being intentionally absurd to point out that either you make a serious attempt or you are just making the motions.

Thank you for clearing up the details this is exactly what i was trying to find out.
Conclusion:
ICGA decision was based on an internal investigation.
It used its own internal methodology to come to its decision.
The entire discussion of guilt or innocence is moot because it has no relevance outside of the ICGA.

Remarkable. You clearly haven't even read the report, and yet you've figured out that the ICGA's approach was a conspiracy to discredit Vas, that their methods were arbitrary, and that none of the evidence is valid. Again, all this without even studying the documents. And furthermore, you have the hubris to insult their methodologies, when your methodology of ignoring the report documents is far more flawed. How can you possibly hope to have an informed opinion if you haven't even looked at thge frickin' report? What a joke. And here I thought you were actually trying to understand this stuff. What a sad waste of time.