ChessVibes publishes a balanced article re: Rybka

[Judoka]

The commenter (Andrew Dalke) is indeed quite adept at disassembly. However, the document to which he refers (RYBKA_FRUIT) was a preliminary (Mar 11) enumeration of (all) possible evidence [therein it indicates that some of this comes from as-of-yet unchecked sources], and one can note the UCI parsing elements that he mentions were dropped in (say) my recent Recapitulation (and appeared nowhere in the Report, was never much discussed in the Panel, etc.). I agree largely with his sentiments regarding the questionable nature of this evidence (it was removed, after all). I do not agree that this is characteristic of the whole.

This has been my entire point.
Hyatt on another post attempted to dismiss the use of outside experts but seems they are quite capable of examining evidence.
The fact that the document was found lacking it was still a publicly released document with the goal of laying the foundation of guilt. It was found flawed by peers.
I agree but the problem is that the experts in question also have a strong bias due to the years of drama on various blogs. It seems that outside experts are equally capable of making a comparison and fairly quickly as well if presented the evidence in question. Your argument is the correct evidence needs to be presented then do so to these 3rd party programmers.

Two unbiased programmers have said that they do not see enough proof of guilt but this is just dismissed? Why not present the evidence alone to this Hackers website and see if they come to the same conclusion? It can only support the case if they agree but it could also raise questions as well but that is the risk right?

I agree largely with his sentiments regarding the questionable nature of this evidence (it was removed, after all). I do not agree that this is characteristic of the whole.

[/quote]
Finally! THis is why an independent 3rd party review the evidence was critical! The decision of guilt was based on an expert opinion. If YOU make a decision of guilt or innocence based on the evidence people can raise questions of bias because of your direct involvement in the area. This can not be made if the decision was reached by a 3rd party group.

Regarding the other post,
normally if a defendant does not defend themselves the verdict is then based on the evidence and process. This is why I am very interested in the evidence and process used to convict. The panel was clearly tainted by members with a strong bias of guilt. It was in fact formed with a bias. Then the evidence was presented in a biased or misleading manner, just one example was the modified code.
Just because someone has a wart and you dress them like a witch doesn't make them a witch.

[Judoka]

Another Post from Chessvibes
Another 3rd party chimes in....
Andrew Dalke

For computer programs, the relevant legal concept is the "abstraction filtration comparison" test, mentioned earlier. http://en.wikipedia.org/wiki/Abstractio ... rison_test . Lines of code are one obvious indicator of copyright infringement, but so are non-literal elements like function decomposition and module layout. But that's not sufficient. There are "scènes à faire" exceptions, and "de minimis" exceptions. For example, there's only a small number of reasonable ways to parse the UCI position command string, so a similarity may fall under scènes à faire.

I mention that case because that's the part of Mark Watkins's analysis which I decided to investigate on my own. I found the analysis flawed, and I came to this forum to report my observations and get feedback. I went into details elsewhere here - indeed, I now see that you replied to that comment so I need not repeat it. The disassembly shows that the implementations are different (they use different call parameters, different algorithm) so cannot come from a verbatim copy of the original Fruit code.

Furthermore, under a copyright infringement claim, the scènes à faire exception covers most of the similarities. While the moves[-1] = '\0' code is unusual, and is the key evidence of that admittedly minor example, it was not shown that that it is unusual in the context of Rybka and so does not convince me.

I am indeed nearly clueless on this topic, which is why I read through this PDF. I then spent many hours to understand this one part of code and claim; there's no way I can do a full analysis of the entire document.

However, I do know some about copyright law, and I know that major transformations like (quoting again from the Watkins report) "Rybka uses a look-up table of patterns, while Fruit does bit-scanning", "Rybka 1.0 Beta uses bitboards, making direct code comparison ineffective", "Fruit 2.1 visibly computes these while Rybka 1.0 Beta just has an array", "Fruit’s is linear, while Rybka’s is a bit more complicated", "Rybka also has a “lazy eval” which is not found in Fruit" means that the code falls towards the "idea" side of the idea-expression divide.

Another facet is that Fruit's source code is available, and appears to be widely studied. Under US law (Computer Associates v. Altai, quoting another case) "p]laintiffs may not claim copyright protection of an ... expression that is, if not standard, then commonplace in the computer software industry". A hard question, and I quote again from Watkins, is "While a large (indeed, almost complete) match is found, it is presumably feasible to opine that the Fruit source code can be taken as a “manual” for chess programming (perhaps in the sense of a modern version of How Computers Play Chess), and if this paradigmatic view is accepted, then the re-use of the same evaluation components might arguably be less derelict."

Yet another perhaps relevant US court case is USL v. BSDi, where the University of California Berkeley has a license to AT&T Unix. Quoting from Wikipedia: "Berkeley students and staff, ... audited the code, looking for freely available copies of the source code and methods. When they could find none, they said, they removed the code and rewrote it using publicly known techniques—and so any remaining similarities existed because AT&T had effectively abandoned the copyright to them." In other words, even though BSD started as a copy of Unix, over time little of the original source remained. That part was removed and rewritten, although with a clean room development model in the BSD case. By analogy, even if Rybka started as the Fruit code, it's still possible for it to no longer contain copyrightable items even if has some structural similarities.

Hence for a copyright claim - and I stress again that this has nothing to do with the ICGA uniqueness criteria - there are many factors which make it less than certain that a US court will find this to be a copyright violation.

In any case, my personal opinion is that large-scale verbatim copying would be much easier to spot, and not require teams of people trawling through disassemblies to find. The problem with data mining is that you must define your criteria beforehand, otherwise you are too liable to find false correlations. Cf http://xkcd.com/882/

[BB+]

I hope to be able to address Dalke's points soon. There is too much of import to give a brief answer. I am currently fighting a French keyboard; but hopefully that will not constrain me too much.

[wgarvin]

For any onlookers wondering if Judoka's posts above will remain unanswered...
He posted similar stuff in the other thread that drew some responses.

[BB+]

I should say at the beginning that I suspect Dalke is over-influenced by the more-preliminary RYBKA_FRUIT, and would do better to read the RECAP. In particular, it gives some more striking examples of Fruit code adapted into Rybka 1.0 Beta. I agree that the more-abstracted "evaluation features" can be hard to digest in a vacuum, but this is bolstered by the other evidence, including the probative elements.

However, I do know some about copyright law, and I know that major transformations like (quoting again from the Watkins report) "Rybka uses a look-up table of patterns, while Fruit does bit-scanning", "Rybka 1.0 Beta uses bitboards, making direct code comparison ineffective", "Fruit 2.1 visibly computes these while Rybka 1.0 Beta just has an array", "Fruit’s is linear, while Rybka’s is a bit more complicated", "Rybka also has a “lazy eval” which is not found in Fruit" means that the code falls towards the "idea" side of the idea-expression divide.

I won't address copyright. For the ICGA purposes, there was a specific Panel discussion regarding what was/wasn't "protectable", and it was determined that evaluation features (in their collative aspects) were protected. I used the example of a plot of a book elsewhere, but another (more computery) example would be comparing video games by listing all the "power-ups" one could obtain. For instance, games X and Y both have 30 of these, with 25 are quite similar, while other games have an overlap closer to 10 or 15. Unless there is a scenes a faires consideration (both X and Y are in a specific subgenre), there could well be infringement. The fact that the power-ups are implemented somewhat differently (either internally or externally) does imply that the "copying" would be nonliteral, but does not excuse the issue.

Another facet is that Fruit's source code is available, and appears to be widely studied. Under US law (Computer Associates v. Altai, quoting another case) "p]laintiffs may not claim copyright protection of an ... expression that is, if not standard, then commonplace in the computer software industry". A hard question, and I quote again from Watkins, is "While a large (indeed, almost complete) match is found, it is presumably feasible to opine that the Fruit source code can be taken as a “manual” for chess programming (perhaps in the sense of a modern version of How Computers Play Chess), and if this paradigmatic view is accepted, then the re-use of the same evaluation components might arguably be less derelict."

My sentence here seems to be a remnant of RYBKA_FRUIT. When I wrote this I had hoped someone would push the "manual" idea, but in the end no one did, and the Panel essentially rejected it.

In any case, my personal opinion is that large-scale verbatim copying would be much easier to spot, and not require teams of people trawling through disassemblies to find. The problem with data mining is that you must define your criteria beforehand, otherwise you are too liable to find false correlations.

When I spoke with Dan Bernstein about 6 months ago, he essentially said, "Well, duh, of course the cases that go to trial are almost always non-literal, as else it's so obvious that no one will contest it." The Panel tried to the extent possible to ensure that the correlations observed were real, such as comparing to other open-source chess programs, and programmers relying on their own knowledge of how similar one might expect programs to be.

[BB+]

I also am quite saddened that it seems VR is now relying on the fact that most of the matters are technical, and that he can bluff the masses about it.

Having read a little more, I was perhaps a bit too harsh. Indeed, VR still seems confused about the main issue: David Levy and a bunch of other programmers accused me of copying source code. This is what I assumed was the problem, and what I was prepared to defend myself again. I can find no example of such an accusation of "copying source code" anywhere (such as in the Open Letter). The more typical phrase that is used is "derived", though again perhaps VR has a different notion of what this means.

Later in the same post, VR says: Is source code copying a dead issue and algorithmic copying now the problem? Again he seems to have strict concepts of each (with no connecting layers of abstraction), and from what I can tell (from other comments), believes that the evaluation function "algorithm" has no creative content in its realisation (I will discuss this construal of "algorithm" in a later post).

[BB+]

Here is a Hacker News thread from about 2 weeks ago (cited elsewhere, but usually not at the top post):
http://news.ycombinator.com/item?id=3437028

Obviously you can guess which of Dalke and Snell looks correct to me. Some particular quirks of Dalke are: he imports his general programming knowledge (DRDOS copied MSDOS bugs, SCO/Linux disputes) into the discussion here, where I think they form at best an imperfect analogy; he has read the ChessBase article and RYBKA_FRUIT but seemingly nothing else (such as the RECAP); and that he is leisurely in his use of the word "algorithm": Instead, the complaint is that the underlying algorithms were re-implemented, instead of being original algorithms.

This last issue needs some attention, as many seem to get it wrong. An EvaluationFunction typically maps ChessPositions to Scores. There is no pre-defined method to do this. One can only speak of the evaluation function "algorithm" once a specific EvaluationFunction is chosen. Designing an evaluation function is itself something that involves various creative choices (there is some overlap with implementation and strength issues, but those are not of primary import here). The usual framework involves first deciding what aspects of the position to consider, and secondly how much to weight them. The conclusion of the EVAL_COMP analysis was that Rybka, from versions 1.0 to 2.3.2a at least, was derivative of Fruit in its choice of what to evaluate. Furthermore, the Panel agreed that this was sufficient to breach ICGA Rule #2 (the notion of Fruit being a "manual" was thus rejected).

Contrariwise, the examples adduced by Dalke are much more "functional" in their aspects (involving operating systems), where there is little creative component in deciding what is to be done (note that "look-and-feel" of a computer program often falls under a different heading). As most OSes have greatly similar tasks to do, it is not surprising that they have many common points. Dalke notes this task-similarity about the UCI parsing, and seems then to extrapolate that a chess engine in its totality is quite formulaic in its construction. Again I don't think that chess programmers would much agree with this for search and specifically evaluation.

I do find it a bit odd that Dalke concludes: it is not and cannot be plagiarism any more than how West Side Story plagiarizes Romeo and Juliet -- when, disregarding the 400-year time issue, this is exactly an example given by Nimmer of copyright infringement(!) at the level of plots. (Others argue that "plots" are no more than abstract ideas, and case law is, well, diffcult).

[BB+]

Indeed, VR still seems confused about the main issue: David Levy and a bunch of other programmers accused me of copying source code. This is what I assumed was the problem, and what I was prepared to defend myself again. I can find no example of such an accusation of "copying source code" anywhere (such as in the Open Letter). The more typical phrase that is used is "derived", though again perhaps VR has a different notion of what this means.

I looked at Fabien's original post, and the ICGA Panel Report, and neither of them talk of "copying source code" either. Both seem to be rather clear what is being alleged.

The short answer was "no", it [Strelka] was not a verbatim copy of the source code. All the code had been typed (can't say "designed" though, see below) by an individual. So legally there was no issue that I knew of. It was however a whole re-write (copy with different words if you like, similar to a translation) of the algorithms. Not just an extraction of a couple of ideas as is common, and normal.

The word "derived" (and inflections) is used almost throughout in the Report.

Note the rules require programmers to list all authors and the source of code, even “derived” code.

ZW summation: Nearly the entire evaluation function is derived from Fruit.

DD: The Rybka code base was without doubt derived directly from other peoples work and this was never revealed

WG: My opinion is that Rybka 1.0 beta (Dec 5 2005) has an evaluation that is clearly derived from Fruit 2.1, and all Rybka versions through 2.3 (Feb 2007) continued to have an almost identical structure in their eval.

MU: It is clear to me that significant parts of Rybka's code were derived and copied firstly from Crafty in early Rybka versions and later from Fruit in later versions of Rybka

The same features are measured and evaluated and clearly indicate these versions of Rybka are derivatives.

In contrast, the Crafty section explicitly speaks of "code" (and "copying"), meant in a stricter sense, and some statements even make clear the difference in the nature of the evidence in the Crafty/Fruit incidents:

Pre-Rybka 1.6.1 contains much identical code to Crafty, even including large blocks of code with obsolete code inside them, and code that performs tests that make no sense today (code that was left in Crafty by accident, by Robert Hyatt, also shows up in Rybka 1.6.1). It is inconceivable that a second author could duplicate this code purely by chance. At least hundreds of lines of code appear to be copied.

Early pre-Rybka 1.0 beta versions used a great deal of Crafty code and functions,

WG: I am also convinced that some versions older than Rybka 1.0 beta contain direct copies of code from Crafty, and that at least some versions of Rybka newer than 2.3 still contain significant amounts of code that is derived from Fruit 2.1.

My conclusion is that (to be generous) VR did not spend enough time to sort out what was being alleged against him. I should think that this falls under his responsibility.