FIDE Rules on ICGA - Rybka controversy

General discussion about computer chess...
Post Reply
BB+
Posts: 1484
Joined: Thu Jun 10, 2010 4:26 am

Re: FIDE Rules on ICGA - Rybka controversy

Post by BB+ » Wed Apr 29, 2015 5:55 pm

syzygy wrote:I do see why a procedure in 2011 on whether to revise tournament results from 2006, if there must be one at all, should be "fair".
I think we can agree that for any "fair" process to be implementable, Rajlich must be "cooperative" to some sense. One principal leverage the ICGA had in this regard was PR, but as others have pointed out, Rajlich had no reason to cooperate with a "lynch" process (I use the term only by quotation). In the Rule #2 context, I'm not sure (even after Reul) the ICGA really considered that an accused person would be unresponsive toward the investigation (which then induces the need for a more robust framework for dispute resolution).
Rebel wrote:You may find that, I could say the same for your sense for fair procedures to be vexatious as well.
Vexatious has a proper meaning here, namely causing or tending to cause annoyance, frustration, or worry (in the legal setting this would typically be by abuse of process), or something brought up purely to cause annoyance. Asking the second time for an appeal (after the first rather clear response) might be warranted. A third is pushing it. For Rajlich (through his representative) to go beyond that is getting ridiculous, especially with the implied threats in the later emails.
Rebel wrote:An appeal is a fundamental right.
No, in this context it is not. If this was a criminal case, you could be correct. As the EC wrote in the "French team" case, regarding the attempt to invoke 6.3 of the ECHR: this is not a criminal case; this is a case in front of the competent organ of an international sports federation concerning the violation of its rules and of the sports law.
Rebel wrote:I see human being horrified, who just became aware his program is cloned making an emotional public outburst, what's new? Then Hristo comes along and puts some sense back into him.
In contrast to your psychological evaluation, seven months later, he was still saying (as quoted in the Panel report regarding Rule #2 interpretation of "code"): Strelka contains Rybka code. "Isn't that all what matters?"
Rebel wrote:Osipov... the guy who wanted to commercialize Strelka and was pissed with Vas because didn't allow him?
And you believed him?
Turns out he was right. :!: [And if you've gone over my posts from the Rybka forum at the time, you will find that I did not take Osipov too seriously (either in 2007 or after VR's Jan 2008 statement), it was only when ZW was more vocal in Aug 2008 that I became more suspicious -- for instance, see here my mocking of Osipov's "Fruitification" with move_order()].

I guess I could re-write history as: Rajlich ... the guy who wanted to go commercial with "Rybka" already in 2003, but by June 2005 (having followed Fruit development closely for 15 months, as seen by CCC posts) was pissed because his engine still sucked, and didn't even bother to see if Letouzey would allow him to commercialise Fruit (under the Rybka name)? :roll: "And you believed him?"
BB+ wrote:How long should it take to construct a defense to the main claims (of evaluation similarities)
Rebel wrote:It took me (while scrutinizing your and Zach's document) 4-6 weeks to see something fundamental was wrong summarized in one word: tunnel-vision.
In other words, you have no real debate about the claims of evaluation similarities, and took 4-6 weeks to come to a psychoanalysis of the investigators?
Rebel wrote:It took me about 3 months to conclude there is barely evidence Rybka started its life as Fruit.
Rajlich would have the advantage of not having to conduct an outside investigation as you did. :idea: I've also never been sure how you came to this "conclusion" (or what "Rybka started its life as Fruit" would mean); there seem to be many things still unaddressed (or one could say "conveniently ignored"). For instance, to give but one example (as close to "code copying" as possible), you have never responded to the last example in 6.3.2 in RYBKA_FRUIT where (as I say, just a simple example, no need to worry about its chess-playing effects) it is noted that Rybka copies a redundant use of "infinite || ponder" (not to mention that these [and other] variables are allocated in the same order as Fruit, or why the factoring of UCI parsing vis-a-vis time management in Rybka 1.0 follows the idiosyncrasy of Fruit, or why the iterative deepening code [Appendix A of RYBKA_FRUIT] differences are almost nil save numerological, etc., not to mention stuff that was found after RYBKA_FRUIT was written).

But it should be stressed that the ICGA debate ended up being about the (higher-level) evaluation [feature] similarities, and whether these sufficed to break Rule #2, and whereas the above (partial) enumeration of low-level quirks might have required a point-by-point rebuttal, I don't think this is particularly true in the case at hand.
Rebel wrote:In such a complex scenario it's only reasonable the accused gets a proper time to put up a defence and refute the 2 documents point by point. Which was denied to him.
I am not sure which "2 documents" you mean here. If you mean ZW's and RYBKA_FRUIT, the first was made known to him already at the beginning of February, and the second was integrated into the open letter at the beginning of March. It is reasonable to think that he might already begin to familiarise himself with these at that point (though I guess his schedule was already "very full" by then). The principal "new" material (besides the Report itself) from the investigation was EVAL_COMP.

But I also don't think the situation is that complex. Previously I noted that Rajlich had three main branches of argument to make (please add others if you can):
  • (1) the contention that Rybka and Fruit have similar evaluations is erroneous (possibly because the studied Rybka versions are irrelevant to this discussion)
  • (2) they are similar, but it is not out of the ordinary
  • (3) they have extraordinary similarities but this is not prohibited by the rules
Although the "versioning" issue in (1) was debated ex post facto for a significant time at Rybka Forum (well into 2013), already in the ChessBase interview Levy notes that: During the investigation process one member of the ICGA investigation secretariat asked Rajlich "1. Are you able to provide us with the exact .exes that played in all ICGA events particularly those from 2006 and 2007? If not, is it a fair approximation to use Rybka 2.3.2a for the 2007 event?" On the Rybka Forum, Rajlich said "Yes" to the second question, confirming that this version was used in an ICGA event.

With (2), his main evidence would seem to be either the ponderhits, or the later analysis of Riis (as critiqued by Watkins). Note that Levy had already indicated in correspondence to Rajlich that code analysis should have priority over move-matching, but even with the latter the subsequent evidence seems to be rather against VR's claims, as codified in later work of Dailey, Hair, et. al. (and published as Move similarity analysis in chess programs here). Of course, Rajlich could go through EVAL_COMP point-by-point to dispute the "extraordinary" nature of the Fruit/Rybka similarities, but I think it would be easier just to take one or two examples from that text and highlight the difficulties and problems. In the Riis piece, Rajlich doesn't really dispute that the evals are quite similar, but rather lists: material imbalances, passed pawns, and tuning (also "a lot of little things") as being the main Rybka innovations, and also mentions decorrelation of terms. [I do love the pomposity of Rybka was the first engine to understand that major pieces are [relatively] more valuable in endgames -- err, one might argue that KAISSA could be said to already have "understood" this!].

So the dispute would be about (3), namely whether Rule #2 should include high-level copying (and to what extent), and it seems that the ICGA and Rajlich here have significantly different opinions. The result of the "poll" of Panel members (most of whom are ICGA programmers, unlike other lists of programmers that have been enumerated as holding an opposing view) indicated the general opinion that indeed Rule #2 should cover Rajlich's dependence on Fruit's evaluation features (and as FL has pointed out, it's just as much about what Fruit doesn't include as what it does).

User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: FIDE Rules on ICGA - Rybka controversy

Post by Rebel » Wed Apr 29, 2015 9:29 pm

I will answer in parts.

BB+ wrote:
Rebel wrote:You may find that, I could say the same for your sense for fair procedures to be vexatious as well.
Vexatious has a proper meaning here, namely causing or tending to cause annoyance, frustration, or worry (in the legal setting this would typically be by abuse of process), or something brought up purely to cause annoyance. Asking the second time for an appeal (after the first rather clear response) might be warranted. A third is pushing it. For Rajlich (through his representative) to go beyond that is getting ridiculous, especially with the implied threats in the later emails.
Rebel wrote:An appeal is a fundamental right.
No, in this context it is not. If this was a criminal case, you could be correct. As the EC wrote in the "French team" case, regarding the attempt to invoke 6.3 of the ECHR: this is not a criminal case; this is a case in front of the competent organ of an international sports federation concerning the violation of its rules and of the sports law.
As David (see the correspondence) compared the case against Vas with a court case the right to appeal is fundamental. Not sure the laws where you live but here an appeal is granted whether a criminal or civil case.
Rebel wrote:I see human being horrified, who just became aware his program is cloned making an emotional public outburst, what's new? Then Hristo comes along and puts some sense back into him.
In contrast to your psychological evaluation, seven months later, he was still saying (as quoted in the Panel report regarding Rule #2 interpretation of "code"): Strelka contains Rybka code. "Isn't that all what matters?"
Rebel wrote:Osipov... the guy who wanted to commercialize Strelka and was pissed with Vas because didn't allow him?
And you believed him?
Turns out he was right. :!: [And if you've gone over my posts from the Rybka forum at the time, you will find that I did not take Osipov too seriously (either in 2007 or after VR's Jan 2008 statement), it was only when ZW was more vocal in Aug 2008 that I became more suspicious -- for instance, see here my mocking of Osipov's "Fruitification" with move_order()].
It seems you are the author of the myth yourself driven by tunnel-vision.

Let's look at the origins of Strelka, for example in the fundamental MAIN.C

http://www.top-5000.nl/rf_main.htm

1. How does the Strelka code look like, Fruit or Rybka 1.0 ?

2. And related, how do Fruit and Rybka compare? Copied?

Also one of those things you and Zach missed.

3. Then I see in EVAL.C all those too well familiar constants also used in Rybka 1.0 for example (256|64|172|971) as open / semi open rook files evaluation.

I tend to believe the code, not Osipov.

syzygy
Posts: 148
Joined: Sun Oct 16, 2011 4:21 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by syzygy » Thu Apr 30, 2015 12:26 am

BB+ wrote:
syzygy wrote:Btw, when did Rajlich state that he no longer had the source code of a particular version? (If you happen to know this.)
BB+ wrote:Off the top of my head, in his email correspondence to Schüle in June 2010 regarding Rybka 3. Maybe there were prior mentions at RybkaForum in 2008 (regarding Strelka and Rybka 1).
After more thought, the first time I could think of would be in (say) Aug-Sep 2008, when various bugs with R3 were beginning to be reported. VR made some comments along the lines that there would be no proper "patch" release, and that he couldn't fix any specific bugs (such as the crash NH found) anyway since the Rybka source had changed too much since the release. [The never-seen R3+ was talked about for the next year, but was always just going to fix problems under the current code base -- indeed, he was talking about toning the ELO down to the R3 level so as not to disadvantage the gap that future R4 buyers would obtain]. [There were plenty of defenders of this "only keep the latest" model of development (especially with a single person), though others inquired whether a source-control system might be warranted].
OK, so it seems likely that he really did no longer have the source code of old Rybka versions. (I guess one can still speculate that he must have had some backups somewhere, possibly of intermediate versions that were never released, but maybe he didn't even have that.)

User avatar
Harvey Williamson
Posts: 248
Joined: Thu Jun 10, 2010 2:10 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by Harvey Williamson » Thu Apr 30, 2015 12:56 am

syzygy wrote: OK, so it seems likely that he really did no longer have the source code of old Rybka versions. (I guess one can still speculate that he must have had some backups somewhere, possibly of intermediate versions that were never released, but maybe he didn't even have that.)
Lukas did post this which suggests source does exist http://rybkaforum.net/cgi-bin/rybkaforu ... ;hl=source
Vas and I discussed whether or not he should give source code to the ICGA.

syzygy
Posts: 148
Joined: Sun Oct 16, 2011 4:21 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by syzygy » Thu Apr 30, 2015 1:02 am

BB+ wrote:
Maybe it would have been a good idea to involve Rajlich in the discussion on how to shape the investigation procedure.
We just need someone to chime in that control of the process is the control of the result... Or to take the antagonistic point of view, why should Rajlich lend legitimacy to the ICGA processes by consulting with them even about the protocol, when "it was clear from the beginning" that whole thing was a sham. ;)
The way it went, one could indeed say it was clear from the beginning (ChessVibes etc.). And on the other hand you seem to suggest that Vas "was completely unwilling from the beginning". I might be wrong, but I don't think Vas was unwilling towards Levy.

I didn't follow computer chess in those years, but I have the impression that "tensions" that developed from 2006 to 2010 have much to do with what followed after. Levy probably didn't, or at least not openly, pick sides until 2011, but I can imagine that Rajlich did not have a good feeling at all when he learned that some people that, before, he could simply ignore, were now part of a decision-making process aimed at him.

Anyway, what I meant is that, at least in hindsight, things might have gone better if the investigation had not started with one side drawing up the rules (and filling in most of the names). And, again with hindsight, it might have been better not to frame the process as "unmasking a cheat" but as "clarifying what Rule 2 really means and whether all entrants comply with it", non-compliance not necessarily having to lead to voiding of past results (and in the end, it must have been clear that at that time Rybka was much more than just an optimised Fruit). It might be that Vas has really been denying things that are objectively difficult to deny, but I don't know if he would have contested, for example, that the similarity between Rybka's and Fruit's evaluation cannot be explained as a coincidence.

User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: FIDE Rules on ICGA - Rybka controversy

Post by Rebel » Thu Apr 30, 2015 7:32 am

Harvey Williamson wrote:
syzygy wrote: OK, so it seems likely that he really did no longer have the source code of old Rybka versions. (I guess one can still speculate that he must have had some backups somewhere, possibly of intermediate versions that were never released, but maybe he didn't even have that.)
Lukas did post this which suggests source does exist http://rybkaforum.net/cgi-bin/rybkaforu ... ;hl=source
Vas and I discussed whether or not he should give source code to the ICGA.
That (indeed) was a weird posting by Lukas. But it would be nice if you would have told the full story of the lost source and not only half. In December 2012 source code was recovered of an in between R1 - R2 version which was suggested as a base for an indepedent unbiased expert to close the matter once and for all. Levy did chicken out. Like Levy did chicken out to grant Rajlich an appeal via the same circus (Panel | Secratary | Board).

User avatar
Harvey Williamson
Posts: 248
Joined: Thu Jun 10, 2010 2:10 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by Harvey Williamson » Thu Apr 30, 2015 7:49 am

Rebel wrote: it would be nice if you would have told the full story of the lost source and not only half. In December 2012 source code was recovered of an in between R1 - R2 version which was suggested as a base for an indepedent unbiased expert to close the matter once and for all. Levy did chicken out. Like Levy did chicken out to grant Rajlich an appeal via the same circus (Panel | Secratary | Board).
Lukas's post is from 2011 so your story is not relevant. Or did he lose it and find it again?

User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: FIDE Rules on ICGA - Rybka controversy

Post by Rebel » Thu Apr 30, 2015 9:29 am

BB+ wrote:
Rebel wrote:It took me about 3 months to conclude there is barely evidence Rybka started its life as Fruit.
Rajlich would have the advantage of not having to conduct an outside investigation as you did. :idea: I've also never been sure how you came to this "conclusion" (or what "Rybka started its life as Fruit" would mean); there seem to be many things still unaddressed (or one could say "conveniently ignored"). For instance, to give but one example (as close to "code copying" as possible), you have never responded to the last example in 6.3.2 in RYBKA_FRUIT where (as I say, just a simple example, no need to worry about its chess-playing effects) it is noted that Rybka copies a redundant use of "infinite || ponder" (not to mention that these [and other] variables are allocated in the same order as Fruit, or why the factoring of UCI parsing vis-a-vis time management in Rybka 1.0 follows the idiosyncrasy of Fruit, or why the iterative deepening code [Appendix A of RYBKA_FRUIT] differences are almost nil save numerological, etc., not to mention stuff that was found after RYBKA_FRUIT was written).
First of all I was making a logical case why Rajlich should have been granted a reasonable time period to put up a defence. It took you and Zach 1½-2 years to compile your documents?

We can change subject if you wish so and talk about the UCI parser. I assume that's what you meant. The link you quoted is Zach's document and does not have a 6.3.2 chapter, but yours has.

But it should be stressed that the ICGA debate ended up being about the (higher-level) evaluation [feature] similarities, and whether these sufficed to break Rule #2, and whereas the above (partial) enumeration of low-level quirks might have required a point-by-point rebuttal, I don't think this is particularly true in the case at hand.
Well, it defenitely takes time to notice that every evaluation component is coded differently than Fruit.


Rebel wrote:In such a complex scenario it's only reasonable the accused gets a proper time to put up a defence and refute the 2 documents point by point. Which was denied to him.
I am not sure which "2 documents" you mean here. If you mean ZW's and RYBKA_FRUIT, the first was made known to him already at the beginning of February, and the second was integrated into the open letter at the beginning of March.
Yep.

And in the meantime David already released his frontal attack on Rajlich with the Chessvibes article "Attack of the Clones" on February 19, 2011 and 4 days later on February 23, 2011 the investigation started without Rajlich. Such a sudden hurry after 6 years.

It is reasonable to think that he might already begin to familiarise himself with these at that point (though I guess his schedule was already "very full" by then). The principal "new" material (besides the Report itself) from the investigation was EVAL_COMP.
See my above comment.

Every evaluation component is coded differently than Fruit.

But instead of rehashing issues we have already gone through let's see if there is a model we can agree on.

1. There was no code copying. Like Fabien's statement - "Can I claim that I have written it all on my own? "Yes", I typed all the code myself. Without help??? Certainly not, hence my point: "it makes no sense".

2. Based on his on statement of December 2005, the same month of the Rybka 1 release (I went forwards and backwards Fruit and took many things) and giving Fabien a huge credit in the release note I can see 2 things -

2a. That he learned how to write a decent chess engine, worth at least 2000+ elo on the hardware available of these days. I think that is (still) true for every average quality aspirant chess programmer.

2b. That he perhaps (emphasis added) used the Fruit mail-box eval as a model for his bit-board based engine from scratch according his own interpretation as an IM. It would explain the differences, code wise and evaluation wise. It's a wild guess but maybe it's not far from the truth.

User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: FIDE Rules on ICGA - Rybka controversy

Post by Rebel » Thu Apr 30, 2015 9:35 am

Harvey Williamson wrote:
Rebel wrote: it would be nice if you would have told the full story of the lost source and not only half. In December 2012 source code was recovered of an in between R1 - R2 version which was suggested as a base for an indepedent unbiased expert to close the matter once and for all. Levy did chicken out. Like Levy did chicken out to grant Rajlich an appeal via the same circus (Panel | Secratary | Board).
Lukas's post is from 2011 so your story is not relevant.
Since when is Lukas relevant to the investigation?

What is relevant is what I told you above.

There is Rybka source code.

User avatar
Harvey Williamson
Posts: 248
Joined: Thu Jun 10, 2010 2:10 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by Harvey Williamson » Thu Apr 30, 2015 5:08 pm

Rebel wrote: Since when is Lukas relevant to the investigation?

What is relevant is what I told you above.

There is Rybka source code.
Lukas is much more relevant than you as he is probably the only person to have been in regular contact with him throughout Rybka's existence as a commercial product and he is also a business partner with Vas.

Post Reply