FIDE Rules on ICGA - Rybka controversy

[hyatt]

I'm not going to go through every eval term, but for 1.0 beta, the above is certainly NOT true. One example is isolated pawns, and by factoring out the bitboard/mailbox differences, you are left with identical implementations. A simple one, agreed, but you said EVERY one is coded differently, and showing one exception is enough to break that statement.

Nope. You have forgotten Marcel's chess lesson. [ http://rybkaforum.net/cgi-bin/rybkaforu ... #pid396062 ] Something for Mark as well since it's about his COMP-EVAL document.

The given link demonstrates that backward pawns (arguably the most complicated and quirky pawn factor) are indeed different in definition (and other ways). Which marcelk notes was already apparent in EVAL_COMP. The post says nothing about the "coding" (at any abstraction level) of any other eval features. [And as hyatt points out, the implementation of backward pawns a la Kmoch can be encumbered when using bitboards as opposed to mailbox].

I disagree with your logic. You can't use Rybka's main defence, a different data structure (bit-board), against Rybka to explain another difference. The code does (evaluates) different things. Thus Rybka <> Fruit.

Let's sort this out first before going further. It's quite fundamental and a repeating theme.

Your comment is based on ignorance. You have NEVER done that previously. My comments based on experience. In October of 1994 my chess engine (Cray Blitz) was a mailbox program. By December 1994 my NEW chess engine (Crafty) was bit board based. I KNOW what has to be done to make that conversion. I KNOW what kinds of compromises one might make, particularly in the early versions. I KNOW what kinds of things are more straightforward in mailbox or bit board approaches. So my comments are not guesswork, they are based on real experience in doing exactly what we are talking about, which was to take an existing mailbox program and convert it to bit boards. If you were to compare Crafty (today) vs Crafty (earliest version one can find) vs Cray Blitz (only source available is 1989) one would see a common overall design. From the move selection and phase stuff (phase 1 => hash move, phase 2 -> captures, phase 3 -> killers, etc) to things like Swap(SEE) to Make/Unmake, there is a very visible design component even if the code looks completely different due to the board representation issues.

So saying A is bit board and B is mailbox so they are completely different is WAY off the real mark. They could actually be identical and search exactly the same tree, same number of nodes, same PV, same score, and yet they would look different at the level you are focusing on, because the board representations are night and day different. But that does not make both original. So focusing on bit board vs mailbox is broken at the beginning. And it doesn't get any better later on.

[hyatt]

That was not a chess lesson. It was "RE nonsense". If you read what he wrote, he SPECIFICALLY discussed the difference between a bit board implementation and a mailbox implementation. Something we said we EXPLICITLY abstracted away. A loop vs an AND mask? Bit board vs mailbox, nothing more, nothing less.

So please don't call that a "lesson" to someone who actually teaches programming. That was more of a "chess obfuscation" than anything else. Zach did it exactly right there. I don't think much of the "seems to be, or appears to be" nonsense. It is or it isn't.

Your question two posts back in that thread is also nonsensically phrased. You snipped just the 4 bit-wise ANDs (tst instructions) and left out the rest. You DID realize that the masks and stuff were loaded at the top of the code? You seem almost surprised that some rather convoluted mailbox code collapses into 4 AND operations in a bit board program. This is quite common. I've used the term "bit-parallel" because bit boards let me ask a question with one AND that would take a loop with multiple if-statements inside in a mailbox program. That's part of the attractiveness of bit boards for those of us that decided to invest the time to get comfortable with them.

And you are missing it again, you only look at the code and not what the code does. If you had read Marcel carefully you would had seen Fruit and Rybka are evaluating different, Fruit has the exception code for second ranked pawns that can advance 2 squares. Rybka has not, just a quick and dirty evaluation. Fabien uses the Kmoch definition, Vas not. Does the name Hans Kmoch and his basic stuff about pawns every chess player needs to know mean anything anythng to you? There is even a whole page on the CPW about him.

You see, there is this pattern, you ASM geeks (of which I belong too) only look at the code and shout EUREKA!, this piece of Rybka code smells to similarity with Fruit and then you forget to look what the code actually evaluates, as in this case.

So my statement still stands:

COMP EVAL fails to mention that EVERY eval ingrediënt in Fruit and Rybka is coded DIFFERENTLY even after substraction of the Mailboard / Bitboard differences.

Next one?

You miss MY point. Do you know why Rybka doesn't do that? Not so convenient in bitmaps. A pretty natural part of conversion from one to the other. I had several such things that were in Cray Blitz but which I left out of Crafty because (a) it was not clear if they were really beneficial and (b) it was more complicated (affecting speed) to do it the same way. Taking both of those into consideration...

BTW we didn't say "smells to similarity." We said "they do the same thing" or in a few cases "almost exactly the same thing but there is one difference."

The rook scoring (open/half-open files) is a good example. I'd bet that was an outright error in translation, seeing as how it is actually a -25 Elo change.

If you want to hang on to that "every" simply because the code doesn't match, and it obviously could NOT match when using two different board representations" then that's up to you. I know better.

Bet taken. $25,000 that you can't prove beyond any reasonable doubt that the Rybka rook-pawn code we were discussing was a "outright error in translation" within the next two months. I'll send you my lawyers address, we can both deposit $25,000 each. The $50,000 goes to me when you lose, else vice versa.

I can hear the cluck-cluck noises already.

I thought about this overnight. I'll take that bet slightly reworded. YOU prove that he did NOT design the code as I speculated. I can hear the cluck-cluck noises already. Because now the onus would be on YOU. And it is an absolute certainty that you can't prove someone didn't so something. And NO, his word would NOT be proof.

[hyatt]

I'm not going to go through every eval term, but for 1.0 beta, the above is certainly NOT true. One example is isolated pawns, and by factoring out the bitboard/mailbox differences, you are left with identical implementations. A simple one, agreed, but you said EVERY one is coded differently, and showing one exception is enough to break that statement.

Nope. You have forgotten Marcel's chess lesson. [ http://rybkaforum.net/cgi-bin/rybkaforu ... #pid396062 ] Something for Mark as well since it's about his COMP-EVAL document.

The given link demonstrates that backward pawns (arguably the most complicated and quirky pawn factor) are indeed different in definition (and other ways). Which marcelk notes was already apparent in EVAL_COMP. The post says nothing about the "coding" (at any abstraction level) of any other eval features. [And as hyatt points out, the implementation of backward pawns a la Kmoch can be encumbered when using bitboards as opposed to mailbox].

I disagree with your logic. You can't use Rybka's main defence, a different data structure (bit-board), against Rybka to explain another difference. The code does (evaluates) different things. Thus Rybka <> Fruit.

Let's sort this out first before going further. It's quite fundamental and a repeating theme.

A single difference does NOT make the programs different.

I am glad you understand my point.

And as an addition, the Kmoch definition of a backward pawn as practiced by Fabien in Fruit (IMO) isn't found in many programs, I am not aware of one who has it in use. Therefore if it were in Rybka it would have been a huge red flag. But it isn't there and shouldn't have been counted as evidence with a 0.7 similarity to Fruit (on a scale of 1.0) in Mark's document. Considering the absence of an unique Fruit eval feature IN THE LIGHT OF THE ACCUSATION Rybka=Fruit the similarity with Rybka is exactly 0.0.

A second case would be the evaluation of double-pawn rewarded by Mark with an 1.0 similarity, so the suggestion for the reader is that Rybka=Fruit. Evaluation of a double pawn is one of the most simple ingredients of EVAL. Most programs simply penalizes double pawns with a fixed value and that's it. So does Fruit. The better EVAL's (Fruit included) make a distinction with the material on the board and penalize a double pawn more in the endgame. But I haven't seem a program that gives NO PENALTY AT ALL for any double pawn in the middle game and only in the endgame. This is so fundamentally different IN THE LIGHT OF THE ACCUSATION Rybka=Fruit the similarity with Rybka is exactly 0.0.

Code is also different -

Code: Select all

         FRUIT                                 RYBKA
         if (doubled) {                        if (doubled) endgame -= 158
            opening[me] -= DoubledOpening;
            endgame[me] -= DoubledEndgame;

There is NO midgame code in Rybka.

A third case...................

So let me get this clear. You REALLY believe that because A and B evaluate the same thing, but A gives both a MG and EG penalty while B only gives an EG penalty, then they are "fundamentally different"? Even if tuning had shown that the MG penalty for doubled pawns was useless and rather than setting it to zero the line of code was completely removed?

Here's a question? Fruit/Rybka BOTH evaluate weak pawns incorrectly. BOTH like to give penalties when a pawn is weak and on an open file (isolated, backward..) EVEN if there is no rook or queen on the board to use that file to attack the pawn. I recognize such weak pawns and make a note, and then in the appropriate scoring module (rook or queen) I give that penalty when I notice there is a piece that can take advantage of that weakness. How would you compare Crafty to those two in that regard. I would not use the term "fundamentally different". I'd say "they check for exactly the same things, but one gives the penalty no matter what, but the other only gives it if there is a piece that can use the file. I'd say they are probably 80% matching, you would probably say 0%. You seem to think in terms of "binary". On/Off. 0%/100%. Etc. A bunch of 80% matches would be much more serious than a couple of 100%'s and a bunch if 0%'s.

[Rebel]

I'm not going to go through every eval term, but for 1.0 beta, the above is certainly NOT true. One example is isolated pawns, and by factoring out the bitboard/mailbox differences, you are left with identical implementations. A simple one, agreed, but you said EVERY one is coded differently, and showing one exception is enough to break that statement.

Nope. You have forgotten Marcel's chess lesson. [ http://rybkaforum.net/cgi-bin/rybkaforu ... #pid396062 ] Something for Mark as well since it's about his COMP-EVAL document.

The given link demonstrates that backward pawns (arguably the most complicated and quirky pawn factor) are indeed different in definition (and other ways). Which marcelk notes was already apparent in EVAL_COMP. The post says nothing about the "coding" (at any abstraction level) of any other eval features. [And as hyatt points out, the implementation of backward pawns a la Kmoch can be encumbered when using bitboards as opposed to mailbox].

I disagree with your logic. You can't use Rybka's main defence, a different data structure (bit-board), against Rybka to explain another difference. The code does (evaluates) different things. Thus Rybka <> Fruit.

Let's sort this out first before going further. It's quite fundamental and a repeating theme.

Your comment is based on ignorance.

Thanks for the good laugh.

Snipped your derail. Because that's what it is. It's what you always do when the opposition has a point you don't have an answer.

Third time - Both Zach and Mark notice in their documents Rybka <> Fruit, go argue with them.

Good moment to put you on ignore unless you have something substantial to mention. I came here for Mark in the first place. If I would wish to spar with you to test and debug my opinions on the R/F issue I can do that 24/7/365 at Rybka forum and I have done that enough in the past. For which I am still grateful BTW. You turned down a good looking case at first glance on your own overthere that even non-programmers saw through your smokescreens. You are the worst enemy of the ICGA verdict and you don't even realize the damage you have caused.

Same with your comments on the double pawn evaluation. THERE IS NO MIDDLE-GAME CODE in Rybka. Yet it is rewarded with a 100% similarity. FALSE. Do you think people are stupid? Yeah, it's what you do on a ferquent base, underestimating the intelligence of your audience.

[hyatt]

There was no derail. YOU have not taken a mailbox program and rewritten it as bit board. _I_ have. The "based on ignorance" was not an attempt at calling you "ignorant" in the general sense. But "ignorant" in this SPECIFIC circumstance. I've done it. I know the kinds of things that come up. I know the kinds of mistakes that are likely to be made. And I also realize that once you do that conversion, and continue to work on the NEW version, you will change things that will now not match the original "pattern" program. Perfectly normal. Differences don't mean a thing. What matters is "what is similar?"

I assume you don't want to talk about THAT topic because you have no experience. But that doesn't mean you get to ignore all the important stuff that others DO know, and try to use that inexperience as a way to hand-wave evidence under the rug.

Mark and Zach do NOT notice "Rybka != Fruit". They might point out where a term has a slight difference here and there, but that does NOT mean Rybka did not come from Fruit, not in the least.

For your "there is no middle game code in Rybka" what you REALLY mean is "there is no middle game score for doubled pawns." Those two statements are NOT the same. BOTH have code that does the same thing. One chooses to give both MG AND EG penalties, the other only chooses to give an EG penalty. It is a small term. I would call that 75% similarity myself. Same test for ALL positions, then one avoids one of the penalties only.

If you don't want ME to respond, just add "Bob, don't reply to this". Doesn't bother me at all and does make life a bit simpler...

[BB+]

I'd bet that was an outright error in translation, seeing as how it is actually a -25 Elo change.

Bet taken. $25,000 that you can't prove beyond any reasonable doubt that the Rybka rook-pawn code we were discussing was a "outright error in translation" within the next two months. I'll send you my lawyers address, we can both deposit $25,000 each. The $50,000 goes to me when you lose, else vice versa.

How is "beyond any reasonable doubt" the right standard? Who gets to decide?

I'll take that bet slightly reworded. YOU prove that he did NOT design the code as I speculated. ... Because now the onus would be on YOU. And it is an absolute certainty that you can't prove someone didn't so something. And NO, his word would NOT be proof.

I was actually going to comment on this issue (as Mr. Schröder had desired), but I see it has been derailed. Anyway, I still have not heard from a GM (evidently more useful than a mere IM, otherwise I might even have asked Levy ) regarding the chessic notion.

But I haven't seem a program that gives NO PENALTY AT ALL for any double pawn in the middle game and only in the endgame.

There is NO midgame code in Rybka.

Midgame in Rybka is an interpolation of opening (24 phase units) and endgame (0 phase units). Rybka gives a (nonzero) penalty for a doubled pawn once any piece has been captured (on a pure phase basis, 24/25th of time, or 96% which would round up ). Furthermore, EVAL_COMP is primarily about features, not about numerical tuning. In this sense, Fruit and Rybka (and Faile and ExChess) use the "same" doubled pawn feature, while some other engines listed are of a more varying nature.

Considering the absence of an unique Fruit eval feature

All the engines involved have a "unique" backward/weak pawns feature. It is something for which there is a lot of scope for variations in implementation. Fruit/Rybka was higher than the "typical" amount of overlap.

the similarity with Rybka is exactly 0.0

If you wish to calibrate the scoring in that way, you are free to do so. You then end up with only (nearly) exact matches at 1.0, and Fruit/Rybka has (significantly) more of these than any other pair. I personally think adding in partial scores is a superior method. [For that matter, recall that most of the Panel did not particularly find EVAL_COMP's quantification of the situation to be necessary, and it was largely produced in response to comments by van Kervinck and Roberson].

[Rebel]

But I haven't seem a program that gives NO PENALTY AT ALL for any double pawn in the middle game and only in the endgame.

There is NO midgame code in Rybka.

Midgame in Rybka is an interpolation of opening (24 phase units) and endgame (0 phase units). Rybka gives a (nonzero) penalty for a doubled pawn once any piece has been captured (on a pure phase basis, 24/25th of time, or 96% which would round up ). Furthermore, EVAL_COMP is primarily about features, not about numerical tuning. In this sense, Fruit and Rybka (and Faile and ExChess) use the "same" doubled pawn feature, while some other engines listed are of a more varying nature.

the similarity with Rybka is exactly 0.0

If you wish to calibrate the scoring in that way, you are free to do so. You then end up with only (nearly) exact matches at 1.0, and Fruit/Rybka has (significantly) more of these than any other pair. I personally think adding in partial scores is a superior method.

This obviously is going the wrong way. If you can't even admit a (small) mistake of your work then Mr. Schröder is done with you.

Code: Select all

         FRUIT                                 RYBKA
         if (doubled) {                        if (doubled) endgame -= 158
            opening[me] -= DoubledOpening;
            endgame[me] -= DoubledEndgame;

There is NO midgame code in Rybka. In your document it is rewarded as a 100% similarity with Fruit.

[For that matter, recall that most of the Panel did not particularly find EVAL_COMP's quantification of the situation to be necessary, and it was largely produced in response to comments by van Kervinck and Roberson].

Is irrelevant, it was another attack document, it influenced all.

I leave it up to you how to continue.

[hyatt]

But I haven't seem a program that gives NO PENALTY AT ALL for any double pawn in the middle game and only in the endgame.

There is NO midgame code in Rybka.

Midgame in Rybka is an interpolation of opening (24 phase units) and endgame (0 phase units). Rybka gives a (nonzero) penalty for a doubled pawn once any piece has been captured (on a pure phase basis, 24/25th of time, or 96% which would round up ). Furthermore, EVAL_COMP is primarily about features, not about numerical tuning. In this sense, Fruit and Rybka (and Faile and ExChess) use the "same" doubled pawn feature, while some other engines listed are of a more varying nature.

the similarity with Rybka is exactly 0.0

If you wish to calibrate the scoring in that way, you are free to do so. You then end up with only (nearly) exact matches at 1.0, and Fruit/Rybka has (significantly) more of these than any other pair. I personally think adding in partial scores is a superior method.

This obviously is going the wrong way. If you can't even admit a (small) mistake of your work then Mr. Schröder is done with you.

Code: Select all

         FRUIT                                 RYBKA
         if (doubled) {                        if (doubled) endgame -= 158
            opening[me] -= DoubledOpening;
            endgame[me] -= DoubledEndgame;

There is NO midgame code in Rybka. In your document it is rewarded as a 100% similarity with Fruit.

How do you KNOW there is no "middle game code in Rybka"? If that value is tuned to zero, the compiler would optimize it out. The main pattern recognition is done EVERYWHERE, opening, middle game and endgame. There is no MG score added which could be explained by (a) score was tuned to zero and compiler optimized it out or (b) score was tuned to zero and Vas removed that one line. If that was the ONLY similarity, one would dismiss it instantly. But with all the others, it shows a "common structure" between the two programs...

[For that matter, recall that most of the Panel did not particularly find EVAL_COMP's quantification of the situation to be necessary, and it was largely produced in response to comments by van Kervinck and Roberson].

Is irrelevant, it was another attack document, it influenced all.

I leave it up to you how to continue.

[Rebel]

Code: Select all

         FRUIT                                 RYBKA
         if (doubled) {                        if (doubled) endgame -= 158
            opening[me] -= DoubledOpening;
            endgame[me] -= DoubledEndgame;

There is NO midgame code in Rybka. In your document it is rewarded as a 100% similarity with Fruit.

How do you KNOW there is no "middle game code in Rybka"? If that value is tuned to zero, the compiler would optimize it out.

I am tired of assumptions. It's not there. Period.

But I understand how your VIG mind works.

He is always guilty no matter what.

You just ASSUME and the VIG world according to Bob is okay again.

Right?

"If the facts don't fit the theory, change the facts."
--Albert Einstein

It's what you are doing here.

[Chris Whittington]

That was not a chess lesson. It was "RE nonsense". If you read what he wrote, he SPECIFICALLY discussed the difference between a bit board implementation and a mailbox implementation. Something we said we EXPLICITLY abstracted away. A loop vs an AND mask? Bit board vs mailbox, nothing more, nothing less.

So please don't call that a "lesson" to someone who actually teaches programming. That was more of a "chess obfuscation" than anything else. Zach did it exactly right there. I don't think much of the "seems to be, or appears to be" nonsense. It is or it isn't.

Your question two posts back in that thread is also nonsensically phrased. You snipped just the 4 bit-wise ANDs (tst instructions) and left out the rest. You DID realize that the masks and stuff were loaded at the top of the code? You seem almost surprised that some rather convoluted mailbox code collapses into 4 AND operations in a bit board program. This is quite common. I've used the term "bit-parallel" because bit boards let me ask a question with one AND that would take a loop with multiple if-statements inside in a mailbox program. That's part of the attractiveness of bit boards for those of us that decided to invest the time to get comfortable with them.

And you are missing it again, you only look at the code and not what the code does. If you had read Marcel carefully you would had seen Fruit and Rybka are evaluating different, Fruit has the exception code for second ranked pawns that can advance 2 squares. Rybka has not, just a quick and dirty evaluation. Fabien uses the Kmoch definition, Vas not. Does the name Hans Kmoch and his basic stuff about pawns every chess player needs to know mean anything anythng to you? There is even a whole page on the CPW about him.

You see, there is this pattern, you ASM geeks (of which I belong too) only look at the code and shout EUREKA!, this piece of Rybka code smells to similarity with Fruit and then you forget to look what the code actually evaluates, as in this case.

So my statement still stands:

COMP EVAL fails to mention that EVERY eval ingrediënt in Fruit and Rybka is coded DIFFERENTLY even after substraction of the Mailboard / Bitboard differences.

Next one?

You miss MY point. Do you know why Rybka doesn't do that? Not so convenient in bitmaps. A pretty natural part of conversion from one to the other. I had several such things that were in Cray Blitz but which I left out of Crafty because (a) it was not clear if they were really beneficial and (b) it was more complicated (affecting speed) to do it the same way. Taking both of those into consideration...

BTW we didn't say "smells to similarity." We said "they do the same thing" or in a few cases "almost exactly the same thing but there is one difference."

The rook scoring (open/half-open files) is a good example. I'd bet that was an outright error in translation, seeing as how it is actually a -25 Elo change.

If you want to hang on to that "every" simply because the code doesn't match, and it obviously could NOT match when using two different board representations" then that's up to you. I know better.

Bet taken. $25,000 that you can't prove beyond any reasonable doubt that the Rybka rook-pawn code we were discussing was a "outright error in translation" within the next two months. I'll send you my lawyers address, we can both deposit $25,000 each. The $50,000 goes to me when you lose, else vice versa.

I can hear the cluck-cluck noises already.

I thought about this overnight. I'll take that bet slightly reworded. YOU prove that he did NOT design the code as I speculated. I can hear the cluck-cluck noises already. Because now the onus would be on YOU. And it is an absolute certainty that you can't prove someone didn't so something. And NO, his word would NOT be proof.

Ah, the bet!! You want to do it, or just trying to pretend not to be big chicken? hahaha?

So, Hyatt makes some convoluted negation to the logic, where I am either quite lost in parsing, or too lazy to work out. In plain English what exactly are you proposing the bet to be? Is it balance of probabilities I have to exceed, or are you going to be so lacking in confidence you need beyond reasonable doubt, and what exactly is the proposal? It appears to be, "I need to prove that he did not copy the code and make an outright translation error" - right? On balance of probabilities? 51% It's irregular, because this assumes him guilty unless he can prove innocent, but that is Hyatt VIG bias all along, I suppose ....

Is that your bet? $25,000 or $50,000?





*