Thoughts on Fruit=Rybka EVAL

General discussion about computer chess...
hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: Thoughts on Fruit=Rybka EVAL

Post by hyatt » Wed Aug 17, 2011 2:28 pm

Rebel wrote:
BB+ wrote: Overall, I just can't see much either way. There are various places where Rybka follows a different agenda (such as re-factoring the code), but on the other hand, there are an assortment of orderings (most rather minor by themselves) that are candidates for being different, that in fact are not. One could argue that in most of the places where there is a re-ordering, there is some external reason for this, possibly from the performance standpoint (like shelter_storm in pawn_eval), or maybe due to ease of development. However, I'm not completely convinced that every Rybka/Fruit difference could be explained by this.

An alternative example where Rybka does re-order a few Fruit elements is in the UCI parsing, where Fruit works alphabetically (binc/btime/depth/infinite/mate/movestogo/movetime/nodes/ponder/searchmoves/winc/wtime), while Rybka reorders this (winc/wtime/binc/btime/depth/infinite/movestogo/movetime/ponder), placing winc/wtime at the front (and not having mate/nodes/searchmoves). So other than winc/wtime at the front, it's alphabetical in Rybka -- but the winc/wtime change shouldn't be simply ignored. Again it is unclear to me whether there might be some external "testing"(?) reason why one might want to move winc/wtime to the front. Zach or I could track down the order in Rybka 1.6.1, if it is thought relevant.
Thank you for clarification.

To say it in a few words, you basically discovered that comparing the eval of 2 good chess programs contain about the same amount of chess knowledge and its implementation look similar.

I could have told you that from the beginning.

I don't see where that is the case at all. One could certainly compare Crafty and Fruit evals, Crafty is definitely stronger, but not by hundreds of Elo. That will shoot holes in your conclusion.

I don't get this claim of "everybody does the same things, in the same order, in the same way..." That is contrary to anything I have seen in looking at other programs over the years. It is more like "everybody does lots of the same things, but in different ways, in different orders, in different places..."

User avatar
kingliveson
Posts: 1388
Joined: Thu Jun 10, 2010 1:22 am
Real Name: Franklin Titus
Location: 28°32'1"N 81°22'33"W

Re: Thoughts on Fruit=Rybka EVAL

Post by kingliveson » Wed Aug 17, 2011 4:56 pm

My thought is Vas should release Rybka 1.0 source code, and that is what you guys should be asking. There's no secret in Rybka 1.0, so it is not as if he would be revealing anything new. As far as am concerned, he was correctly found guilty and will remain so until he releases Rybka 1.0 source. If am able to dig up non-consequential code written about a decade ago when learning programming, am sure he has the source at his disposal. Ed, Chris, and the rest of our friends at Rybka forum can speculate from here till kingdom come -- it is meaningless and carries no weight.

Code: Select all

/*
Assignment:         2 (Simple GUI Composer)
Author:             Franklin Titus
Date:               02.07.02
*/

import java.awt.*;
import java.awt.event.*;
import javax.swing.*;



public class siGuCo extends JFrame implements ItemListener, ActionListener,
                                            MouseListener, MouseMotionListener{
  JButton button;
  JList list ;
  JTextField text;
  JComboBox combobox;
  JLabel label;
  String[] num = {"One", "Two", "Three"};

  JLabel color;
  JLabel mouse;
  JLabel sel;

  Object source;
  Font font;

  int i;
  int r, g, b;

  JPanel west = new JPanel();
  JPanel center = new JPanel();
  JPanel south = new JPanel();

  String[] strOpt = {"Button", "List Box", "Text Box", "Combo Box", "Label"};
  private JRadioButton rButtons[];
  private JComboBox  fonts;
  private JButton undo = new JButton("Undo");
  private JButton clear = new JButton("Clear All");
  private ButtonGroup group;
  private JButton BColor = new JButton("Background Color");

  public siGuCo(){
    super("Simple GUI Composer");

    Container c = getContentPane();

    rButtons =new JRadioButton [strOpt.length];
    group = new ButtonGroup();
    west.setLayout(new GridLayout(0,1,0,0));
    for(i = 0; i < rButtons.length; i++){
      rButtons[i] = new JRadioButton(strOpt[i]);
      group.add(rButtons[i]);
      west.add(rButtons[i]);
      rButtons[i].addItemListener(this);
    }

    fonts = new JComboBox(GraphicsEnvironment.getLocalGraphicsEnvironment()
                          .getAvailableFontFamilyNames());

    undo.addActionListener(this);
    clear.addActionListener(this);
    BColor.addActionListener(this);
    center.addMouseListener(this);
    center.addMouseMotionListener(this);

    west.add(fonts);
    west.add(undo);
    west.add(clear);
    west.add(BColor);

    south.setLayout(new FlowLayout(3,20,0));
    sel = new JLabel();
    mouse = new JLabel();
    color= new JLabel() ;
    south.add(sel);
    south.add(mouse);
    south.add(color);

    west.setBackground(Color.blue );
    c.add(west,BorderLayout.WEST);
      center.setBackground(Color.getHSBColor(104.0f,401.0f,140.0f));
    c.add(center);
      south.setBackground(Color.cyan );
    c.add(south,BorderLayout.SOUTH);

    setSize(500,500);
    show();
     r = center.getBackground().getRed();
      g = center.getBackground().getGreen();
      b = center.getBackground().getBlue();
      color.setText( "Background" + " " +r+"R,"+ " " +g+"G," + " " +b+"B");

  }

   public static void main( String args[] ){

      siGuCo app = new siGuCo();

      app.addWindowListener(
         new WindowAdapter() {
            public void windowClosing( WindowEvent e ) {
               System.exit( 0 );
            }
         }
      );

   }

  public void itemStateChanged(ItemEvent e){
     font = new Font(fonts.getSelectedItem().toString() ,Font.PLAIN, 12 );
     source = e.getSource()  ;
     if ( source == rButtons[0] ){
      sel.setText("Button Selected");
      repaint();
      validate();
    }
    else if(  source == rButtons[1] ){
       sel.setText("List Selected");
       repaint();
       validate();
    }
    else if ( source == rButtons[2] ){
      sel.setText("Textfield Selected");
      repaint();
      validate();
    }
    else if ( source == rButtons[3]){
      sel.setText("Combo Box Selected");
      repaint();
      validate();
    }
    else if(  source == rButtons[4] ){
      sel.setText("Label Selected");
      repaint();
      validate();
    }



  }

  public void actionPerformed(ActionEvent butt){

    if (butt.getSource() == undo){
       if(center.getComponentCount()==0 ){
          center.removeAll();
          center.repaint();
          repaint();
          validate();
        }
        else {
          center.remove(center.getComponentCount()-1);
          center.revalidate();
          center.repaint();
          repaint();
          invalidate();
        }
     }
    if (butt.getSource() == clear){
      repaint();
      center.removeAll();
      center.repaint();
      repaint();
      invalidate();
    }

    if (butt.getSource() == BColor){
      center.setBackground( JColorChooser.showDialog(center,"color",Color.cyan )) ;
      r = center.getBackground().getRed();
      g = center.getBackground().getGreen();
      b = center.getBackground().getBlue();
      color.setText( "Background" + " " +r+"R,"+ " " +g+"G," + " " +b+"B");
      south.add(color).
      repaint();
      south.repaint();

    }

  }

  public void mouseClicked( MouseEvent m){
    mouse.setText("Clicked at" + " " + m.getX() + "x" + " " + m.getY()+"y");

  }

  public void mousePressed(MouseEvent m){
     mouse.setText("Pressed at" + " " + m.getX() + "x" + " " + m.getY()+"y");
  }

  public void mouseReleased(MouseEvent m){


   mouse.setText("Released at" + " " + m.getX() + "x" + " " + m.getY()+"y");
   center.setLayout(null);
   if ( source == rButtons[0] && rButtons[0].isSelected()   ){
      button = new JButton("Button");
      sel.setText("Button Selected");
      button.setFont(font);
      button.setSize(106,61);
      button.setToolTipText("This is a Button");
      button.setLocation(m.getPoint());
      center.add(button);
      center.repaint();
      repaint();
      validate();
    }
    else if(  source == rButtons[1] && rButtons[1].isSelected() ){
       list = new JList(num);
       sel.setText("List Selected");
       list.setFont(font);
       list.setVisibleRowCount(2);
       JScrollPane sPane = new JScrollPane(list);
	   sPane.setSize(56,39);
	   sPane.setLocation(m.getPoint());
	   center.add(sPane);
	   center.repaint();
       repaint();
       validate();
    }
    else if ( source == rButtons[2] && rButtons[2].isSelected()){
      text = new JTextField(9);
      sel.setText("Textfield Selected");
      text.setFont(font);
      text.setSize(80,60);
      center.add(text).setLocation(m.getPoint());
      center.repaint();
      repaint();
      validate();
    }
    else if ( source == rButtons[3] && rButtons[3].isSelected()){
      combobox= new JComboBox(num);
      sel.setText("Combo Box Selected");
      combobox.setFont(font);
      combobox.setSize(81,41);
      center.add(combobox).setLocation(m.getPoint());
      center.repaint();
      repaint();
      validate();
    }
    else if(  source == rButtons[4] && rButtons[4].isSelected()){
      label = new JLabel("Label");
      sel.setText("Label Selected");
      label.setFont(font);
      label.setSize(43,34);
      center.add(label).setLocation(m.getPoint());
      center.validate();
      center.repaint();
      repaint();
      validate();
    }



  }

  public void mouseEntered(MouseEvent m){
      mouse.setText("Mouse in Window");
  }

  public void mouseExited(MouseEvent m){
    mouse.setText("Mouse outside Window");
  }

  public void mouseMoved( MouseEvent m){
    mouse.setText("Moved at" + " " + m.getX() + "x" + " " + m.getY()+"y");
  }

  public void mouseDragged( MouseEvent m){
    mouse.setText("Dragged at" + " " + m.getX() + "x" + " " + m.getY()+"y");
  }

}
You want to help Vas? Ask him to release Rybka 1.0 source code.
PAWN : Knight >> Bishop >> Rook >>Queen

hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: Thoughts on Fruit=Rybka EVAL

Post by hyatt » Wed Aug 17, 2011 8:28 pm

No doubt that would answer a lot of questions. I am afraid, however, that the answers would still not be very popular with a few, because there is no doubt that the source will show _exactly_ what the disassembly has shown. It would just be easier to read.

BB+
Posts: 1484
Joined: Thu Jun 10, 2010 4:26 am

Re: Thoughts on Fruit=Rybka EVAL

Post by BB+ » Wed Aug 17, 2011 9:54 pm

Rebel wrote:Thank you for clarification.

To say it in a few words, you basically discovered that comparing the eval of 2 good chess programs contain about the same amount of chess knowledge and its implementation look similar.

I could have told you that from the beginning.
You specifically asked me for the ordering of terms in evaluation. I even sent you an message, saying that it was a nontrivial task for me to chase this down and present in a proper manner, and that if you weren't serious about this, that I considered it to be trolling. Then you respond in the manner above ("look similar" and "could have told you"). I conclude you are trolling.

Furthermore, it is not the "amount" of knowledge that matters, but the "type" also. Even if we merely weighed the "amount": One can note that Rybka 3, a good chess program, has much more chess knowledge, with a different implementation. One can note that HIARCS from 2005, a good chess program, has much more chess knowledge, with a different implementation. One can note that...
Rebel wrote:But you are not a chess programmer.
If I had distributed the engine I wrote, perhaps then you would argue that I am a "jealous competitor" of Vas? This two-way straddle gets tiresome.
Rebel wrote:And you haven't tried Junior, Hiarcs, Fritz.
If I thought it might convince you, I would ask MarkU, SMK, and the Junior team from a code sample from 2005 (Fritz seems harder). I think they are all sufficiently professional in their archiving that this well may be possible. But then should I expect some new angle would be contrived, such as Vas being an IM so he understands evaluation (why then, pray tell, did the evaluation from an IM like Larry Kaufman look so different?), or something of similar limited value?
Rebel wrote:But you can take Rebel, most of the eval stuff is an easy read.
One concludes that REBEL is a good chess engine with a notably different (and dissimilar) evaluation function than either Rybka or Fruit. E.g., neither Fruit nor Rybka looks at hanging pieces, neither Fruit nor Rybka has a king safety module that looks much like REBEL's, neither Fruit nor Rybka has handles isolated pawns via a square-based criterion. These are the three principal specific things mentioned in Inside REBEL. It is hard to say too much about implementation as that is more dependent on the source code, as compared to a general description.

User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: Thoughts on Fruit=Rybka EVAL

Post by Rebel » Wed Aug 17, 2011 10:05 pm

BB+ wrote: I even sent you an message, saying that it was a nontrivial task for me to chase this down and present in a proper manner, and that if you weren't serious about this, that I considered it to be trolling.
Yes you did sent me a PM.

May I post it ?

BB+
Posts: 1484
Joined: Thu Jun 10, 2010 4:26 am

Re: Thoughts on Fruit=Rybka EVAL

Post by BB+ » Thu Aug 18, 2011 1:22 am

Rebel wrote:Yes you did sent me a PM.
May I post it ?
As the thread is spammed anyway at this point:
BB+ wrote: Dear Ed,

If you are not being serious about your comments, please stop it, as I would consider it to be rather annoying trolling.

You perhaps do not realise how much time I have spent/wasted into tracking down various technical facts/details regarding the things you bring up, not to mention the amount of time need to draft and re-draft (and re-re-draft) posts so that they are phrased in the proper manner, etc. If you are at best only semi-serious about them, perhaps I shall consider billing you for the time spent in humoring you with an answer? :shock:

And if you merely wanted a comparison of the Fruit/Rybka evaluation ordering, just ask for it?!

On a personal note, I've been having partial insomnia (perhaps due to jetlag from recent travels, I slept from 4pm to 9pm today/yesterday) the last few days, and perhaps am not in as a good mood as I might be.
So you specifically requested me to do something, then said "I could have told you that from the beginning" when the result came. This combined with puerile histrionics about it being an "atomic bomb" of evidence for you personally (interspersed with cognitions about all top level evaluation functions being much the same), even though I explained ahead of time that structural differences meant that one could not hope to say too much either way [the most notable similarity in "evaluation ordering" is the overall top-level enumeration, and that eval_pattern proceeds in the same manner -- the most notable difference is that various things are re-arranged here and there, though as I mentioned, some of this seems possibly related to bitboard or performance issues].

As such, I conclude from your response [which I could call it rude, or at least dismissive], that indeed you weren't too serious about the request. Indeed, as it's not even clear whether I am a "computer chess programmer" by your standards in any case, I see no reason to perform any more slave-boy tasks for your passing gratification.

Billing: It took me around 3 hours to prepare/draft/re-check the above, plus all the (amortised) invested capital in knowing about Fruit/Rybka in the first place, etc. Since I enjoyed REBEL so much in the 90s, I will give you a rate of only $100/hour, resulting in a fee of $500. You might recall (from an rgcc post in 1997) that I offered [never got a response] to put up a similar amount for a Rebel/GM match, so we can call it even.

BB+
Posts: 1484
Joined: Thu Jun 10, 2010 4:26 am

Re: Thoughts on Fruit=Rybka EVAL

Post by BB+ » Thu Aug 18, 2011 1:23 am

BB+ wrote:
Rebel wrote:But you can take Rebel, most of the eval stuff is an easy read.
One concludes that REBEL is a good chess engine with a notably different (and dissimilar) evaluation function than either Rybka or Fruit. E.g., neither Fruit nor Rybka looks at hanging pieces, neither Fruit nor Rybka has a king safety module that looks much like REBEL's, neither Fruit nor Rybka has handles isolated pawns via a square-based criterion. These are the three principal specific things mentioned in Inside REBEL. It is hard to say too much about implementation as that is more dependent on the source code, as compared to a general description.
Now I realise that the PDF description ended at this juncture, while the underlying webpage continues more in-depth.

Again I will ask you ahead of time: is it worth my time to perform an extended analysis? If I do so, and then respond with an in-depth critical enumeration of REBEL versus Rybka/Fruit, will this suffice? Or can I expect there to be another swerving of what the principal issue is, such as a return to the "it's all general chess knowledge" position? I'm not being completely facetious here -- in the Panel, there was a initial debate as to whether EVAL_COMP was a worthwhile endeavor, so I produced a few examples, and there was general agreement that it was imperfect, but superior to just leaving the question as a qualitative one. Since you've already said that you don't care about EVAL_COMP, it seems unreasonably burdensome to expect me to carry this out, just to please you.

At any rate, the first few sections (Bishops/Rooks) tend to confirm my contention that REBEL is indeed "dissimilar" in contrast to the "overlaps" with Fruit and Rybka. For instance, the logical idea that the value of open files for rooks increase according to number of pawns (this appears in other programs) is not part of the Fruit/Rybka feature list. You use x-ray mobility (R3 has something similar, with pawn skeletons), while Fruit/Rybka do not. The knight/bishop/rook mobility in REBEL is computed "net" (white minus black), and then passed to a lookup table (which starts linear, then increases as the mobility differential becomes extreme). Again this is not much like what Fruit/Rybka do, where each piece type has its own mobility adjustment and the computation is strictly linear. Another point is that REBEL double-counts vertical rook mobility (again LK put something like this in R3).

For passed pawns, the "own support" is different in nature (Fruit/Rybka measure the distance), and the enemy tropism scaling is not as simple as with Fruit/Rybka. There is no "compelling" bonus, and no "heavy" immobility penalty (from a rook). With the next two sections, there is no "minimal mobility" consideration in Fruit/Rybka, and pins also are not considered. They both have no consideration of rook connections or doubled rooks (in all these cases, various other engines in EVAL_COMP do in fact have at least some overlap with what REBEL does).

In short, I think the REBEL evaluation is about as different from either Fruit or Rybka as any other of the "non-matching" pairs of engines in EVAL_COMP. As I say, if you want something more concrete, I would first prefer some indication that you will take the result seriously.

BB+
Posts: 1484
Joined: Thu Jun 10, 2010 4:26 am

Re: Thoughts on Fruit=Rybka EVAL

Post by BB+ » Thu Aug 18, 2011 1:55 am

Chris Whittington wrote:One might also wonder whether the comparators were not cherry picked
I have not seen any argument to substantiate this. If such an issue is to be raised, the evidential burden would seem to rest upon those who suggest it (the rationale for such a requirement is noted rather pointedly in that link). I am more than willing to discuss the issue of "cherry-picking" [which indeed is but one issue that could be raised], but as of yet I can't see anything beyond the mere voicing of the possibility. One way to overcome the aforesaid burden is to give your own listing of evaluation features, in a similar sense to that which EVAL_COMP used.

User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Re: Thoughts on Fruit=Rybka EVAL

Post by Rebel » Thu Aug 18, 2011 8:31 am

BB+ wrote: Billing: It took me around 3 hours to prepare/draft/re-check the above, plus all the (amortised) invested capital in knowing about Fruit/Rybka in the first place, etc. Since I enjoyed REBEL so much in the 90s, I will give you a rate of only $100/hour, resulting in a fee of $500.
The PM is interesting for the following reasons,

1. Is basically telling me to shut up.

2. Raises questions, such as why that urgent need to reply to my (or anyone else for that matter) postings? Since when Is that an obligation on the Internet? And since you are talking to an (although ex) but still a trained commercial able to read between the lines in business terms lemme ask you this simple yes or no question:

Did the ICGA compensate you for your work ?

Perhaps with some aftermath duties ?

It would explain.
BB+ wrote:You might recall (from an rgcc post in 1997) that I offered [never got a response] to put up a similar amount for a Rebel/GM match, so we can call it even.
Sorry, must have missed it. Can't even remember I posted that :lol:

mjlef
Posts: 43
Joined: Thu Jun 10, 2010 6:51 pm
Real Name: Mark Lefler

Re: Thoughts on Fruit=Rybka EVAL

Post by mjlef » Thu Aug 18, 2011 12:34 pm

Rebel wrote:
BB+ wrote: Billing: It took me around 3 hours to prepare/draft/re-check the above, plus all the (amortised) invested capital in knowing about Fruit/Rybka in the first place, etc. Since I enjoyed REBEL so much in the 90s, I will give you a rate of only $100/hour, resulting in a fee of $500.
The PM is interesting for the following reasons,

1. Is basically telling me to shut up.

2. Raises questions, such as why that urgent need to reply to my (or anyone else for that matter) postings? Since when Is that an obligation on the Internet? And since you are talking to an (although ex) but still a trained commercial able to read between the lines in business terms lemme ask you this simple yes or no question:

Did the ICGA compensate you for your work ?

Perhaps with some aftermath duties ?

It would explain.
BB+ wrote:You might recall (from an rgcc post in 1997) that I offered [never got a response] to put up a similar amount for a Rebel/GM match, so we can call it even.
Sorry, must have missed it. Can't even remember I posted that :lol:
Ed,

Your anger is taking away your sense of humor.

Mark

Post Reply