ComStock 1.0.2 Stockfish with robobases

[BigThief]

I found this:

ComStock is ComradesStockFish with RobboBases. I percept he is aimed to emit soon. UpDate: Now Emitted!

New RobboBases solely now, Old I deem possible, yet more hassle upon evasions.

I confounded 5 bugs (total) at now.

Bug with StockFish
"go infinite" immediately then "stop" (buffer) not attended

Bugs (2) with ComradesGUI
ComboBox and names with spaces (UCI "var")
IOF_UCI fails to impair unto NULL

Bugs (2) with IvanHoe
OpenTripleDisk murks out, overflowing fopen limit
Robbo_mossa fails with set to *cp on promotion


http://ippolit.wikispaces.com/file/d...tock-1.0.2.tar

Seems to be interesting. I am curios of the first Window version, 64 bit

kind regards

BigThief
There now also has no testing with WINDOWS in yet.

[mcostalba]

Bug with StockFish
"go infinite" immediately then "stop" (buffer) not attended

I have found and applied this code chunk from GreKo:

Code: Select all

// When using Standard C input functions, also check if there
// is anything in the buffer. After a call to such functions,
// the input waiting in the pipe will be copied to the buffer,
// and the call to PeekNamedPipe can indicate no input available.
// Setting stdin to unbuffered was not enough. [from Greko]
if (stdin->_cnt > 0)
    return 1;

It should work now, thanks for reporting.

[BB+]

Code: Select all

if (stdin->_cnt > 0)
    return 1;

This is also in Fruit (posix.cpp), at least in the #ifdef WINDOWS part:

Code: Select all

   if (stdin->_cnt > 0) return true; // HACK: assumes FILE internals

Cutely, it is rejected in IvanHoe (again in #ifdef WINDOWS):

Code: Select all

#if 0 /* purpose? */
  if (stdin->_cnt > 0)
    return 1;
#endif

[BB+]

If you only patched the #ifdef WINDOWS, I'm not sure this fixes everything, for I can replicate the bug with Linux. The ComStock source code has a meandering description:

Code: Select all

int data_available()
// this fails when "stop" demands exactly (direct) after "go infinite",         
// within the I/O time to consist (load) the buffer, see "go infinite\nstop\n"  
// * the buffer provides "go infinite" plus "stop" in it, reaching ComStock     
// * getline () chews up one line, onely "go infinite", while "stop" in buffer  
// * the UCI "go infinite" calls think (), while "stop" sits still by buffer    
// * Yet: now data_available () is "false" by OS demands (buffer)               
// we fixed this for IvanHoe months ago, read byte-by-byte (one) until newline  

So what I think this means is: have the GUI (or InBetween) send "go infinite\nstop\n" (maybe send "go infinite" with no flush, then "stop"). Stockfish will not recognise the "stop" command. I can try to see if I determine what is going on.

[Michel Van den Bergh]

for I can replicate the bug with Linux.

Are you sure? Under Linux, Fruit uses "select" with timeout zero to do polling. This is the standard method.
I would be surprised if there was a bug in that.

[BB+]

I did not test Fruit. I will look at it too. The problem is in how the engines parse input (they use read or getline at one point, and then query for "input_available" at another, and the data in the pipe is now in a buffer I think -- still needs more investigation).

EDIT: Fruit 2.1 seems OK.

[Michel Van den Bergh]

I did not test Fruit. I will look at it too. The problem is in how the engines parse input (they use read or getline at one point, and then query for "input_available" at another, and the data in the pipe is now in a buffer I think -- still needs more investigation).

EDIT: Fruit 2.1 seems OK.

OK good!

Since select is a libc function I am quite sure that it takes the I/O buffers into account.

The problem with PeekNamedPipe is that it is a WIN32 function and not a C library function. So it does not look at the C buffer. In its infinite wisdom Microsoft has chosen not to implement select in its C library.

[BB+]

I think I have a better grasp of the problem.

Stockfish has a main.cpp parsing loop of:

Code: Select all

     do {
          // Wait for a command from stdin
          if (!getline(cin, cmd))
              cmd = "quit";
      } while (execute_uci_command(cmd));

and execute_uci_command() calls various search functions when a "go" token is found.
The in-search parsing of (search.cpp) is:

Code: Select all

  void poll(const Position& pos) {

    static int lastInfoTime;
    int t = current_search_time();

    //  Poll for input
    if (data_available())
[...]

Now it seems to me that the following happens. In the main.cpp parsing loop, the cin pipe has "go infinite\nstop\n", as the "stop" came quickly (within a parsing) of the "go infinite". The execute_uci_command("go infinite") then runs, and the "stop\n" is still present in cin. In particular, there is no data_available(), as the "stop\n" has already been moved from the pipe into buffers (I guess -- I am no expert on what is happening internally). So the fact that "stop\n" is in cin is not picked up until more data is sent. You can check this by printing data_available() in the above do/while loop, and it returns false. (Again, I am not sure why, but this is what it does). The main difficulty in testing/demonstrating this is to ensure that the controlling GUI sends "go infinite\nstop\n" fast enough.

Fruit 2.1 on the other hand (protocol.cpp) uses get(string,65536); as opposed to getline -- theoretically this could fail if the GUI dumped more than 64K in one go, but...

[BB+]

Here is a test file to run (testfunc.c, also attached):

Code: Select all

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <signal.h>

static int queryin(int fd)
{fd_set S[1]; struct timeval tv[1]; int val; FD_ZERO(S); FD_SET(fd,S);
 tv->tv_sec=0; tv->tv_usec=0; val=select(fd+1,S,NULL,NULL,tv); return (val>0);}

static char A[1<<16];
static void readloop(int from)
{int a=0,b=0; char *u,bm[16]; usleep(1000);
 while (1)
 {if (queryin(from))
  {while (read(from,A+a,1)) {if (A[a]=='\n') break; a++;}
   if (A[a]!='\n') {usleep(1000); continue;} A[++a]=0;
   printf("%s",A); fflush(stdout); return;} a=0;} usleep(1000);}

static void stdin_line(char *str) {fgets(str,65536,stdin);}

char PROGNAME[1024]="";
char CMD[32]="go infinite\nstop\n";
int main(int argc,char **argv)
{pid_t p1; int to_prog[2],from_prog[2]; int i; char str[256];
 if (argc>1) strcpy(PROGNAME,argv[1]);
 pipe(to_prog); pipe(from_prog); p1=fork();
 if (!p1)
 {dup2(to_prog[0],STDIN_FILENO); dup2(from_prog[1],STDOUT_FILENO);
  close(to_prog[0]); close(to_prog[1]);
  close(from_prog[0]); close(from_prog[1]); execl(PROGNAME,PROGNAME,NULL);}
 close(to_prog[0]); close(from_prog[1]); write(to_prog[1],CMD,strlen(CMD));
 for (i=0;i<100;i++) readloop(from_prog[0]); kill(p1,SIGKILL); usleep(1000);}

Compile, and run with the engine as the command line option ("./testfunc fruit").

As can be seen, this sends "go infinite\nstop\n" to the engine. Fruit stops after 6 ply, while Stockfish does not stop (until 100 lines are read, when the tester aborts).

As a complimentary test to the Stockfish main.cpp parsing, here is a function (attached as getme.cpp) that replicates its behaviour:

Code: Select all

#include <cstdio>
#include <cstdlib>
#include <string>
#include <cstring>
#include <ctime>
#include <iostream>

#  include <sys/resource.h>
// #  include <sys/select.h>                                                    
#  include <sys/time.h>
#  include <sys/types.h>
#  include <unistd.h>

using namespace std;

bool input_available()
{

  int val;
  fd_set set[1];
  struct timeval time_val[1];

  FD_ZERO(set);
  FD_SET(STDIN_FILENO,set);

  time_val->tv_sec = 0;
  time_val->tv_usec = 0;

  val = select(STDIN_FILENO+1,set,NULL,NULL,time_val);

  return val > 0;

}


int main()
{
  string cmd;
  getline (cin, cmd);
  cout << cmd << endl;
  cout << input_available () << endl;
  getline (cin, cmd);
  cout << cmd << endl;
}

Compile this, and run the first programme with the second as the argument. You should get

Code: Select all

go infinite
0
stop

In particular, there was no input_available() (the second line has "0"), but getline was able to read "stop" from it. Again I don't claim to know why, though maybe there is some funky mis-matching of low/high-level input routines?

[BB+]

Fruit does not use cin.get() but rather

Code: Select all

void get(char string[], int size) {

   ASSERT(string!=NULL);
   ASSERT(size>=65536);

   if (!my_file_read_line(stdin,string,size)) { // EOF
      exit(EXIT_SUCCESS);
   }
}

and

Code: Select all

bool my_file_read_line(FILE * file, char string[], int size) {

   char * ptr;

   ASSERT(file!=NULL);
   ASSERT(string!=NULL);
   ASSERT(size>0);

   if (fgets(string,size,file) == NULL) {
      if (feof(file)) {
         return false;
      } else { // error
         my_fatal("my_file_read_line(): fgets(): %s\n",strerror(errno));
      }
   }

   // suppress '\n'

   ptr = strchr(string,'\n');
   if (ptr != NULL) *ptr = '\0';

   return true;
}

So Fruit is not using the "getline" method (high-level?), but rather the lower-level "fgets" code. This might make the difference with the "input available" functions, due to buffering. That is, fgets could preserve select correctly, while getline might not. Again I am more musing on what could be wrong than anything else.