What do you folks make of this ?

[hyatt]

However, one does not use setjmp()/longjmp() in an asm program, so the comment by Chris is misplaced. And as I repeatedly stated, that is a HORRIBLE way to unwind a search in C. It creates a real debugging problem, and since most intend on doing a parallel search one day since nearly every machine sold today has multiple cores, it greatly adds to the misery of doing a parallel search...

Everybody has done trickery with registers. We were at the top of that list on the Cray. But that's irrelevant here.

[marcelk]

However, one does not use setjmp()/longjmp() in an asm program, so the comment by Chris is misplaced. And as I repeatedly stated, that is a HORRIBLE way to unwind a search in C. It creates a real debugging problem, and since most intend on doing a parallel search one day since nearly every machine sold today has multiple cores, it greatly adds to the misery of doing a parallel search...

Everybody has done trickery with registers. We were at the top of that list on the Cray. But that's irrelevant here.

Would you mind to elaborate? I unwinded the stack with longjmp-like methods when my program
was still assembly. The advantage is that there is no extra logic in the critical path of the computation.
You just bail out, bypass everything.

In my C programs I do the same, for the same reason. If you take care of what the C standard
guarantees with respect to volatility of local variables, there is no problem I can think of.
You have to understand what C is guaranteeing and what not, and why. But it is all very reasonable.

Recently I made my program parallel, it still uses setjmp/longjmp and I had absolutely no debugging issues
with it when going SMP. What am I missing?

[hyatt]

However, one does not use setjmp()/longjmp() in an asm program, so the comment by Chris is misplaced. And as I repeatedly stated, that is a HORRIBLE way to unwind a search in C. It creates a real debugging problem, and since most intend on doing a parallel search one day since nearly every machine sold today has multiple cores, it greatly adds to the misery of doing a parallel search...

Everybody has done trickery with registers. We were at the top of that list on the Cray. But that's irrelevant here.

Would you mind to elaborate? I unwinded the stack with longjmp-like methods when my program
was still assembly. The advantage is that there is no extra logic in the critical path of the computation.
You just bail out, bypass everything.

In my C programs I do the same, for the same reason. If you take care of what the C standard
guarantees with respect to volatility of local variables, there is no problem I can think of.
You have to understand what C is guaranteeing and what not, and why. But it is all very reasonable.

Recently I made my program parallel, it still uses setjmp/longjmp and I had absolutely no debugging issues
with it when going SMP. What am I missing?

The concept of "longjmp()" does not apply to assembly language programming. But as far as problems go...

make/unmake. How do you "jump out of the recursion stack" and get back to the root with a valid chess board?

parallel search. How do you jump out of the search, while threads are still busily updating their local data and global data, and prevent a nightmare of races?

Any idea: what is reset to where it was (local stack) and what is not (global data)? Unwinding makes sure that everything is backed out normally, no kludges necessary, no variables left accidentally changed or else no variables that have to be manually saved and restored after a longjmp() is executed. It is simply a lousy way of writing a program and most do not do that for some or all of the reasons I quoted. In asm, things are different. Most don't even do a recursive search in asm although it is possible. Most use a more natural iterated approach which is different. The concept of local and global variables are not quite the same since everything is done at a very low level.

You say you use that approach with a parallel search. I'd bet you don't just longjmp() out, you first set some sort of indicator that tells the threads to back out, or quit, or something FIRST. Else you do have some race conditions, you just haven't found them yet.

[marcelk]

And as I repeatedly stated, that is a HORRIBLE way to unwind a search in C. It creates a real debugging problem, and since most intend on doing a parallel search one day since nearly every machine sold today has multiple cores, it greatly adds to the misery of doing a parallel search...

If you take care of what the C standard guarantees with respect to volatility of local variables, there is no problem I can think of. You have to understand what C is guaranteeing and what not, and why. But it is all very reasonable.

Recently I made my program parallel, it still uses setjmp/longjmp and I had absolutely no debugging issues
with it when going SMP. What am I missing?

make/unmake. How do you "jump out of the recursion stack" and get back to the root with a valid chess board?

In my C programs I keep the search stack separated from the function invocation stack. I longjmp() out of the latter but unwind the first. (In my assembly programs they were intermingled, but that is not an issue there)

parallel search. How do you jump out of the search, while threads are still busily updating their local data and global data, and prevent a nightmare of races?

For my SMP implementation my priorities were quite clear: I wanted correctness first, meet the deadline for DOCCC'10 second and simplicity third (and speed last, speed must wait if you prioritize this way). Therefore I designed my SMP around processes, not threads. Work on it started on Sep/18 and I fixed the last SMP-related bug on Nov/25, just in time for the tournament. I have not found other SMP-related bugs afterwards either and the program is playing in SMP mode continuously on the servers. With processes sharing is off by default and sharing can be added where needed instead of the other way around. The choice of processes vs threads has a much more profound impact on debugging misery than the choice of how to unwind the stack.

Any idea: what is reset to where it was (local stack) and what is not (global data)?

You say you use that approach with a parallel search. I'd bet you don't just longjmp() out, you first set some sort of indicator that tells the threads to back out, or quit, or something FIRST. Else you do have some race conditions, you just haven't found them yet.

I'm quite sure I have no race conditions in my program except for those originating from transposition table clashes. I resolve those by being resilient against reading illegal moves, and accepting that once in a while I use the wrong score from the table.

[hyatt]

I've been writing parallel programs since 1978 and I would NEVER assume there are no bugs, just because nothing crashes. I have fixed more subtle bugs, and it really has nothing to do with threads vs processes. Except that threads are more efficient for some things (egtb access is one since egtb.cpp is thread-aware and you don't duplicate the cache N times...

But experience suggests that the term "debugged parallel program" is an oxymoron...

[marcelk]

I've been writing parallel programs since 1978 and I would NEVER assume there are no bugs, just because nothing crashes. I have fixed more subtle bugs, and it really has nothing to do with threads vs processes. Except that threads are more efficient for some things (egtb access is one since egtb.cpp is thread-aware and you don't duplicate the cache N times...

But experience suggests that the term "debugged parallel program" is an oxymoron...

I don't assume I have no bugs because it doesn't crash. I assume I have bugs because I have been writing parallel programs with wooden sticks on clay disks and the wheel was a big thing at the time. I sure have bugs (last month I found one in my PGN handling code and it was 15 years old, and not in difficult code). I assume I have no SMP bugs that -affect- me because the SMP code is simple, passes my stress tests, plays strong chess on servers AND doesn't crash... At that point worrying about such bugs is not just low on my priority list, it is not on the list at all.

Regarding EGT: simplicity and processes is why I have placed the EGT data in a huge shared library and let the kernel handle the access and sharing. The EGT code is nearly absent because Linux provides it for me. I noticed Crafty doesn't use EGTs at all anymore and misses endgame wins because of that.

But back to the point: My program uses setjmp/longjmp and I don't agree that it is a sign of copying chess playing code. I got the idea from K&R, not from Fruit. I wasn't aware that Fruit was using it as well for the same purpose.

[hyatt]

I've run a ton of tests. My finding on the cluster suggested a tiny Elo LOSS, not gain, for using EGTBs. The problem is the slow-down that occurs everywhere in endgames, while the exact information doesn't seem to make that much of a difference overall (I will add that some endgames, like KRP vs KR find code in Crafty to help. But I'm no longer a big believer in the things. The memory-resident bitbase approach might actually be viable, I have not tested that since I do not use them in Crafty, yet...

Yes, it will miss something here and there without them. It misses a little more with them because of the speed loss...

[geotsp8]

From what I understand there is nothing in Rybka 1 Beta that is a direct identical copy of Fruit.

Perhaps you should give this a looksee then.



https://webspace.utexas.edu/zzw57/rtc/eval/eval.html

The expert witness had already found the defendant guilty. All that remained was the writing of a suitable report. The language is biased, the reader is told what to think, where there are dissimilarities the reader is assured, for example, 'this is done for optimisation reasons only' and is left to assume the dissimilarity doesn't really exist. Where Rybka uses tables, the expert witness 'recreates' the code he imagines was used to build the table data (although he never saw the code in binary form!) and unsurprisingly it matches directly with Fruit code - well, if you want to find/prove something and are sufficiently biased and committed to the result ...... Please read that last bit carefully, again: the expert witness took a lookup table of data, imagined up the code required to recreate that data (this without access to the program that created the data which he has never seen), wrote the code as identical to some Fruit source, and says "it's the same, but optimised". Ridiculous, but it gets worse in this same code section ..... for the sub-function called in the above that looks at pawn defence in front of the king returns completely different values for each program, the sub-functions are given the same name by the witness, but they do different things. And, lastly on this section, what program doesn't look in some way at open files in front of the king? Just not good enough.

In king_safety as another example both Fruit and Rybka look at piece attacks on the eight squares around the king. Er, who doesn't do this? But it is treated as a similarity proof. The expert witness skips over the fact that Rybka does it all very fast with btiboards (who wouldn't with a bb program?), while Fruit code is longer and slower, done without bb's (well it would be wouldn't it?). We're told that the 'sum of weights' for the attacks are different for each program, but the fact that pawns are treated as zero (what a surprise!!) is yet more 'proof' of similarity, while all differences (massive I would imagine) are just the result of 'optimisation' (again). Hahahaha!!!

Just for starters.

In a court the witness document would be torn apart and rapidly discredited. I am shocked that elements of comp chess community have placed so much weight on it.

I don't have to know crapola about programming and C language to see that Whittington has all guns blazing, and Hyatt is frantically running in circles trying to figure a way out of here and saving face at the same time.

There is no way ANYONE can convince me that a hell of a lot of programmers and C language experts don't understand exactly what Chris is saying. Which leads to the question of why they ignore it and won't even debate him on it. I have a great deal of respect for Dailey, I think. But he is quick to jump on my threads, telling me I am a vicious person and much worse. Where is he when Whitty posts? In fact, where are a hell of a lot of them? ....... Pick who you want- I will stick with Whitty, Miguel and Sven. If I have them, I can ride into hell, put the fire out, and then set it on blocks so the water can drain out.

gts

[hyatt]

Your problem is that you don't SEE anything at all, because you have your eyes closed. I'm not "frantically running around." The evidence stands on its on. So far, NONE has been discredited in the least. I don't expect that to happen in the future either. If you want to buy stock in what CW says, go for it. People lose their shirts every day making bad choices after taking bad advice... You won't be the last.

[orgfert]

I don't have to know crapola about programming and C language to see that Whittington has all guns blazing, and Hyatt is frantically running in circles trying to figure a way out of here and saving face at the same time.

There is no way ANYONE can convince me that a hell of a lot of programmers and C language experts don't understand exactly what Chris is saying. Which leads to the question of why they ignore it and won't even debate him on it.

Because unlike you, they know more than "crapola" about chess programming. The question is why you failed to consider this fact.

I have a great deal of respect for Dailey, I think. But he is quick to jump on my threads, telling me I am a vicious person and much worse. Where is he when Whitty posts? In fact, where are a hell of a lot of them?

They are smart enough to refrain from feeding the troll.

....... Pick who you want- I will stick with Whitty, Miguel and Sven. If I have them, I can ride into hell, put the fire out, and then set it on blocks so the water can drain out.

So has the Devil promised all his followers. Good luck with that.