Eman 9.30 vs Stockfish pre release

[Desvelemosafrica]

n this 1000 match match, the latest version of Omar takes on the pre-release version of stockfish, on the cutechess gui at 1 minute with a second increment. With a core for each chess engine and 256 mb hash. Maybe I'll be wrong, but I think that to assess the strength of a chess engine the time controls should be classical time controls. In any case, there is no opening book, but in the case of Eman, although he started from scratch on cutechess with the first tests, he has an experience file already worked on, although I insist on being a bullet, said file would not be useful in control games long time. I leave you the games to date and a screenshot with the current results. Sorry for the issue that you can not see the images, but click on the link and you can see the results. A cordial greeting.

download games:
https://pixeldrain.com/u/5EzE8qw6

[deeds]

...With a core for each chess engine and 256 mb hash...

So each Eman's instance will read/write on the same Eman.exp at the same time ?
Please, don't do the same error as Sedat.

...he has an experience file already worked on...

If you use an experience file, 256 MB are not enough.

[Desvelemosafrica]

In this case, when using these time controls I was unaware that when using an exp.file you had to use more hashes. Yes, I know that to use the 7 tbs in the architecture that I use, two Hp proliant dl360p gen 8. with 224 gb of ram ecc yes I use above 6144 even 8192 for blitz but due to the issue of using a large number of finals of 7 pieces. I am clear that in the next test, I am going to use only 3,4,5. But using a core and 1 minute with a second increment and a 300 mb experience file, I don't think it's necessary to use more than 512 mb if you rush me 1024. The truth is that since my friend sedat uses cutechess, logically I've used their configurations, because I understand that he has been testing and testing for many years. In any case, with 738 games finished, the result is in favor of Eman 27-18-693. but it must also be said that the experience file is the one that has helped to come back because until close to 500 games, stockfish won by three difference. I'll take into account what you tell me about the hash tables, although for bullet and for a relatively small experience file compared to the ones I've seen, I can't say I've played wrong so far. Thanks for your comment.

[deeds]

The main idea behind a hash of 512 MB is when the engine defrags its experience file. Unenlightened daily testers as Sedat, Oliprog, Cubail, etc don't know at each new game the learning engines reorder experience entries, remove duplicated, recalculate the score's quality , etc. Several milllions of experience entries to defrag, it is faster on ram (large page) than on ssd/hdd.

[Desvelemosafrica]

On this subject of the experience files, although I have been with eman for many months now, I am completely unaware of what you have just commented on. I will also ask the author himself now by email about this topic, because although I know that the best performance is in long games, it is in those bullet times, for example (they have recommended me to use a different experience file than the one I use in long games and It has its logic) the subject of the ram if I just understood it according to the explanation that you just gave me. Then thanks for your comment, in fact I have visited your website and I see that you have experimented with this topic.

[deeds]

On training mode (=default settings like "experience book" set to false, no d.c.s, etc) there is a logic if the score improves only after 500 games since Khalid advises between 500 and 1000 games per opening. On training mode, the moves engine only are at 70-75% in conformity with their experience data, very often the engine tries worst moves in order to learn. After hundreds of games, several moves of the main "key" positions have been played enough in order to know which ones are the most efficient. The engine has learned, the scores improve.

[Sedat Canbaz]

So each Eman's instance will read/write on the same Eman.exp at the same time ?
Please, don't do the same error as Sedat.

Sad really...you are talking and talking...however,
Do you have own hashtable testings under many hardwares ?

Btw, why each time we have to concentrate over comments...
In other words, what about instead of Theory to talk about
Practice? would not be better...?

Anyhow, I mean especially for Blitz 2m+1s, 1 Core, 2x EPYC 7B12?
Do have any hashtable testings that 128 MB hash is not so good?
Even if you have...it does not mean that, your testings will be
As good recommendation for all Harwdare users, Engines, GUIs. etc. !
Meanwhile, even GUIs are affecting the performance of engines:
https://sites.google.com/site/computers ... testings-5
Just saying...

To be more clear,
Any chess engine's hash size performance depends on
Our used hardware speeds, Core/s, Time controls, etc.
Even large exp files can affect the engine performance..
For this, I suggest plenty of ram.. especially if we run
Concurrent, parallel matches..

As another reference, during CS XLVII's strength testings:
https://sites.google.com/site/computers ... 47-details
We noticed that 128 MB hash performed better than 256 MB hash

3rd Test: CF EXT with 256 MB Hash

Code: Select all

1   CF EXT 161120 1 Core  +25/-6/=83 58.33%   66.5/114
2   SF200620MZ 3 Cores    +6/-25/=83 41.67%   47.5/114

CF EXT 161120 used to play via nn-cb26f10b1fd9 and 256 MB hash
SF200620MZ is played with Contempt 40, 3 Cores, 512 MB hash
The Elo Difference is 58

4th Test: CF EXT with 128 MB Hash

Code: Select all

1   CF EXT 161120 1 Core  +127/-9/=334 62.55%  294.0/470
2   SF200620MZ 3 Cores    +9/-127/=334 37.45%  176.0/470

CF EXT 161120 used to play via nn-cb26f10b1fd9 and 128 MB hash
SF200620MZ is played with Contempt 40, 3 Cores, 512 MB hash
The Elo Difference is 89
------------------------------------------------------------------

In another Duel, I mean that Raptora 2.2 with 128 MB hash is
performed really so good (even Raptora played with huge Experience file):
https://sites.google.com/site/computers ... k-vs-exp-3

-------------------------------------------------------------------
As another, but new reference (both engines used with 128 Hash),
I mean in the recent Duel match: we noticed also that both
(Eman and Raptora) are performed almost identically:
https://sites.google.com/site/computers ... vs-raprora
----------------------------------------------------------------------
Forget everything e.g on my older machine (2x E5-2686), e.g
Via fast Bullet 30sec+0.6sec, even smaller hash is better:
https://sites.google.com/site/computers ... provements

3. Hash-Table Test: 64 MB vs 128 MB

Code: Select all

1   Stockfish 15 64mb    +10/-5/=185 51.25%  102.5/200
2   Stockfish 15 128mb   +5/-10/=185 48.75%   97.5/200

---------------------------------------------------------

What does it mean, after all ?)
As I stated before (for best conditions):
Please run your own hardware hash testings!

Otherwise, like what currently Deeds says...
It is a simply a wrong way...hope helps..

As final words,, for a better conclusion:
My 1st job will be to run a new Raprora hash test!
But not with Eman, reason..not so stable...crashes with concurrent: 64
Sure I will do Raptora test later...due to nowadays I have
No much free time for chess eng testings...

Greetings

[deeds]

Then thanks for your comment, in fact I have visited your website and I see that you have experimented with this topic.

No problem, yes I used Eman a bit... (11th learning session at the moment lol)

At least, you aren't confused between the hash size for defragmenting the experience file and the hash size for engine speed. I haven't this problem because i never run Eman with concurrency > 1 nor with only 1 thread, there is no interest because Eman hasn't "concurrent experience" like BrainLearn.

[rulleeeee]

"Concurrent Experience" can you explain it to me simply because from what asmchess wrote for brainlearn I don't really understand these options. if i use Chessbase and Banksia gui should i use this option in Brainlearn?

[Sedat Canbaz]

Hello dear Jordi,

Many thanks for the test...

n this 1000 match match, the latest version of Omar takes on the pre-release version of stockfish, on the cutechess gui at 1 minute with a second increment. With a core for each chess engine and 256 mb hash. Maybe I'll be wrong, but I think that to assess the strength of a chess engine the time controls should be classical time controls.

Well, I noticed also that,
Your used time control is not exactly 1min+1sec, for example after checking
I see it as 1'+1''/500

About using Classical Time Controls,
It looks like as best choice, but I am afraid to say that
It's not so good idea, the reason: very likely (under optimal cond.)
All games will be ended as draw...
Plus, as you are single tester, no a way to spend so much CPU time
Just in case..it will take a long period of time..maybe months..or year..
So.. you are on right way that you did not prefer Classical TC )

In any case, there is no opening book, but in the case of Eman, although he started from scratch on cutechess with the first tests, he has an experience file already worked on, although I insist on being a bullet, said file would not be useful in control games long time. I leave you the games to date and a screenshot with the current results. Sorry for the issue that you can not see the images, but click on the link and you can see the results. A cordial greeting.

By the way, can you inform us please,
1 core test..if not... with how many cores?
I wonder too, what is the concurrent option ?
And how much RAM is installed on hardware system ?

Be aware of that too (according to my experience):
If you are testing the engines (in parallel matches)... then
You need plenty of ram.. because huge exp files consume a lot of ram!
E.g in my testings, my hardware (2x Epyc 7B12) is based on 256 GB of ram
I mean during my test: hardware's RAM usage is usually lower than 30%
And the CPU usage is usually about 50%

In other words, if Exp testings are via parallel matches:
Be careful please with your tour setup ! otherwise
With wrong setup...the engines performance may suffer a lot...


Best,
Sedat