SedatChess

[Sedat Canbaz]

Hello there,

After so many comments about Eman engine
I realized to run several quick Eman testings...
Sure as main target to see the learning influence..
E.g Eman's and GUI concurrency option as 1 (one)
Plus Eman's and GUI concurrency option as 8 (eight)
And Eman's Experience MultiPV is Disabled

1st Test: Ruleeee exp + Eman 8.91 vs SF-PB 060623
Note: Cutechess GUI's concurrent games: 1

2nd Test: No Pre Learned Exp + Eman 8.91 vs SF-PB 060623
Note: Cutechess GUI's concurrent games: 8

3rd Test: Pre Learned Exp + Eman 8.91 vs SF-PB 060623
Some Notes about 3rd test:
Cutechess GUI's concurrent games: 8
Eman's Experience MultiPV is Disabled
Pre Learned Exp is the same one of 2nd test...

Both (1st+ 2nd) testings are ended...
Right now the 3rd test is running..

Sure I hope to publish all results via more details very soon

And stay tuned please.. )

Greetings

[Sedat Canbaz]

And another new idea is born : Experience MultiPV is Enabled

4rd Test: Pre Learned Exp + Eman 8.91 vs SF-PB 060623
Some Notes about 4rd test:
Cutechess GUI's concurrent games: 8
Eman's Experience MultiPV is Enabled
Pre Learned Exp is the same one of 2nd + 3rd test...

[Sedat Canbaz]

UPDATE

1st of all, here are the conditions:
Ryzen7 4800H, CuteChess, 1 Core, Ponder OFF, 1m+1s, 128 MB Hash, 4-MEN
As book, Unique v50 opening suite is used for all and played as twice!

Code: Select all

1st Test Results: +17 Elo (in favor for SF-PB)

1  SF-PB 060623  +12/-7/=81 52.50%   52.5/100
2  Eman 8.91     +7/-12/=81 47.50%   47.5/100

Note: Eman via Rulleeeee.exp / Experience MultiPV ON
----------------------------------------------------
2nd Test Results: +17 Elo (in favour for SF-PB)
                   
1  SF-PB 060623  +16/-11/=73 52.50%   52.5/100
2  Eman 8.91     +11/-16/=73 47.50%   47.5/100

Note: No Pre Learned Exp / Experience MultiPV ON
----------------------------------------------------

3rd Test Results: +14 Elo (in favor for Eman)
                                   
1   Eman 8.91     +14/-10/=76 52.00%   52.0/100
2   SF-PB 060623  +10/-14/=76 48.00%   48.0/100

Note: Pre Learned Exp of 2nd test/Exp. MultiPV OFF 
----------------------------------------------------
4rd Test Results: +7 Elo (in favor for Eman)
                                                            
1 Eman 8.91      +14/-12/=74 51.00%   51.0/100
2 SF-PB 060623   +12/-14/=74 49.00%   49.0/100

Note: Pre Learned Exp of 2nd+3rd test/Exp. MultiPV ON
----------------------------------------------------

As final words,
Sure for a better conclusion we need more games...
But anyhow it's clear that Eman's learning is quite
Effective even via concurrent (parallel matches..)
So this is wrong who claims that Eman is buggy or not
Able to learn even via parallel/concurrent matches!

On other hand,
Experience MultiPV ON is performed as best (than all)
As 2nd best performance, we noticed comes in 4rd test!
So..once more we noticed that Eman's learning is valid
Even under SCCT conditions (even via parallel matches)

What I can say more about Eman 8.91 engine:
Via pre learned exp file ( sure learned on same openings):
In next re-run: Eman is performing much better for sure!

That means also that...
There is a possibility to see Exp Champs or higher ranks at:
https://sites.google.com/site/computers ... t-ben-cs-1

Hope helps...

Note: very soon all games will be uploaded...

Greetings

[Sedat Canbaz]

As I promised,

GAMES:
https://mega.nz/file/20h0xaZC#osOID5DqV ... 1TsPkdsApU

[Sedat Canbaz]

EDIT:
Experience MultiPV OFF is performed as best (than all)

[IbaiBuR]

UPDATE

1st of all, here are the conditions:
Ryzen7 4800H, CuteChess, 1 Core, Ponder OFF, 1m+1s, 128 MB Hash, 4-MEN
As book, Unique v50 opening suite is used for all and played as twice!

Code: Select all

1st Test Results: +17 Elo (in favor for SF-PB)

1  SF-PB 060623  +12/-7/=81 52.50%   52.5/100
2  Eman 8.91     +7/-12/=81 47.50%   47.5/100

Note: Eman via Rulleeeee.exp / Experience MultiPV ON
----------------------------------------------------
2nd Test Results: +17 Elo (in favour for SF-PB)
                   
1  SF-PB 060623  +16/-11/=73 52.50%   52.5/100
2  Eman 8.91     +11/-16/=73 47.50%   47.5/100

Note: No Pre Learned Exp / Experience MultiPV ON
----------------------------------------------------

3rd Test Results: +14 Elo (in favor for Eman)
                                   
1   Eman 8.91     +14/-10/=76 52.00%   52.0/100
2   SF-PB 060623  +10/-14/=76 48.00%   48.0/100

Note: Pre Learned Exp of 2nd test/Exp. MultiPV OFF 
----------------------------------------------------
4rd Test Results: +7 Elo (in favor for Eman)
                                                            
1 Eman 8.91      +14/-12/=74 51.00%   51.0/100
2 SF-PB 060623   +12/-14/=74 49.00%   49.0/100

Note: Pre Learned Exp of 2nd+3rd test/Exp. MultiPV ON
----------------------------------------------------

As final words,
Sure for a better conclusion we need more games...
But anyhow it's clear that Eman's learning is quite
Effective even via concurrent (parallel matches..)
So this is wrong who claims that Eman is buggy or not
Able to learn even via parallel/concurrent matches!

On other hand,
Experience MultiPV ON is performed as best (than all)
As 2nd best performance, we noticed comes in 4rd test!
So..once more we noticed that Eman's learning is valid
Even under SCCT conditions (even via parallel matches)

What I can say more about Eman 8.91 engine:
Via pre learned exp file ( sure learned on same openings):
In next re-run: Eman is performing much better for sure!

That means also that...
There is a possibility to see Exp Champs or higher ranks at:
https://sites.google.com/site/computers ... t-ben-cs-1

Hope helps...

Note: very soon all games will be uploaded...

Greetings

Hello, just some friendly comments about this, with the aim to help, please dont misunderstand the intention:

If you think that Eman "learned" something after a few 100 game matches, with parallel concurrent matches, and different openings you are quite wrong, believe me...

Eman usually needs 500 games or more per single opening line to learn something and this is something that even the Eman's author said, it is not something I am inventing.

Eman may have saved some of the moves played in the experience file, but probably not all of them because of the parallel matches!

Anyways, thanks a lot for the test and the games. I hope you see my comment in a friendly, constructive way, because this is they way it is intended

Regards,

Ibai

[Sedat Canbaz]

Hello, just some friendly comments about this, with the aim to help, please dont misunderstand the intention:

If you think that Eman "learned" something after a few 100 game matches, with parallel concurrent matches, and different openings you are quite wrong, believe me...

Eman usually needs 500 games or more per single opening line to learn something and this is something that even the Eman's author said, it is not something I am inventing.

Eman may have saved some of the moves played in the experience file, but probably not all of them because of the parallel matches!

Anyways, thanks a lot for the test and the games. I hope you see my comment in a friendly, constructive way, because this is they way it is intended

Regards,

Ibai

Well,
I never said that 400 games is enough...

But what we noticed,
1 concurrent ( parallel) does not make any sense, even weaker!
Sure I mean if Eman is not trained vs/in same opening positions...

What we noticed also that,
SCCT is valid...

The proof is proof...

That's why since 1997,
I prefer to relay on results than comments...

On other hand, that's true that,
Many more trainings are needed.. sure with same openings, then
Very likely not only Eman or I mean any other Top engine, which is
Capable to learn properly.. will be performed better...!!

In other words, this is nothing new...
This happens in every area...with learning...
Including books too... if I start to create a very strong book,
I will run and test many games I mean in various openings..
So my new created book will be performed much better... !)
It all depends on the techniques...on the conditions etc.

But this is also true that,
A very strong exp may be Champ / overcome some strong books as well...
But before, a very serious testings required... sure that's all from me )
In other words, the turn is now in the hands of Exp authors.. !)

Greetings

[Sedat Canbaz]

And for anyone missed,

More testings, but via 64 parallel matches

And for those who say 400 games is a small number..

I suggest to check to this tour..where 2200 games (per player)
https://sites.google.com/site/computers ... t-nn-cs-32

And the leading players are engines with learning capacity..!!

If STILL not enough proof...just let me know please
I may share more...

Hope helps...


Greetings

[IbaiBuR]

And for those who say 400 games is a small number..

I suggest to check to this tour..where 2200 games (per player)
https://sites.google.com/site/computers ... t-nn-cs-32

Just checked, thats a little bit better, tho, still quite far from the 500 games per opening recommendation, but yeah, definetely better, thanks for sharing

Btw, I was thinking a bit... Do you have / recommend any small testsuite for testing engines (not more than 50 opening lines)?

I may train a modded version of Stockfish + learning on it and I'll report the results of training and testing against other engines (big number, as you said)

This will take a big amount of time tho...

Greetings and thanks in advance

[Sedat Canbaz]

And for those who say 400 games is a small number..

I suggest to check to this tour..where 2200 games (per player)
https://sites.google.com/site/computers ... t-nn-cs-32

Just checked, thats a little bit better, tho, still quite far from the 500 games per opening recommendation, but yeah, definetely better, thanks for sharing

Btw, I was thinking a bit... Do you have / recommend any small testsuite for testing engines (not more than 50 opening lines)?

I may train a modded version of Stockfish + learning on it and I'll report the results of training and testing against other engines (big number, as you said)

This will take a big amount of time tho...

Greetings and thanks in advance

You are welcome )

Now I feel much better..I mean all in friendly way )

Agreed...500+ games (per opening) would be much better!
But in my guess, even several games (per same opening) can be
Ranked at higher places..than the ones which never tested..

Why I say like this (if check latest current test-400 games),
We see that even via small number of learned games
Eman's performance is increased...and I strongly believe that
Pre-learned Eman. exp file is helped a lot...

Btw, BIG thanks to you, for your idea: Experience MultiPV OFF
And it seems that Eman 8.91 is stronger under these settings...

On other hand,
Nowadays I have no much free time for chess but later,
Maybe in September, I may test RapTora 2.2 as without learning:
https://sites.google.com/site/computers ... t-nn-cs-33

Then we will have chances to compare, e.g
RapTora 2.2 NON-Learning will be Champ or not..
And frankly in my estimation: no much chances!

About your book question,
You can try to test the full version of Unique - 300 openings:
https://sites.google.com/site/computers ... test-suite

Greetings )