2018-09-28

TCEC Cup in the next days!



 The TCEC(Top Chess Engine Championship) tournament is currently running with engines playing in the premier division and this division will finish in a few days. But this time, after premier division end, there would not be an immediate starting of the superfinal between the top 2 placed engines in premier, but a new tournament is going to happen that Chessdom has announced, the TCEC CUP, with all engines of TCEC but in a different format, that of knockout games, one that brings in mind the traditional Cup competitions of football.
(TCEC superfinal will he held after the TCEC Cup of course)

So the 32 engines of season 13 of TCEC will participate and the pairs will be according to their places that had in the current 13th season. That means 1st engine in premier will play against 8th placed engine in 4th division, 2nd premier engines will play against 7th placed engines in 4th division, etc....
Each pair will will play 8 games, 4 with black and 4 with white with 4 predefined positions with reversed colors.
After 8 games are complete the engine with higher points will promote to next round.
In case of a tie after 8 games, 2 more games(1 with black and 1 with white for each engines) will be played to determine the winner(the one who will score more points). In case of a tie another 2 games will be played and so on, until an engine wins the 2-game tiebreak.

Obviously with 32 engines there would be a round of 32, then a round of 16, then the quarter-finals, then semi-finals and at the end the final.


All games will be played with time control of 30 minutes plus 10 second per move increment with pondering off.
Hardware for traditional engines will be 43 cores(2 x Intel Xeon E5 2699) while for the 2 neural net engines(Leela and DeusX) it would be 2x Tesla V100 GPUs provided by Google.


These are the current pairings and a probable, according to how Football and Basketball competitions are being run, format between the pairings. But only the round of 32 pairings holds as per the TCEC announcement, since they may choose different match ups for the next rounds.

(click to zoom image)



So Leela will have to face Laser in the TCEC Cup in 8 games. The odds should be with her and she should promote as she is stronger than Laser but we will see.....



Knowing that premier division is running and Leela is not participating as she got stuck in 3rd division due to the known issues with overheating problems because of the hardware, TCEC had been running various bonus games with most time Leela participating.
 And in these bonus games Leela has so far a fantastic performance against the mighty Stockfish development version(dev) with 43 cores or even more!

So Leela test 10 nets running on good GPUs(P100s or V100s), on 20'+5" time control has till now in TCEC bonus games against 43 cores or better Stockfish dev a result of:
1 win, 7 losses and 42 draws in 50 games.
That means a -42 ±38 Elo performance and that Stockfish has won 14 % of the games against Leela.

Right now Stockfish on premier division has 21 wins, 1 loss and 25 draws in 47 games.
That means a +158 ±67 Elo average performance and that Stockfish has won 45 % of its games against the other premier engines.


So one has to wonder. What super resistance Leela puts against Stockfish compared to the premier division engines?
Is Leela so much better than average premier division engines?
The answer is not easy.
Since Leela is peculiar.
She, in a way, adapts to her opponent so a great result against Stockfish doesn't necessarily mean she will have a equivalent, Elo-wise, result against a weaker engine, or even against a much weaker engine. Transitivity of Elo doesn't really work that well with Leela.

 This must be due to Leela is still having many blind spots resulting in blunders from time to time mostly in endgame and mainly because of her ineffectiveness in many occasions in the endgame, not being able to convert a better or even winning position.
 Blunders with equal distribution occurrence against weak or strong engines, cost her much more Elo against weaker opposition. Also not being able to convert in endgame her advantage against weaker engines costs her again much more Elo than when playing against stronger opponents. It's obvious from her current playing that in opening and middlegame she can hold her own against Stockfish and she is probably even be better(except tactical shots of course that Stockfish is out of this world), but when it comes to endgame she loses a whole lot of her strength, with many misevaluations and incorrect play compared to top 20 engines. This is mainly because her puct search is not able to compete against AlphaBeta search that is extremely efficient to resolve positions especially with few pieces like we have in endgames. But this happens of course also due to not optimal evaluation in many endgames. So more training is needed to improve all these and even better training techniques that are currently tried in test 30.
For more about that curiosity of Leela one can read HERE!


Leela's only win against Stockfish dev 43 cores in TCEC bonus games. Stockfish was blind to the g4->g5->g6 march of Pawn.
[Event "?"] [Site "?"] [Date "2018.08.29"] [Round "7.1"] [White "Lc0 v17 11089 (4xP100, 10CPU)."] [Black "Stockfish dev (43 cores)"] [Result "1-0"] [ECO "B90"] [WhiteElo "3247"] [BlackElo "3519"] [PlyCount "52"] [EventDate "2018.??.??"] 1. e4 c5 2. Nf3 d6 {End of predefined opening position. Now engines had to think for themselves.} 3. Nc3 Nf6 4. d4 cxd4 5. Nxd4 a6 6. Bd3 g6 7. h3 Bg7 8. Be3 O-O 9. O-O e6 {Till now it was all theory. Now Stockfish chooses a rarely played line. Statistics on human games, favored black in the opening theory till now.} 10. Qd2 b6 {[%cal Gc8b7]} 11. f4 $1 {Leela aggressive as always.} Bb7 12. f5 gxf5 13. Bh6 fxe4 14. Qg5 $1 {[%csl Rg7]} Nh5 {The only move for black that after Qxh5 regains the Piece with tempo(check). So seemingly black except the fact that its King is a little vurnerable seems to do fine. But not quite the truth.} 15. Qxh5 {[%csl Rg1][%cal Rg7d4]} Bxd4+ {[%csl Rg1]} 16. Kh1 Bxc3 {Stockfish gave +0.58 for white, that means it thought it controlled the situation.} 17. bxc3 f5 18. Bc4 d5 19. Be2 $3 {[%csl Rf8] Stockfish expected Bxf8 and after this move it went to 0.00 evaluation thinking he is now fine. But missed completely Leela's powerful continuation so it tried to reorganize its defense saving its Rook also from the Bishop.} Rf7 $2 {This loses.} (19... Nd7 20. g4 f4 21. g5 $4 {This doesn't work now.} (21. Bxf8 Qxf8 22. Qg5+ Qg7 23. Qxg7+ Kxg7 24. Rxf4 {White is better but there is much play left and black can hope.}) 21... f3 $19) 20. g4 {The beginning of an unstoppable attack.} f4 { [%eval 0,0] Stockfish still thinks this is equal giving 0.00 evaluation} 21. g5 $1 {This is the move Stockfish missed!} Qe7 (21... f3 22. g6 Rf5 23. gxh7+ Kxh7 24. Bg5+ $18) 22. g6 Rf5 {Stockfish realized its position giving +6.02 for white. Too late for any defense, game is over.} 23. gxh7+ Kh8 24. Qg4 Qxh7 25. Qh4 Nd7 26. Rg1 Rf7 {Stockfish gives +7.88 and game is over.} (26... Rf7 27. Rg5 {[%cal Ga1g1]} Nf6 28. Rag1 Rg8 29. Rxg8+ Nxg8 30. Rxg8+ Qxg8 (30... Kxg8 31. Qd8+ Rf8 32. Qxf8#) 31. Bxf4+ Qh7 32. Be5+ Rg7 33. Qd8+ Qg8 34. Qf6 {Black is paralyzed. } Bc8 35. Qh6+ Qh7 36. Bxg7+ Kg8 37. Qxh7+ Kxh7 $18) 1-0



15 comments:

  1. Which net will be used in TCEC cup?

    ReplyDelete
    Replies
    1. I believe 11250 running v18 w/ full TB support

      That's what they've been testing in TCEC bonus games so almost certain what will be used

      Delete
    2. 11248 it is https://discordapp.com/channels/425419482568196106/445928688115122176

      Delete
  2. This is great news, along with the CCCC Komodo-Leela Match. I think that SF does not,in general 'understand' Leela's middle-game play and very soon will be loosing more and more games to Leela (t20/t30, EGTB etc.).

    ReplyDelete
  3. Very excited, loving the blogs!!

    ReplyDelete
  4. Given that one of the mods at TCEC forbids any Leela chat (with timeout punishment), Leela fans at the Cup may get a frosty reception.

    ReplyDelete
    Replies
    1. I can't understand why you'd tell this lie, but I've been following TCEC for quite some time and no one has ever been timed out or banned (or threatened with a timeout or ban) for discussing Leela, or anything of the sort. People talk (and argue) about Leela (and many other topics) in TCEC chat all the time without issue.

      CCCC, on the other hand, does instantly and permanently ban people in chat for anything resembling even the mildest criticism, which I've personally witnessed several times. If you're not just intentionally spreading disinformation, perhaps you've confused the two tournaments?

      Delete
    2. I have zero motivation for "telling a lie", and, Nickolas, I would like your apology for that assertion. I was told by kanchess that "TCEC was not a Leela forum" and that "Discord is for that". I was then given a timeout. This is outrageous behaviour in my book. If you disbelieve me, please request the Sunday afternoon (PDT) chat log so that your apology is firmly grounded in fact. Thank you.

      This interchange occurred after I'd requested to post a link to the github guide to Leela training. Since I'd never posted a link at TCEC before, I muffed it up by simply typing a request for a link. I then realised I needed to use the "!link" command and did so. At this point I was handed the timeout and received the messages I've described.

      Delete
    3. You took that singular experience and transformed it into "one of the mods at TCEC forbids any Leela chat (with timeout punishment)". From where I’m sitting, that’s a gross misrepresentation of whatever happened with you. Additionally, it's obviously untrue to anyone that's spent any amount of time observing TCEC chat. People there discuss Leela all the time without issue.

      So yes, I think it’s entirely consistent with reality to simply say that you lied. As far as I can tell, you knowingly made a categorically false claim. It appears to me that you told the lie with the intent to mislead anyone who read your comment, so as to elicit sympathy and predispose uninformed people to sharing your prejudices. It’s a common and understandable motivation, however ignoble it may be.

      Now, because the description of your alleged timeout is inconsistent and lacking relevant details, as far as I can tell it's as likely as not that you were simply automatically purged by Nightbot for posting a link. This is a fully automated and standard anti-spam measure in most Twitch channels with more than a few people, as I'm sure you're aware. I'm sure you're also aware that it doesn't prevent you from further chatting, so it's not a "timeout", even if it's technically implemented as a one-second timeout.

      Let's say that's not the case, and you were formally granted permission to post a link. (Keep in mind, this requires a moderator to type "!permit Username" in response to your !link request. You can't merely type !link and then immediately post a link. Nightbot will purge you.)

      In this perhaps accurate / perhaps hypothetical scenario (again, we can't tell based on the partially contradictory information you provided), the moderator "kanchess" disagreed with the content of your link on the grounds that they don't want technical support discussion in TCEC chat. That’s a reasonable request that should be easy for you to abide by.

      By your own admission kanchess informed you why your link was purged, and told you to refer people to the definitive resource for Leela technical support -- the Leela Discord. That's very sound advice, as it's the first place I'd go if I needed help with Leela.

      I truly don’t see how a truthful, rational observer could interpret such a scenario as “one of the mods at TCEC forbids any Leela chat (with timeout punishment)”.

      Delete
    4. The remarks "TCEC is not a Leela forum" and "Discord is for that" are not remarks which I would characterise as "Leela-friendly". Sorry.

      Delete
    5. You made the specific and inflammatory claim that "one of the mods at TCEC forbids any Leela chat (with timeout punishment)". That is a lie.

      It's also telling that you chose not to elaborate on the missing details in your story. That suggests to me that you were, in fact, merely purged by Nightbot for posting a link, and that kanchess then told you that TCEC chat wasn't a technical support forum in order to explain why he wasn't going to !permit you and allow you to repost the automatically purged link.

      If my supposition is correct, as your conspicuous lack of candor makes it seem, you've ascribed malice to the standard, automated processes of a bot, and used that as the philosophical basis with which to justify your smear campaign.

      Anyway, since you now seem to be shifting the goalposts away from your obvious lie to the claim that TCEC chat isn't "Leela-friendly", I disagree with that characterization as well. I have personally witnessed many productive, civil discussions between fans and non-fans alike.

      In the time since Leela began participating in TCEC, I've never seen a mod purge, timeout, or ban anyone for talking about Leela, or any other reasonable subject. The only purges I've witnessed have been automated. The only timeouts and bans I've witnessed have been for obvious trolls spewing curses, racial epithets, and other nonsense.

      Delete
    6. An 87 second timeout is not a bot timeout. I maintain the position that TCEC is not Leela-friendly. I also demand an apology from you for accusing me of lying.

      Delete
    7. If you don't want to be called a liar, I suggest you stop telling lies.

      Delete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Leela beat Ethereal in the TCEC Cup R2 after 16 games but could have closed out in game 14 as it had a win at position 94w with ply-count zero. Finalgen declares a win in 80 moves (which it assumes after a P-conversion) but Leela made this a 'win or draw' with 106. Ke3?? and a definite draw with 120. b6?? The line I followed reaches a '50-move-rule-safe' 7-man position at 116w.

    ReplyDelete