2018-09-09

First half of CCCC ended. 23/46 rounds. Leela is 4th! Results and conclusions....


First half of the CCCC tournament has ended.
It's a 46 rounds tournament and so 23 rounds have been played. Time control is 15'+5" and engines will play all against all twice, one with black and one with white, while pondering is on and no opening books are used and 6 pieces tablebases are being used.
Top 8 engines promote to next round.


Till now there are 4 distinctive groups that have been formed. The top 3 that are consisted of the known "big 3" of computer Chess this moment and for the last 4-5 years, Stockfish, Komodo and Houdini, the other group the engines that fight for the top 8, Leela, Shredder are the leaders of this group that show they might have the advantage and the other Fire, Booot, Ethereal and Andscacs.
There is a group 8-9 engines that seems to just be in the middle without having any prospects of going to top 8 and there is the low end of the tournament that does not do that good.
There are 4 engines that till now haven't lost any game so far. Stockfish, Houdini, Komodo and Leela.


The standings after round 23, finishing the first half of the tournament.


Player Score (SB) H K S L S F B E A F X V P T G F A L N W I S C N +/-/=
1: Houdini 6.03 19.0/23 198.50 X = = = = 1 = = 1 1 1 = 1 1 1 1 1 1 1 = 1 1 1 1(+15 -0 =8)
2: Komodo 2118.00 18.5/23 195.25 = X = = 1 1 1 = = = = 1 1 1 = 1 = 1 1 1 1 1 1 1(+14 -0 =9)
3: Stockfish 220818 18.5/23 193.00 = = X = = = 1 1 = 1 = 1 1 1 1 1 = = 1 1 1 1 1 1(+14 -0 =9)
4: Lc0 17.11089 16.5/23 179.25 = = = X = = = 1 = 1 1 1 = 1 1 = 1 1 = = 1 = = 1(+10 -0 =13)
5: Shredder 13 16.0/23 164.25 = 0 = = X 1 = = 1 = = = = 1 1 = 1 = 1 = 1 1 1 1(+10 -1 =12)
6: Fire 7.1 15.5/23 151.50 0 0 = = 0 X = 1 = 1 = = 1 1 1 = 1 1 1 = = 1 1 1(+11 -3 =9)
7: Booot 6.3.1 14.5/23 140.00 = 0 0 = = = X = = = = = = 1 = = = 1 1 1 1 1 1 1(+8 -2 =13)
8: Ethereal 10.88 14.0/23 138.50 = = 0 0 = 0 = X 1 = 1 = = 0 1 = 1 1 1 = 1 = 1 1(+9 -4 =10)
9: Andscacs 0.94 13.5/23 128.50 0 = = = 0 = = 0 X 0 0 1 = = 1 = 1 1 1 1 = 1 1 1 (+9 -5 =9)
10: Fritz 16.10 12.0/23 116.75 0 = 0 0 = 0 = = 1 X = 1 1 0 = = = = = 1 = = 1 1(+6 -5 =12)
11: Xiphos 0.3.17 11.5/23 114.50 0 = = 0 = = = 0 1 = X 0 = = = = = = 1 = 0 1 1 1(+5 -5 =13)
12: Vajolet 2.6 11.0/23 104.50 = 0 0 0 = = = = 0 0 1 X = = = = = 1 0 = = 1 1 1(+5 -6 =12)
13: Pedone 1.8 11.0/23 101.00 0 0 0 = = 0 = = = 0 = = X = = 1 = = = = = 1 1 1(+4 -5 =14)
14: Texel 1.07 11.0/23 96.00 0 0 0 0 0 0 0 1 = 1 = = = X = = = = 1 = = 1 1 1(+6 -7 =10)
15: Gull 3.syz 10.0/23 88.50 0 = 0 0 0 0 = 0 0 = = = = = X = = = = 1 1 1 1 =(+4 -7 =12)
16: Fizbo 1.9 9.5 / 23 96.75 0 0 0 = = = = = = = = = 0 = = X 0 = = = = = 1 =(+1 -5 =17)
17: Arasan CCCC-2018 9.5 / 23 89.75 0 = = 0 0 0 = 0 0 = = = = = = 1 X = 0 1 0 = 1 1(+4 -8 =11)
18: Laser 1.6 9.0 / 23 78.00 0 0 = 0 = 0 0 0 0 = = 0 = = = = = X 1 = 0 1 1 1(+4 -9 =10)
19: Nemorino 5.00 8.5 / 23 71.25 0 0 0 = 0 0 0 0 0 = 0 1 = 0 = = 1 0 X = 1 1 = 1(+5 -11 =7)
20: Wasp 3.25 8.0 / 23 84.00 = 0 0 = = = 0 = 0 0 = = = = 0 = 0 = = X 0 1 = =(+1 -8 =14)
21: Ivanhoe 999946h 8.0 / 23 81.50 0 0 0 0 0 = 0 0 = = 1 = = = 0 = 1 1 0 1 X 0 = 0(+4 -11 =8)
22: Senpai 2.0 4.5 / 23 42.00 0 0 0 = 0 0 0 = 0 = 0 0 0 0 0 = = 0 0 0 1 X = =(+1 -15 =7)
23: Crafty 25.2 3.5 / 23 25.75 0 0 0 = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 = = = = X 1(+1 -17 =5)
24: Nirvana 2.4 3.0 / 23 24.00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 = = 0 0 0 = 1 = 0 X(+1 -18 =4)



And the ratings:
Rank Name               Elo    +    - games score oppo. draws 
   1 Houdini 6.03       495  130  113    23   83%   265   35% 
   2 Stockfish 220818   480  125  111    23   80%   266   39% 
   3 Komodo 2118.00     478  126  111    23   80%   266   39% 
   4 Shredder 13        392  111  104    23   70%   270   52% 
   5 Lc0 17.11089       389  109  103    23   70%   270   61% 
   6 Fire 7.1           385  114  107    23   67%   270   39% 
   7 Booot 6.3.1        351  106  102    23   63%   272   57% 
   8 Ethereal 10.88     343  111  107    23   61%   272   43% 
   9 Andscacs 0.94      339  111  108    23   59%   272   39% 
  10 Fritz 16.10        286  106  106    23   52%   275   52% 
  11 Xiphos 0.3.17      285  104  104    23   50%   275   57% 
  12 Vajolet 2.6        268  104  105    23   48%   275   52% 
  13 Pedone 1.8         265  101  102    23   48%   275   61% 
  14 Texel 1.07         254  106  107    23   48%   276   43% 
  15 Gull 3.syz         252  103  105    23   46%   276   57% 
  16 Arasan CCCC-2018   229  106  109    23   41%   277   48% 
  17 Fizbo 1.9          227   99  101    23   41%   277   74% 
  18 Laser 1.6          216  106  110    23   39%   278   43% 
  19 Wasp 3.25          193  103  107    23   35%   279   61% 
  20 Nemorino 5.00      190  110  116    23   37%   279   30% 
  21 Ivanhoe 999946h    180  110  118    23   35%   279   35% 
  22 Senpai 2.0          74  115  132    23   20%   284   30% 
  23 Crafty 25.2         27  120  144    23   15%   286   22% 
  24 Nirvana 2.4          0  126  156    23   13%   287   17% 

**This is with Lc0 - Gull game counted as draw. Since this is a rating list the real result should be counted, as the CCCC result by their rules because Gull crashed, was 1-0 in favor of Lc0.



•Houdini has an exceptional tournament thus far, winning against almost every engine below top 8. In this short time control Houdini is to be expected to continue doing great.
•Komodo is very solid also and the only engine that has won against an engine of the top 5 with its win against Shredder. Komodo played some wonderful games. This short time control doesn't fit perfectly to Komodo, but Komodo is super strong at every time control of course.
•Stockfish as expected is very high in the ranking but being the strongest engine in the world this time and being 2nd can be considered underperformance but of course too small sample size with just 23 games to draw any conclusions. Stockfish just plays unbelievably solid with just a bit waiting of its opponents to play even the smallest inaccuracies that is expert in taking advantage of them.
Leela being 4th and undefeated is a nice performance so far for an engine that was born before 6 months. It has shown in many games a different approach from traditional games and understanding of positions. It has played some very spectacular games. She has played also some of the most bizarre games. She still suffers a bit from not creating complications against weaker opponents so she draws a lot against weaker opponents.
•Shredder started perfectly but its performance has fallen a bit. Still plays aggressively but also very solid. Had just one defeat so far from Komodo.
•Fire is very solid as expected, very good at tactics and doing exceptionally against weaker engines. It has 3 losses though from the top 5.
•Booot is having a surprisingly good tournament so far and this is because it is killing the low end engines! It has a lot of draws though with middle group of engines but lost only 2 times from the top 5.
•Ethereal started badly, very badly but as expected it climbed to top 8 and will probably stay there. IT seems unstable though and while it has some good moments where it shows its true potential, it has some moments where it plays dubiously.
•Andscacs is also killing the weakest engines but has many problems against the first 11 having lost already 5 times. It's expected to be on top 8 replacing Booot, but perhaps Booot is going to hold.

•Fritz is solid enough but is not having enough wins against weaker engines to fight for top 8. Probably will be 9-10 not having a chance to try for top 8.
•Xiphos is relatively solid having 13 draws so far but it's strength is not for top 8.
•Vajolet is doing ok for the lowest 12 of the tournament but is losing a lot when facing the highest 12. Even though she drew against Houdini.
•Pedone plays some clever Chess, has 14 draws so far but suffers a lot against the top 10 engines.
•Texel plays an active Chess and is relativelly good at tactics but stands no chance for the top 8 where it lost most of the games aganist them.
•Gull is not winning enough even though it is rather solid and its place is expected.
•Fizbo is underperforming here so far with all these draws. It has 1 win 5 losses and 17 draws! It is a better engines than this.

•Arasan is having a logical performance of its strength so far. It got also some nice results like the 2 draws against Stockfish and Komodo.
•Laser is severely underperforming here. This is very strange so far.
•Nemorino is playing some risky Chess but this does not mean it's for its own good. Its place is to be expected.
•Wasp seems very solid most of the times early simplificating the position in many games. It got 3 draws against Houdini, Leela and Shredder.
•Ivanhoe seems very unstable and dubious playing many times to be able to be higher in the ranking.
•Senpai is one of the weakest engines of the tournament so it is to be expected this low place.
•Crafty, the good old Crafty is trying to compete with the new generation but obviously it is too much for it. It is getting outplayed in most games. Took a draw from Leela though (even in a lost position where Leela didn't manage to find the win).
•Nirvana for some reason is severely underperforming. It plays very weak Chess and this is strange since it's usually much stronger than this.


45% of the games are draws. A rather good result of decided games.
A whole lot of games have French defense! Engines for some reason prefer 1...e6 over "normal" 1...e5 after 1.e4.


                 Score      Game length             Frequency       
                          1-0    =-=    0-1    1-0     =-=     0-1  
 All games       56.1%     79     96     78   33.6%   44.9%   21.3% 


    Move      Frequency    Score  Draw 
 1: e4        152: 55.0%   52.9%  44%  
 2: d4        102: 36.9%   61.2%  46%  
 3: Nf3        22:  7.9%   54.5%  45%  












8 comments:

  1. I wonder, was the author of this post aware that some engines are not playing at full capacity due to pre-tournament decisions regarding stability during testing? For example, Fizbo's underperformance may be due to the fact that it is only version 1.9, while the latest version 2.0 has been out for a while and is considered much stronger, but was disqualified because apparently it crashed on the CCCC hardware a lot during testing. A similar story with Gull, I believe.

    ReplyDelete
    Replies
    1. Sure these are known, but still Fizbo 1.9 should be a bit stronger. Especially Laser and Nirvana.

      Delete
  2. Thanks for the stats. Indeed, lots of French games for some reason!

    ReplyDelete
  3. Yes, lots of French Defense. Same thing happened during early seasons of TCEC when there were no books used. Almost every game was French. This is the reason why they introduced short book moves in the opening.

    ReplyDelete
  4. Adverbs and adjectives differ. You don't appear to know this.

    ReplyDelete
    Replies
    1. Well being a non native English speaker it's a wonder i'm even talking to you.

      Delete
    2. The author took the time to type up this detailed blog post and that's your comment?

      https://www.youtube.com/watch?v=VlSkPA60ujQ

      Delete
  5. The author took the time to type up this detailed blog post and that's your comment?

    https://www.youtube.com/watch?v=VlSkPA60ujQ

    ReplyDelete