Zadar, Zadar, Zadar

Tiger Lilov's video about the Zadar Open 2012 motivated me to look at the relevant data in some detail. The issue involves a non-grandmaster player (NGM) who played against grandmasters (GM) with such surprising strength that many people wondered whether he had an unfair advantage, possibly in the way of computer assistance.

I took a sample of moves from games 58, 75, 87, 100, 117, and 136 in which the NGM played GM. The NGM won three of these games and lost one. The other two ended in draw by mutual agreement. The question raised by Tiger concerns the use of computer generated moves - specifically moves recommended by Houdini 3 Pro. Since I don't have access to Houdini 3 Pro, I used Houdini 3 and collected data from the tournament PGN file provided by chessbase.com. The sample data are summarized below.

To exclude rote opening moves, I tallied the data from move 10 in each game. I counted 321 moves in these six games - 160 by the NGM and 161 by the GM players. Of the 321 moves I identified 195 as "computer moves" because each move coincided exactly with the move recommended by Houdini at a depth of 19 to 20 half moves. The other 126 moves are likewise identified as "non computer moves". So we find that almost 2/3 of the moves in all six matches, 60.7% to be exact, are coincident with Houdini recommendations.

Of the 160 moves made by the NGM 115, or 71.9% , coincided with Houdini moves while the 161 moves made by GM contain only 80 Houdini moves at the considerably lower rate of 49.7%. We can carry out a statistical test to determine whether the observed difference in the rate of computer moves could have happened by chance in the small sample of moves taken.

The Chi-Square test shows that if the GM and the NGM had the same propensity to make Houdini moves, the probability of realizing the observed difference in this sample is 0.3%. In non medical statistics, we normally consider a probability of less than 5% to be something we would not expect to happen. So in this case, the sample data show that there is in fact a real difference in the computer move rate between the NGM and the GM. In particular, we conclude that the computer move rate of the NGM is higher than that of the GM.

The additional question raised by Tiger is whether the NGM's computer move rate is unusually high. To answer that question I took a sample of moves from the 2012 world championship between Anand and Gelfand counting from move number 10 to move 29 or the end of the game if the game was shorter than 29 moves. Of 398 moves in my sample, 280 or 70.4% were computer moves with Anand slightly higher at 73.5% and Gelfand somewhat lower at 67.2%. In this context, we can see that a computer move rate of 71.9% is actually as unusually high as Tiger had suspected as it is comparable to the rate of the reigning world champion.

Incidentally, the positive relationship between player rating and rate of computer moves shows that computer moves are not an aberration but consistent with good chess and the high rate of Houdini moves by the NGM may imply only that he was simply playing well.

Another question raised is the the occurrence of long sequences of consecutive Houdini moves by the NGM. The longest sequence of Houdini moves by the NGM in my sample is 14 moves. There are two of these sequences - one starts from move 15 in game 75 and the other from move 13 in game 87. The longest sequence of such moves by the GM is 4 moves and these occur in all the games. These sequences often occur because of a series of exchanges. I found 40 instances of consecutive Houdini moves in the sample, 20 by the NGM and 20 by the GM. The average length of these sequences for the NGM is 5.15 moves. The average length for the GM is almost half that at 2.75.

The t-statistic for this difference is more than 12 which implies that the observed difference could not have occured in our sample if there were not a real difference in the propensity to make sequences of Houdini moves. We conclude that the data provide sufficient evidence that the NGM has a higher propensity to make sequences of Houdini moves - just as Tiger had suspected.

By way of comparison, the average length of Houdini move sequences in the Anand-Gelfand 2012 world championship match is 4.17 (Anand 4.19, Gelfand 4.15). The NGM's average sequence length of 5.15 exceeds the length achieved in the world championship by about one move.

The data seem to show that the NGM tends to play a lot like Houdini. As to the question of how the NGM developed this ability, we really have no actual data to go on and thus all we can do is speculate. It could be that the NGM has played so many games with Houdini that he developed a Houdini-like pattern in his moves. It is also speculated that the NGM used devious high-tech methods to access Houdini during the match and that therefore he enjoyed an unfair advantage over the GM.

Yet, although there is insufficient evidence to conclude that the NGM used any of the high tech cheating devices described by Tiger, the important message in Tiger's videos may be that these devices exist and they are not very expensive; and that alone should be sufficient reason to install the cell phone jammers in tournament halls that Tiger has proposed.

Cha-am Jamal

Thailand

Comments

seems ivanov has been banned from playing for 4 months by bulgarian chess federation whilst they use some scientific means to check if he did cheat. the wheels of chess justice turn slowly but is on the move now it seems :).

hello frankBgambit, what an excellent piece of work you have done. thank you for sharing your findings. i will go over your report more carefully tomorrow and write to you with some hopefully helpful feedback. i would like to learn more about the ipr. let's stay in touch.

with a performance IPR of only 1818 with a poor top move rate of 19% to me this is overwhelming evidense that this one factor of stopping the broadcast is main effect on outcome of this game.

of all the games in history that i have checked this performace for tornement is highest i have ever found for any player ever. combined with the dramatic change to the performance of Ivanov once the broadcasting of moves was stopped.

As for the consecutive houdini moves made by Ivanov and his opponents:

The PGN downloaded from FIDE has the following players at the numbers you listed (with the result appended as W-white wins, D-draw, B-black wins):

58 [White "Kuljasevic, Davorin"] [Black "Ivanov, Borislav"] D

75 [White "Ivanov, Borislav"] [Black "Zelcic, Robert"] W

87 [White "Kozul, Zdenko"] [Black "Ivanov, Borislav"] B

100 [White "Petrusic, Toni"] [Black "Videnova, Iva"] W

117 [White "Biliskov, Vjekoslav"] [Black "Paljusaj, Edmond"] D

136 [White "Veleski, Robert"] [Black "Paljusaj, Edmond"] W

The above does not correspond to what you described in your post and for a while got me royally confused as to what you were doing. But I guess that you meant games 103, 120 and 139 in the FIDE PGN (which happens to be the same as the one on the tournament site).

103 [White "Sumets, Andrey"] [Black "Ivanov, Borislav"] D

120 [White "Ivanov, Borislav"] [Black "Predojevic, Borki"] B

139 [White "Saric, Ivan"] [Black "Ivanov, Borislav"] B

Overall, good work, and another independent confirmation of quite suspicious play on the part of "NGM"; analyzing all 9 of his games would be more convincing, though.

news from chessbase.com " The latest incident involves the former mayor of an Italian town who has been banned for allegedly using a hidden micro camera and earpiece to receive electronic assistance in tournament games. "

the longest streak of houdini moves by NGM in my sample was 14 moves long. curiously, the longest streak of houdini moves in the sample taken from the 2012 world championship was also 14 moves long (by anand).

Keep in mind "changes at different depths" must take into account that Ivanov did not have forever on each turn. Clearly in the marathon games where he had to move faster, he was getting faster moves from the engine. I'm quite certain at no point did he take more than 10 minutes on any given move where he was cheating. As such, we can't simply wait unil the engine changes its mind and say "Ah see! He didn't match the engine!". That's a false premise to argue from. When you review the games, It's just a little more than pure coincidence that a quad-core PC with Houdini 3 likes almost all of Ivanov's moves within 5 minutes given per turn (most of which instantly match).

I think the key to this sort of argument is doing what Jamalov has done. You have to select a given depth, and then compare the results with different players. Its not enough to say someone played the 'top move', because that changes at different depths. It would be interesting for someone to run all the world championship matches through such a proceess, creating a fairly clear baseline.

"There was only one game there where he didn't play like Houdini at all (obviously wasn't cheating in that game), and got crushed "

------------------------------------

yes, i do recall that game. of course your analysis has more depth because you are a better chess player and also you have h3pro. i was just trying to move the data from selective anecdotes to random samples to see if we come to the same conclusions and i think that we do.

I do have access to Houdini 3 pro, and my results when going over the games were quite blatant.

In the first game where he beat a 2400+, virtually every single move he played beyond the first 6 opening book moves were a perfect match for Houdini. All but one move matched the #1 choice, and the remaining move matched #2.

In the 117-move game against a GM where he blundered at the very end and lost in time trouble, 87 out of the 100 non book moves were deemed to be flawless by Houdini. At one point in the game at around move 64 to 71, he lost contact with the computer and started playing bad moves to the point of technically being lost. Then on move 72, he started cheating again and pulled back to equal, only to later lose at the very end on move 115 with a silly blunder.

These are just two examples of the incredible Houdini-like play by Ivanov at Zadar. There was only one game there where he didn't play like Houdini at all (obviously wasn't cheating in that game), and got crushed like a clueless patzer in it.

Help us finish translating:

We are working hard to make Chess.com available in over 70 languages. Check back over the year as we develop the technology to add more, and we will try our best to notify you when your language is ready for translating!