Zadar, Zadar, Zadar

Tiger Lilov's video about the Zadar Open 2012 motivated me to look at the relevant data in some detail. The issue involves a non-grandmaster player (NGM) who played against grandmasters (GM) with such surprising strength that many people wondered whether he had an unfair advantage, possibly in the way of computer assistance.

I took a sample of moves from games 58, 75, 87, 100, 117, and 136 in which the NGM played GM. The NGM won three of these games and lost one. The other two ended in draw by mutual agreement. The question raised by Tiger concerns the use of computer generated moves - specifically moves recommended by Houdini 3 Pro. Since I don't have access to Houdini 3 Pro, I used Houdini 3 and collected data from the tournament PGN file provided by chessbase.com. The sample data are summarized below.

To exclude rote opening moves, I tallied the data from move 10 in each game. I counted 321 moves in these six games - 160 by the NGM and 161 by the GM players. Of the 321 moves I identified 195 as "computer moves" because each move coincided exactly with the move recommended by Houdini at a depth of 19 to 20 half moves. The other 126 moves are likewise identified as "non computer moves". So we find that almost 2/3 of the moves in all six matches, 60.7% to be exact, are coincident with Houdini recommendations.

Of the 160 moves made by the NGM 115, or 71.9% , coincided with Houdini moves while the 161 moves made by GM contain only 80 Houdini moves at the considerably lower rate of 49.7%. We can carry out a statistical test to determine whether the observed difference in the rate of computer moves could have happened by chance in the small sample of moves taken.

The Chi-Square test shows that if the GM and the NGM had the same propensity to make Houdini moves, the probability of realizing the observed difference in this sample is 0.3%. In non medical statistics, we normally consider a probability of less than 5% to be something we would not expect to happen. So in this case, the sample data show that there is in fact a real difference in the computer move rate between the NGM and the GM. In particular, we conclude that the computer move rate of the NGM is higher than that of the GM.

The additional question raised by Tiger is whether the NGM's computer move rate is unusually high. To answer that question I took a sample of moves from the 2012 world championship between Anand and Gelfand counting from move number 10 to move 29 or the end of the game if the game was shorter than 29 moves. Of 398 moves in my sample, 280 or 70.4% were computer moves with Anand slightly higher at 73.5% and Gelfand somewhat lower at 67.2%. In this context, we can see that a computer move rate of 71.9% is actually as unusually high as Tiger had suspected as it is comparable to the rate of the reigning world champion.

Incidentally, the positive relationship between player rating and rate of computer moves shows that computer moves are not an aberration but consistent with good chess and the high rate of Houdini moves by the NGM may imply only that he was simply playing well.

Another question raised is the the occurrence of long sequences of consecutive Houdini moves by the NGM. The longest sequence of Houdini moves by the NGM in my sample is 14 moves. There are two of these sequences - one starts from move 15 in game 75 and the other from move 13 in game 87. The longest sequence of such moves by the GM is 4 moves and these occur in all the games. These sequences often occur because of a series of exchanges. I found 40 instances of consecutive Houdini moves in the sample, 20 by the NGM and 20 by the GM. The average length of these sequences for the NGM is 5.15 moves. The average length for the GM is almost half that at 2.75.

The t-statistic for this difference is more than 12 which implies that the observed difference could not have occured in our sample if there were not a real difference in the propensity to make sequences of Houdini moves. We conclude that the data provide sufficient evidence that the NGM has a higher propensity to make sequences of Houdini moves - just as Tiger had suspected.

By way of comparison, the average length of Houdini move sequences in the Anand-Gelfand 2012 world championship match is 4.17 (Anand 4.19, Gelfand 4.15). The NGM's average sequence length of 5.15 exceeds the length achieved in the world championship by about one move.

The data seem to show that the NGM tends to play a lot like Houdini. As to the question of how the NGM developed this ability, we really have no actual data to go on and thus all we can do is speculate. It could be that the NGM has played so many games with Houdini that he developed a Houdini-like pattern in his moves. It is also speculated that the NGM used devious high-tech methods to access Houdini during the match and that therefore he enjoyed an unfair advantage over the GM.

Yet, although there is insufficient evidence to conclude that the NGM used any of the high tech cheating devices described by Tiger, the important message in Tiger's videos may be that these devices exist and they are not very expensive; and that alone should be sufficient reason to install the cell phone jammers in tournament halls that Tiger has proposed.

Comments

By the way, do you know that:- Houdini is NOT the best chess engine to use if one is trying to "cheat" !

Why?

-- Houdini is not one of the "fastest" engines. That is, in short time (1-2 minutes), it does not "see" as many moves (plies) ahead as some other programs (engines). But of course, it does not miss much in the lines that it sees!

This makes Houdini actually "unsuitable" for a player who try to cheat because:- After deducting the time it takes to "input" the opponents' moves (during ongoing game, so presumably very discreetly!!) into Houdini, and to "receive" moves from Houdini (again, between moves, so discreetly!!), Houdini may have only 1 to 2 minutes/move on average.

-- Also, Houdini's moves are "less human" than many other engines (which are also 3000+ELO). Using other "human-like" engines, such as Rybka, is thus much more "logical" for cheating purposes.

Personally, I have looked at the discussions and feel that it is very UNFAIR to Borislav Ivanov (that NGM mentioned):-

- Accusations based on "moves correlation" with engines are completely NON-SCIENTIFIC and USELESS. Chess is a game played on "FACTS" (the Board), where any player can decide on his moves. It is highly probable that many (or all!) of his moves in his game(s) coincide with engines' considered best moves (or Kasparov's or Carlsen's, etc.)

- What "engines" did he allegedly use? Why do we "assume" that it was Houdini? or just because Houdini is the best engine at this time? Try asking this simple question: "If I were Borislav Ivanov, what engine(s) should I use to cheat?" I am sure he would answer "Let's use Rybka 4(3200ELO) or Fritz 10 (3000ELO), instead of Houdini 3 (3300ELO); any engine can easily beat GMs anyway!"

- In criminology, to accuse someone of something, that person must have:- A) motive, B) opportunity, and C) means. Are we not trying to prove that he has the "means" to commit cheating, while ignoring the other factors. What "motive" did Borislav Ivanov have for cheating? The prize of EUR2,000+? And did he have the "opportunity" to consult chess engine(s) on EVERY move?

- Borislav Ivanov is 25 yr-old. This is the age when many players make most improvements. The fact that he beated (and also drew + lost to) 2600+ players had no statistical significance. Remember Kasparov (and Carlsen, and others!) who destroyed many GMs when he was "unrated."

Help us finish translating:

We are working hard to make Chess.com available in over 70 languages. Check back over the year as we develop the technology to add more, and we will try our best to notify you when your language is ready for translating!