Fact check: Is the kick off glitch fixed?

All FIFA players have conceded goals right after kick off. You can call it a glitch or a convenience or whatever you desire. No matter what, it’s indisputable that it’s a lot easier to score right after kick off due to some extremely uncoordinated defending. The glitch has existed for ages and has been part of the scripting / handicapping debacle, as it allows people to score easy equalizers, thereby in effect making matches more even.

Then, at January 24th, EA finally released a title update addressing the issue by a combination of various measures:

After kickoff, the defending team will be more compact in defense.
– Their attacking players will be less aggressive in pushing up the field.
– Their defensive players will close the space between them and their attacking players to reduce the gap and limit the attacking team’s space.
(– EA’s patch release notes)

The billion dollar question that arises here is: Did it help, then? The answer is: Yes.

Kick off goals before and after the glitch

Redditor Arlington69 wrote a post, which unfortunately didn’t receive the attention it deserves. In the post, he reveals some data on the subject. He made a sample of 398 matches played pre patch, and recorded the time of all goals scored. He then counted how many goals were scored by either team within 5 minutes of that team conceding a goal. Apparently, the sample doesn’t consider goals scored in the first 5 minutes of the half, even though these would constitute kick off goals as well.

He found that 397 of 2442 goals (16.3 %) were scored within 5 minutes of the same team conceding a goal.

As the average 5-minute interval contains 5.6 % of all goals, Arlington69 concludes that it’s more likely to score soon after kick off (after having conceded) than at any other time during the match. Hence, his data confirms that there is a problem, which however hardly was in dispute. Afterall, EA’s is a direct acknowledgement thereof.

As for the post patch situation, Arlington69 did another – but unfortunately small – sample of just 40 matches. He found that the corresponding percentage now had dropped to 12.9 % (no absolute numbers provided). This is still more than 5,6 % but on the other hand also less than previously.

Hence, Arlington69’s data would suggest that the patch had an effect, although it didn’t remove the glitch.

Criticism

There are certain limitations in regards to the reach of the conclusions set out above. First and foremost, the medtod applied herre fails to consider confirmation bias, i.e. that people could be trying to score kick off goals to a lesser extent because they don’t expect it to work. Second, the sample size is a big issue, which we will revert to below.

We can’t tell whether confirmation bias is at play here, but the sample size issue is something we can look into. Below, we have inserted the facts provided by Arlington69. The numbers attached with a tilde (~) are estimated based on the assumption that the number of goals per match remained at 6.14 after the patch.

Pre patch

Post patch

Matches

398

40

Goals

2442

~245

Kick off goals

397

~32

Kick off goal %

16,3%

12,9%

While the kick off goal % became lower after the patch in the sample, that doesn’t guarantee that it became lower if we were to look at the entire population. To handle that uncertainty, we calculated a confidence interval around each of the two samples.

In many cases, the default confidence level will be 95 %, meaning that you will be able to conclude with 95 % certainty that the percentage of kick off goals did go down.

However, the post patch sample size is an issue here: At 95 % confidence level, the percentage of kick off goals pre patch lies between 14.8 and 18.7 %, whereas the same percentage post patch lies between 8.7 and 17.1 %. Since the intervals overlap, we can’t conclude with 95 % certainty that the percentage of kick off goals actually did go down. If we however lower the confidence level to 75 %, we find that the two confidence intervals aren’t overlapping.

It follows from the above uncertainty that it is difficult to assess with accuracy the size of the impact.

Conclusion

It’s definitely more likely than not that the percentage of kick off glitch goals did go down after the patch. What we don’t know with certainty is whether the apparent decrease happened due to people being less prone to attempt scoring right after kick off or due to the adjustments to the game performed by EA.