Copyright Notice: This article is Copyright AI Factory Ltd. Ideas and code belonging to AI Factory may only be used with the direct written permission of AI Factory Ltd.
For years we have been battling off claims that our games cheat. We have tried various ways to combat this with some success, but actually there is an intrinsic mechanic here that undermines our efforts!
Background
This problem is originally rooted in the issue that humans are poor at intuitively understanding probability. Part of this is explained at length in Backgammon cheating. The issue here is that if you have millions of people playing a game at the same time, then a few of these will see extraordinary dice rolls. However, they do see these dice rolls and using simple probability theory assert that it was almost impossible to get such rolls. This ignores the fact that hundreds of thousands of players at the same time did not see such rolls and that actually it was inevitable that someone would see these.
To battle this we have in various ways attempted to explain this and finally we resorted to using the professional "fairness assessors" Gaming Labs International (GLI) to independently build and test our program, while having full access to analyse our source code.
Of course, we do not cheat and GLI affirmed that. This gave us an independent authority to support our claims. This has seen some benefit in that previously in the top 50 of "helpful" reviews, where 62% claimed cheating, this is now reduced to 42%. We can further expose this GLI certification and will be doing so.
Perpetuating?
Of course an issue here is that a user that suspects cheating, but sees no other such reviews, will probably say nothing. However having seen other reports of cheating, they are affirmed in their belief and it becomes a tribal rush to also report their claims. This is a closed system revolving around just one app and those users of that app. We had a similar deluge of cheating claims against our Gin Rummy and, by responding to reviews with counter claims, we were able to overturn a situation where most top reviews claimed cheating to a situation where there were almost no such claims
However this was not enough for Backgammon! There were too many claims held by people that are convinced they are right.
Closed System?
In practice the paradox of being "the only one" who sees a bad roll is not just confined to just considering that app. Very many users claim cheating and recommend people play other "non-cheating" apps they have tried. At first glance one would assume that all Backgammon programs are accused of cheating and indeed that is the case.
Given that surely users will not recommend other apps as they are also accused of cheating?
This is where a level 2 paradox occurs. The user had their freaky roll, so they try another app and (of course) it does not show freaky rolls. This was inevitable as their attempt to sample another app does not suffer from the fact that it was just that one special sample from millions of possible ignored samples by other players which showed no skew, as here they have elected to take just one new single sample. Simple probability then works. They are then further affirmed that our app cheats and that other apps do not. The paradox here is that the reverse is true for other apps. In consequence there must be a large body of users out there who have seen their tested Backgammon app show freaky rolls and then found that other Backgammon apps they tested do not appear to cheat. So there will be loops of Mexican standoffs where Player 1 claims A cheats but B does not, and Player 2 that B cheats and A does not.
So this makes it much worse as now they have 3 reasons to assume it cheats:
| (1) | their own test | ||
| (2) | the other cheating reviews and | ||
| (3) | that a rival program does not cheat (so their testing reasoning is "sound"). | ||
What remains is a population of reviewers who are convinced their tested app cheats and that others do not. It's a kind of Mexican stand-off. Each of these are sure they are right and that others must be wrong. However, of course, they are all wrong.
Jeff Rollason - February 2019
