ON MAY 26TH Palmer Luckey, an American best known for making virtual-reality headsets, alerted the world to an odd phenomenon. YouTube was deleting all comments which mentioned Wumao, slang for propagandists paid by the Chinese Communist Party (CCP) to flood online forums with pro-CCP views. “Who at Google [YouTube’s parent] decided to censor American comments on American videos hosted in America by an American platform that is already banned in China?” Mr Luckey asked on Twitter.
Mr Luckey was not the first to notice this, but his tech heft drew an immediate response from the right of the political spectrum, with which he has had connections. Ted Cruz, a Republican senator from Texas, called it “very disturbing” and asked why YouTube was “censoring Americans on behalf of the CCP”. Jim Banks, a Republican congressman from Indiana, fired off a letter to Sundar Pinchai, Google’s boss. One would expect, he wrote, that the “spirit” of the First Amendment would be extended into the American firm’s online platforms.
Google says the moderation of Wumao and other anti-CCP terms was an “error”, which it has fixed. YouTube’s moderation systems, which are highly automated, had not taken “proper context into account and incorrectly removed some comments”.
But what context did the systems miss, and why? Google will not say. The conspiratorial suggestions popular with more hyperbolic sectors of the internet commentariat do not wash. The idea that Chinese spies who had infiltrated YouTube would risk blowing their cover just to mess up the moderation system is nonsense. So too is the suggestion that Google is somehow being paid off by the CCP; few issues cut across the partisan divide in American politics more effectively than the perception of a looming threat from China. For Google knowingly to censor any criticism of the CCP would have been political madness.
More likely, the explanation lies in the nature of the software Google uses to moderate content automatically, which uses a set of computer-science techniques called machine learning. Such software can update itself based on how users interact with the website, without any intervention from human programmers. This automated nature, combined with the software’s complexity, make it plausible for errors to arise in ways that are difficult to understand.
For example, if YouTube comments about Wumao and other CCP-critical phrases are flagged enough times by enough users as spam, hate speech or bullying, then the system could start removing them automatically. This could be the result of something as harmless as a furious comment war between pro- and anti-China factions, or of a campaign designed to influence the moderation software. Google says this was not the source of the error, but would not say what was.
It is also possible that the error occurred on Google’s side. Machine-learning systems need to be given context to understand the material they are to be used on, and Google probably contracted out the job of labelling the Chinese phrases that are being blocked. Its supplier may have labelled Wumao and other words as being in breach of Google’s community guidelines, without Google checking its work. Google declined to comment.
Problems with complex systems like this can be solved, but they are opaque not just through Google’s choice, but inherently through their technical design. If YouTube’s comment-moderation systems were nudged into the automated deletion of anti-CCP phrases, despite the application of the highest tech, similar holes will surely pop up in future.
The only way for Google to win this game of cat-and-mouse is to shut down YouTube comments entirely. Or it could gain an edge by spending a lot of money on humans to do the moderating. Neither is likely. Instead, such issues will probably keep providing sticks with which Mr Cruz and friends can beat the tech giant. With the outcome of a Department of Justice investigation into Google’s monopoly power looming, that may prove a liability. ■
This article appeared in the United States section of the print edition under the headline “Everything in moderation”
