China’s largest district prosecution office in Shanghai has developed a new artificial intelligence system that can identify and file charges against citizens for crimes including credit card fraud, gambling, assault, and — yep, you guessed it — political dissent. “The system can replace prosecutors in the decision-making process to a certain extent,” explains a research paper published in the journal Management Review, according to the South China Morning Post.
Although developers allege the AI can accurately identify and assess crimes with a 97 percent accuracy rate based on verbal case descriptions alone, we’d like to take this time to offer our hot take on the news:
Oh good, sweet Lord, no. No, no, no, no. No. N-to-the-goddamn-O.
No.
System 206 —
If that wasn’t ominous and terrifying enough, China’s new AI legal assistant is reportedly based on a preexisting program with the appropriately dystopian moniker of System 206. Chinese law enforcement has utilized System 206 since 2016 to assess the “strength of evidence, conditions for an arrest and how dangerous a suspect is considered to be to the public,” according to SCMP.
However, this new AI — which can be run on a standard desktop computer, by the way — was trained on more 17,000 cases, and analyzes 1,000 “traits” taken from human-written descriptions, “most of which are too small or abstract to make sense to humans.” System 206 reportedly then assesses the available evidence.
A simply terrible idea —
Sure, we could list out all the ways this is a horrifically awful idea, but c’mon. The moral, ethical, and human-rights implications are pretty damn obvious already.
Trusting an AI system to explicitly police and charge human citizens for crimes including political dissent is a central plot point to countless cautionary dystopian sci-fi stories, and for good reason. There might even be a movie about too-powerful robots in theaters right this very moment.
