For years, companies have hosted bug bounty programs to entice well-meaning hackers to spot flaws in software so they can patch them. The programs—participants usually get money for flagging securities holes—are a recognition by businesses that they can't find every vulnerability on their own.
Now, tech companies like Microsoft, Nvidia, and Twitter are hosting bug bounty programs specifically for artificial intelligence. The goal is for outsiders to spot flaws in A.I. software so that companies can improve the technology and reduce the risk of machine learning discriminating against certain groups of people.
For example, last week, Microsoft and Nvidia detailed a new bug bounty program during the annual Defcon hacker conference. The companies plan to reward hackers who manage to alter computer viruses so that they go undetected by some of Microsoft’s A.I.-powered malware-detection services. Hackers who can create scammy emails that evade Microsoft’s machine-learning powered email phishing detection software will also earn some money in the form of Microsoft gift cards and other prizes.
Meanwhile, Twitter pitched a bug bounty aimed at spotting bias in its A.I. The program comes after users discovered that Twitter's image-cropping tool disproportionately removed women and people of color from photos so that the images would feature white men in the center.
Outsiders were invited to inspect and find flaws in the now-deactivated machine-learning algorithm that powered Twitter's photo cropping tool.
Researchers discovered other bias problems with the same algorithm used in the image-cropping tool. One discovered that it would tend to crop older people from photos. Another found that the algorithm would remove people wearing head garments, showing a bias against those wearing turbans, yamakas, and hijabs.
The first-place winner of Twitter’s bug bounty used A.I. to modify photos of people’s faces to be more appealing to the algorithm. Through this process, the researcher discovered that the algorithm favored faces that were thin, young, and white—all indications that the technology was trained on datasets mostly of people who conform to today's conventions of beauty.
It’s unclear what Twitter will do with the findings, but executives implied that they would be used to improve the company's tech.
During a panel related to Twitter’s bug bounty program, data scientist Patrick Hall reflected on the need for more scrutiny of corporate A.I. He expressed surprise that A.I.-tailored bug bounty programs haven’t become widely adopted considering the technology's many flaws.
“Just because you haven't found bugs in your enterprise A.I. and machine learning offerings, certainly doesn't mean they don’t have bugs,” Hall said. “It just means that someone you don't know might be exploiting them, and I think for those of us in the responsible A.I. community, we wanted people to try bug bounties for so long.”
Jonathan Vanian
@JonathanVanian
jonathan.vanian@fortune.com
