After the somewhat awkward experience last month of having an AI Twitter bot go full-on racist in a few hours once it interacted with humans, Microsoft have released a new AI experiment on to the internet - CaptionBot.
The idea is that you upload a photo to the service, and it tries to automatically generate a caption that describes what the algorithm sees. You are then able to rate how accurately it has detected what was on display. It learns from the rating, and in theory, the captions get better.
The bot, from Microsoft’s Cognitive Services team, is the result of some hefty research into how to model objects in photographs so that a computer can understand them. They claim that their system can recognise “a broad range of visual concepts” and also performs entity extraction so that it can recognise celebrities. Which it clearly can do.
It incorporates three separate services to process the images. The Computer Vision API identifies the components of the photo, it mixes that with data from the Bing Image Search API, and runs any faces it spots through their Emotion API. This analyses human facial expressions to detect anger, contempt, disgust, fear, happiness, sadness or surprise.
Other services produced by the group include “How Old Do I Look” and the “What Dog?” robot.
Of course, we immediately tested CaptionBot by uploading pictures from Doctor Who, with admittedly very little success.
Although this result was possibly an astute comment on the state the show was in by the late 1980s.
The internet has also wasted no time testing the limits of the technology - uploading pictures of Hitler to see whether they can make CaptionBot racist, or making it the butt of some pretty good jokes.
Microsoft's image captioning tool sees through the so-called "moon landings" https://t.co/WWr7O1XeE3 H/T @robmanuel pic.twitter.com/BDd9X4KdNp
— Ralph Harrington (@ralphharrington) April 14, 2016
I was hoping to get a definite answer from https://t.co/b5DYRwRxWz but it's raised a lot more questions pic.twitter.com/WVz4gC9kLi
— Paul Curry (@cr3) April 14, 2016
A mixture of amazing, complementary and wrong from https://t.co/zMWockgew4 :-) Really great work @Microsoft pic.twitter.com/AfNTfJzhuf
— Nicholas Tollervey (@ntoll) April 14, 2016
Lol. https://t.co/CcyyDR1Vus #startrek pic.twitter.com/bIiZ1cxQYW
— BurlapJack (@burlapjack) April 14, 2016
On the ranking of slightly dodgy AIs, not recognising K-9 is a significant step up from “genocidal racist”, so congratulations Microsoft.
Has CaptionBot been able to recognise any of your images? Have you tormented it with your own fondly remembered childhood TV shows? Share what you found in the comments below.
