An outage at Facebook and Instagram has inadvertently revealed the usually hidden messages attached to the photos posted by users.
The global problems – which started simultaneously across Facebook products, and also took in WhatsApp – left parts of the sites working but others completely broken.
But the issues appeared to have had a particularly pronounced affect on photos. On WhatsApp, pictures could not be sent; while on Instagram and Facebook, images were not loading, leaving the boxes were they would normally appear empty.
However, those same boxes are filled with the usually hidden text, revealing something important about what Facebook does with the images uploaded to it and Instagram.
The pictures include a message reading "image may contain". It then goes on to list the things that Facebook's artificial intelligence systems have spotted inside the images, and even occasionally attempts to guess which people are in them.
They can be seen on Mark Zuckerberg's own profile, for instance – which mostly loads as normal, but contains a series of errors in the "Photos" section, at the bottom right of this image:
They can be seen on your own profile by simply heading to it in your browser, where the images should show the same. Sometimes, Facebook and Instagram pages will suffer more severe faults and simply refuse to load, though they will sometimes reappear when the page is refreshed.
They are often much more detailed than those shown on Mark Zuckerberg's profile. They appear to be able to identify what a person might be doing in a picture, such as dancing or walking; whether there something like a pet or certain object included in the image; and information about they were taken, such as at a wedding, outdoors, or at night.
Facebook's systems to guess what is in pictures serve a range of purposes.
Importantly, the tools are used to add accessibility to the site. If people are unable to see the photographs, then they can instead use technology to read what the images contain – allowing them to understand the images even if they cannot see them.
Such notes, which are known as alt-text, can be vital to people who are blind or partially sighted, allowing them to read what is in a picture and engage with it. Traditionally, that alt-text is added manually by the person who shares an image – sites such as Twitter give people a feed to describe the thing they are uploading – but the rise of artificial intelligence has offered a way to allow computers to do the work by recognising what an image contains.
But those automated descriptions are also a reminder of the sophisticated processing technologies that are available to platforms like Instagram and Facebook. By using facial recognition technology to try and guess who appears in photographs, social media sites could be able to build a graph of who a certain person is associating with – and then use that to decide what to show them, or potentially for advertising.
Facial recognition technology is banned on social media sites in Europe, because of privacy concerns about how it can be used, meaning that not everyone will be able to see the messages about who Facebook has guessed is in a certain photograph.
The same error was showing on Instagram as well as Facebook. That could be an indication that the same underlying technology is being used across both sites, which might explain why both of the sites went down at the same time.
