Facebook Screenreader for blind users is rolling out a feature called ‘Automatic Alternative Text’ (AAT), which essentially describes an image to help such users ‘see’ photos. The feature though is currently restricted to Facebook’s iOS app.
With the help of Artificial Intelligence (AI), the Facebook app will describe what’s in an image, so that these users will be able to imagine what the photo may look like. So for example, instead of just reading out the name of the user who shared the image, now Facebook will say something like, “image may contain two users, smiling, outdoors, waters with who and all reacted to the photos.”
The engineers at 1 Hacker Way trained their object recognition tech by feeding it millions of images as examples. This kind of machine learning is called neural network. The team then made sure that alt text describes photos in a specific order, starting with (the number of) people, then objects and then scenes in the background.
Facebook says that iOS users who have screen readers enabled on their device, will be able to use the new AAT feature. According to reports, this new feature is quite reliable and can identify stuff like car, boat, snow, sunset, sushi, baby, smiling and beard among others. It can also reliably identify a selfie.
The company says that the new AAT feature is possible, because of Facebook’s neural network-based object recognition technology. This network, Facebook says, has ‘billions of parameters and is trained with millions of examples’.
Facebook’s AAT feature is only available in English and restricted to the iOS app for now. It is being rolled out in the US, UK, Canada, Australia, and New Zealand, and the company says that it will soon be rolling out in more countries, and also support more languages.
