EyeSense and Aipoly Helping The Blind To Recognize Objects — First app EyeSense is an Artificial Intelligence application that uses deep learning-based computer vision to recognize objects and special facial features, like smiles. The application can be used to help blind and visually impaired to interact with their environments.
Objects Recognition:
EyeSense can easily identify your personal objects. All you need to do is point your device camera towards your surroundings.
Smile and Wink Detection:
To improve human connection, EyeSense reads aloud facial expressions, like smiles and winks, registered on a person’s face.
Voiceover:
Simply tap the device screen to hear EyeSense identify previously learnt object.
Adding Text Description:
You can help EyeSense learn any of your objects by adding a text descriprion. Just double tap the screen and the text will appear again.
EyeSense, an iPad app developed in Egypt, has the ability to “learn” objects in its environment, having been trained by its users. A visually impaired person can point their device in the direction of where they think something might be—say, a coffee cup—and a voice will say that the app recognizes that object.
“The key strength of the app is that it also recognizes basic facial expressions, like winks or smiles. This enhances human interaction,” says Joanna Marczak, a spokesperson for its developer, ID Labs.
Teaching EyeSense:
You can add your personal objects in two simple ways;
1. Open the application, press a Learn button, type a description of your object and add a range of positive* and negative** number of image samples (recommended range of acquired images is from 100 to 200). You should move then your camera around a new object to see it from different camera angles and positions.
2. Upload directly a photo album of positive and negative image samples from your photo library.
You can always add more training images to previously learnt objects to enhance the application performance.
The second app, called Aipoly (see video), does something similar. It’s sophisticated enough to recognize clothing and colors, even in abstract works of art.
Aipoly is an object and color recogniser that helps the blind, visually impaired, and color blind understand their surroundings. Simply point your phone at the object of interest and press the large toggle button at the bottom of the screen to turn on the artificial intelligence. It will keep running and recognising objects until you switch to a different tab, or toggle the Aipoly button. It is suited for both the visually impaired and the sighted.
Marczak says ID Labs is working with visually impaired support groups to improve the EyeSense app, which is free to download (versions for Android and other phones are due soon). It also works offline if necessary. Find it here.
