Microsoft CaptionBot — the tool looks at the image you’ve uploaded to describe what it sees in the picture. Previously, they launched other tools like How Old, Project Oxford and Fetch App.
Microsoft CaptionBot with some my photos:
The idea is that you upload a photo to the service, and it tries to automatically generate a caption that describes what the algorithm sees. You are then able to rate how accurately it has detected what was on display. It learns from the rating, and in theory, the captions get better.
The bot, from Microsoft’s Cognitive Services team, is the result of some hefty research into how to model objects in photographs so that a computer can understand them. They claim that their system can recognise “a broad range of visual concepts” and also performs entity extraction so that it can recognise celebrities. Which it clearly can do.
CaptionBot gets it right (or decently close) a lot of the time, but in some cases it can’t even figure out what’s in the photo despite what we think seems rather obvious.
