Tech

Microsoft's incredible new app helps blind people see the world around them — take a look

Dave Smith
2016-04-04T20:59:00Z
Read in app
microsoft seeing ai app
"I think it's a young girl throwing an orange Frisbee in the park," Microsoft's AI will tell you. Microsoft

Artificial intelligence is helping people do extraordinary things. At Microsoft, it's helping blind people see the world around them like never before.

Advertisement

Last Thursday, Microsoft showed off its Seeing AI app for the first time. It's still under development, but it looks extremely promising.

Using a smartphone camera or a pair of camera-equipped smart glasses, the Seeing AI app can identify things in your environment — people, objects, and even emotions — to provide important context for what's going on around you.

Take a look.

Advertisement

Meet <a href="http://saqibshaikh.com/"target="_blank">Saqib Shaikh</a>.

microsoft seeing ai app
Microsoft

Shaikh lost the use of his eyes when he was just seven years old.

microsoft seeing ai app
Microsoft
Advertisement

Shortly after, Shaikh was introduced to talking computers at a school for the blind. This inspired him to become a programmer.

microsoft seeing ai app
Microsoft

Shaikh has been working at Microsoft for the last 10 years as a software engineer.

microsoft seeing ai app
Microsoft
Advertisement

He's been working on an idea he's had since his college days: an app that can tell you at any moment what's going on around you.

microsoft seeing ai app
Microsoft

Shaikh teamed up with like-minded engineers to work on this very app using Microsoft's intelligence APIs.

microsoft seeing ai app
Microsoft
Advertisement

The app, called Seeing AI, works on smartphones as well as Pivothead's SMART glasses, pictured here. Just a simple swipe forward on the glasses tells you what you're looking at.

microsoft seeing ai app
Microsoft

"I think it's a man jumping through the air doing a trick on a skateboard," Microsoft's AI says.

microsoft seeing ai app
Microsoft
Advertisement

This also has applications for things like working in groups.

microsoft seeing ai app
Microsoft

The Seeing AI app can tell you whether or not people are really listening to what you're saying, and it can describe the general age and gender of the people around you, as well as their emotions.

microsoft seeing ai app
Microsoft
Advertisement

"I see two faces: 40-year-old man with a beard looking surprised; 20-year-old woman looking happy."

 

The app is also good at reading out text.

microsoft seeing ai app
Microsoft
Advertisement

You can use the app to take a picture of the text, and the app will actually guide you to ensure you're capturing the full text of what you're looking at.

microsoft seeing ai app
Microsoft

"Move camera to the bottom right and away from the document," Microsoft's AI says.

microsoft seeing ai app
Microsoft
Advertisement

Once you take the picture, Microsoft's AI will recognize the text. It can even identify and read back headings in case you don't want to know about every single item on a restaurant's menu, for example.

microsoft seeing ai app
Microsoft

"Years ago, this was science fiction," Shaikh says. "I never thought it would be something you could actually do, but artificial intelligence is improving at an ever faster rate and I'm really excited to see where we can take this."

microsoft seeing ai app
Microsoft
Advertisement

You can check out the full video of Microsoft's Seeing AI app here.

Read next

Microsoft Artificial Intelligence
Advertisement
Close icon Two crossed lines that form an 'X'. It indicates a way to close an interaction, or dismiss a notification.