Seeing AI app

Microsoft’s Seeing AI app for visually impaired people released in the UK

Microsoft’s Seeing AI app, which helps blind and partially sighted people by narrating the world around them, has been released in the UK.

The free program uses artificial intelligence to recognise objects, people and text via a phone or tablet’s camera and describes them to the user.

Seeing AI, an ongoing research project from Microsoft, is designed to help people with vision impairments complete everyday tasks and offer new levels of independence.

According to the Royal National Institute of Blind People (RNIB), more than two million people in the UK live with sight loss, and almost half of blind and partially sighted people feel “moderately” or “completely” cut off from people and things around them. The RNIB estimates that sight loss costs the UK economy more than £4.3 billion in indirect costs, such as unpaid carer costs and reduced employment rates.

Saqib Shaikh, who is himself visually impaired and led the development of Seeing AI from Microsoft’s offices in UK, said he was excited at the possibilities the technology can offer people.

“Since launching Seeing AI, it’s been amazing to see how people with visual impairments have been using the app to increase their independence, and we are excited to bring the app to the UK – my home country.

“At Microsoft, we aim to empower every person and every organization on the planet to achieve more, and Seeing AI achieves this in a very real sense.

“As someone who is visually impaired, this has become a key tool in my daily life, and I look forward to others in the UK benefitting from it, too.”

The program lets users recognise:

  • Text – speaks text as soon as it appears in front of the camera
  • Documents – provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting
  • Products – scans barcodes, using audio beeps to guide you; hear the name, and package information when available
  • People – allows users to saves friends’ faces in their contact list so they can be recognized later
  • Scenes (early preview) – hear an overall description of the scene captured
  • Images in other apps – just tap “Share” and “Recognize with Seeing AI” to describe images from emails, photos, Twitter and more.

The app is now available for iOS devices in the UK, Ireland and Australia, after being released in the United States, Canada, India, Hong Kong, New Zealand and Singapore earlier this year.

Seeing AI was unveiled at Microsoft’s AI Summit in London in July, and Chief Executive Satya Nadella recently wrote about how it has helped one of his colleagues – Angela Mills.

“Angela is visually impaired, having lost her central vision when she was a child, today she has only her peripheral vision,” he wrote in an essay on Linkedin. “Before [Seeing AI], when she went into a cafeteria for lunch, she would have to find a cafeteria worker and ask for help picking out a salad. Now, she can go to the salad area and use the app on her phone to read the labels of the salads before choosing one by herself. She can walk confidently into a conference room, knowing for sure she’s in the right place with the right people; and when she’s cooking at home, she doesn’t have to rely on her husband to read out the labels on the spice jars.

“It’s incredible to see how this idea that was sparked and developed at our annual hackathon by Saqib Shaikh, [Senior Data Scientist] Anirudh Koul and team, is now a free application that enables Angela and many others to feel more included and connected to the world around them.”