Gerald James

Cities Unlocked: A voyage of discovery

Research by Guide Dogs suggests that in the UK, 180,000 blind or partially sighted people rarely leave home alone. It should come as no surprise then that for these individuals, making any journey, let alone an unfamiliar one, can leave them feeling anxious and vulnerable. But what if we could substitute the one sense they are missing, by augmenting another?

That’s what the Cities Unlocked project aims to do. A technology that places the individual at the centre of the experience, the prototype launched last year and follows the principles of inclusive design, building stronger innovation through exploring all parts of human diversity. Today marks the first update to the project which moves the experience from a collection of prescribed routes designed to get people from ‘A to B’, to a free-form exploration of the world.

Cities Unlocked is a unique collaboration between Guide Dogs and Microsoft, which explores the potential technology has to enable people living with sight loss to get out and about, independently and confidently, in our towns and cities.  The genesis and inspiration of the project was Amos Miller, a visually impaired Microsoft employee and former trustee of Guide Dogs. Becoming a father spurred him on to think about a life where he could enjoy everyday outdoor experiences with his daughter, without the feelings of fear and anxiety that usually accompanied exploring new environments.

Research in phase one highlighted that for Amos and others with sight loss, sound is used as an anchor. Guiding by sound in the same way a lighthouse guides by light, this technology demonstrator paints you a picture with sound. Placing spatially situated synthetic sounds around you – both verbal and non-verbal, it creates a 3D soundscape of the world in a language you can understand.

Phase two update

Designed as an assistive technology, used alongside guide dogs and long canes, today’s update is based on two experience scenarios, “Orientate” and “Look Ahead”. Through this, people living with sight loss can get a sense of what is in their immediate surroundings, empowering them to make informed and spontaneous decisions on the go.

Trialist Gerald James says: “It does paint a picture of the town. It told me all the different shops on each side of the road. But it also told me the names of the streets and also the compass direction. I just think it gives you greater freedom really. And gives you more independence.”

See Gerald in action trialling the technology in this video

The specific phase two technology that gave Gerald that experience includes:

  • An improved app called ‘SoundScape’ with an easy to use interface and access to features making the start of your journey (Orientate mode) and exploration (Look ahead mode) a seamless experience.
  • “Find the Way” mode – allows you to stop at any point and check you’re heading in the right direction using directional audio
  • Working with GN Store Nord, we’re trialling a new over the ears intelligent headset designed as non-occluding – this means you can still hear vital contextual cues e.g. traffic.
  • Phase 1 involved us taking a head scan of each person so that the 3D sounds were placed in the right position for each individual; but as we continue to refine and improve the audio engine and improve the quality of head tracking this is no longer necessary. We are continuing to work with AfterShokz bone conduction headsets to explore the use of off the shelf sensor tags to respond to the movement of the wearer’s head.
  • Our audio builds via MSR audio engine have enhanced the 3D Audio experience to paint an even richer picture of the world around you. Allowing people with sight loss to build a mental map of a familiar or unfamiliar city based on street names, their immediate surroundings and landmarks / points of interest.
  • We’ve developed an integrated application called ‘CityScribe’ which enables people to tag obstacles in their city which most mapping services don’t pick up. This includes park benches, low jutting corners, bins or street furniture. Not only could this make navigating cities easier for people living with sight loss but also for tourists having locally annotated ‘places to go’ in a new city that would mean they could explore off the guide book.
  • We’ve introduced a wearable device worn around the neck, a voice interaction mode, and connectivity of the headset to your phone via Bluetooth, which means there is no need to hold your mobile – it can be put safely away.

AfterShokz Bluez 2 bone conducting headset


GN Store Nord ‘Intelligent Headset’

CU2 4

‘Orientate’ mode

But this isn’t about giving people lots of different pieces of technology. It’s about what it enables you to do when the technology comes together. Put simply, this is a technology that does its job and then gets out of the way. True independence is about going beyond what you know, what you’ve always done, with no hesitation or anxiety. This technology does that.

This is really what Microsoft’s mission is all about. Empowering people with technology to be more productive and to achieve more of the things they want in life. With continual learning from Guide Dogs, we are building a technological proof point that not only has the potential to change lives for those living with sight loss but, by focusing on the specific challenges they face every day, we are realising the wider benefits for sighted people. From better connecting mobile workforces like the police force, and keeping people safer who work in challenging and complex work environments, like oils rigs and manufacturing plants; this technology has the potential to help.

Whilst still at trial phase, and not available in the short term, Microsoft is really excited about the direction this project is going. Even better, this innovation is coming from the UK, in partnership with one of Britain’s most loved charities. We believe that technology can fundamentally change lives, to give people control of their world and how they want to experience it as well as do more of the things they care about. Cities Unlocked is opening up this world of possibility.