NASA: Microsoft HoloLens is helping us find the best sites for bases on Mars

NASA is using Microsoft HoloLens to find the best places on Mars to build bases that would one day house astronauts exploring the planet, a senior member of the Rover team has said.

Jay Torres is part of a group at NASA developing the Mars Exploration Rover, which will be sent to the Red Planet in 2020 to study the landscape, collect samples and look for past life. He has been recreating Mars using real data and HoloLens, Microsoft’s mixed reality headset, so scientists on Earth can better understand the planet and astronauts can prepare for life in space.

Rather than put users in a fully computer-generated world, as virtual reality does, HoloLens allows users to place 3D digital models in the room alongside them.

In a special event at Microsoft’s London office, Torres showed how NASA experts could use data taken from previous Mars missions, upload that into HoloLens and let them “walk around” Mars and explore its surface from the comfort of their own office. Torres was asked if the device could be used to spot areas where astronauts could set up bases. “Absolutely,” he replied. Torres was then asked if NASA was currently doing this. “Yes,” he said.

“We put HoloLens on scientists and they immediately went to the highest point on Mars and looked down on where they were. I asked them why. They said they would do that on Mars to get an idea of the land. They were already acting as if they were really on Mars.”

Researchers at Brown University, in the US, recently released the first high-resolution map of Northeast Syrtis Major, one of three areas NASA could send its 2020 Rover to explore. Using the Mars Reconnaissance Orbiter spacecraft, they found large mineral deposits in the area, which the Rover could take samples of to determine how the planet has changed over time and whether there was life there at some point. The other sites being considered are the Jezero crater, which was once a lake, and Columbia Hills. NASA will make a final decision on the landing site in 2019.

Everything you need to know about HoloLens

“As we turn our eyes to the next target for in situ exploration on the Martian surface,” the Brown University researchers wrote in their paper, published in the journal Icarus, “no location offers better access of the gamut of geological processes active at Mars than Northeast Syrtis Major”.

Ben Riggs, a Mechanical Systems Engineer who is working on the Mars 2020 Rover, uses the Protospace program for HoloLens to “interact with the Rover model in a natural, intuitive way, while still collaborating with other people in the room”. He told attendees at the Vision VR/AR Summit in Hollywood that the Rover design team is using HoloLens to evaluate any potential problems with the mission well ahead of the launch. “Finding potential issues early in the design process is vital to the success of the project, and the Protospace tool gives us that opportunity,” he said.

It is just one example of scientists using HoloLens to prepare for space missions. Astronaut Scott Kelly has used the device to complete tasks on the International Space Station while speaking to colleagues on Earth via Skype (see below); spacecraft designers have used it to improve their hardware; while NASA has even used HoloLens to create a Mars exhibition at the Kennedy Space Centre in Florida featuring Buzz Aldrin, the second man to walk on the Moon.

Lorraine Bardeen, General Manager of Microsoft HoloLens and Windows Experiences, said it’s not just NASA that’s embraced HoloLens, as more companies use the device to become more efficient and ultimately serve their customers better.

“More companies are beginning to understand and deploy the benefits of HoloLens and mixed reality. The eco-system is thriving around HoloLens, enabling more customers than ever to create HoloLens solutions, and we’re excited to look to the future in anticipation for what mixed reality innovations will be created next.”

Torres, whose official role is Deputy Rover Planning Subsystem Lead at NASA’s Jet Propulsion Laboratory, said that in many cases, using HoloLens on Earth to explore space is better than sending a person up in a rocket.

“The virtual world is better than actually being there. We can allow scientists to have these superpowers that they wouldn’t normally have if they were standing on Mars. We can allow them to fly high over the mountains and dunes, or go into the valleys. We can help them see the terrain, rocks and samples in greater detail, and we can give them X-ray vision to see under the ground. We can show them Mars in a way that they wouldn’t get with normal eyesight. All this can be done by using HoloLens.”

Up until now, NASA scientists have studied 2D images of Mars to find interesting rock formations and mineral deposits. However, they found it difficult to judge distances and angles using flat images. Using HoloLens has meant they could walk up to structures, making location predictions up to three times more accurate. Experts from around the world could also join the same experience and add their own points of interest, as well as see others’ notes on the landscape, encouraging collaboration.

Torres believes this could lead to “thousands” of people using HoloLens to “join” astronauts on a future journey to Mars.

“In 1969, when we first put man on the Moon, people were glued to their televisions as they watched those grainy images being sent back by astronauts,” he said. “I believe that when we go back to Mars, and we will go back to Mars because that’s what humans do, although there will be a select group of astronauts walking on Mars, they won’t be alone. They will be joined by hundreds if not thousands of people virtually. You can call them “Telenauts”. These Telenauts will not only see what the astronauts are seeing but also see everything from the orbiters, the landers and the rovers. I believe the future of space exploration has begun and I can’t wait to share that.”