Forget virtual Minecraft castles levitating on tabletops or virtual TVs floating in midair. NASA is taking Microsoft’s HoloLens augmented-reality goggles to space, for the purpose for which they were originally designed: as a tool.
On June 28, NASA will launch Project Sidekick, a program that will launch a pair of Microsoft HoloLens goggles to the International Space Station as part of a scheduled resupply mission. HoloLens will be part of a test to determine whether augmented reality and virtual reality devices can assist astronauts in outer space.
“HoloLens and other virtual and mixed reality devices are cutting edge technologies that could help drive future exploration and provide new capabilities to the men and women conducting critical science on the International Space Station,” said Sam Scimemi, director of the ISS program at NASA Headquarters in Washington, in a statement. “This new technology could also empower future explorers requiring greater autonomy on the journey to Mars.”
YouTube/NASA ASA and Microsoft engineers test Project Sidekick on NASA’s Weightless Wonder C9 jet. Project Sidekick will use Microsoft HoloLens to provide virtual aid to astronauts working on the International Space Station.
What’s more, the way in which the astronauts will use HoloLens sounds exactly like the way in which Microsoft showed the headset originally being used—which few have really experienced, until now.
Why this matters: This could be a win-win for all concerned. Microsoft’s struggled to articulate the productivity aspects of HoloLens for while now, relying on the admittedly impressive concept of Halo and Minecraft to sell the potential of the devices. But what I saw in Redmond in January was far, far more practical, and really excited the possibilities for what the technology could hold. I even suspect that HoloLens’ limited field of view might work to an astronaut’s advantage; there are many more hazards in outer space that an astronaut would want to be aware of, versus, say, an Earth-bound office.
Check out more in the NASA video, below.
A remote tool
Although a NASA press release referred to the two modes of Sidekick, they coincided with what Microsoft showed off in January. The first mode, known as “Remote Expert,” will connect the front-facing camera on the HoloLens to experts on the ground, allowing a remote engineer to see and comment on remote procedures. The second mode, called “Procedure Mode,” will allow that ground operator or someone else to remotely project an augmented reality illustration on top of the HoloLens display—think of a remote engineer virtually “circling” a particular valve or control to catch the astronaut’s attention.
It won’t happen all at once, however. NASA expects astronauts on the station will first use Sidekick by the end of the year—but in a limited, staged rollout. After the HoloLens devices arrive at the space station, crew members will test and verify the software and hardware functionality in a standalone mode. Then a future mission will bring up another set of HoloLens devices with network connectivity built in, to test the remote mode.
Sidekick also will be used and evaluated during the NASA Extreme Environment Mission Operations (NEEMO) 20 expedition set to begin July 21, when a group of astronauts and engineers live in the world’s only undersea research station, Aquarius, for two weeks, NASA said.
Microsoft showed off the other purpose of HoleLens at E3: games.
Still, the future is bright. In January, Microsoft walked us through a demonstration where they put reporters in front of a live light switch and asked us to wire it up. HoloLens projected a virtual Skype window that I could drag and drop in (or out) of my immediate field of view. The remote technician then walked me through the procedure, orally—and more importantly, visually— showing me what to do. That was a brand-new procedure for me, and HoloLens made it extremely simple.
Can HoloLens survive the rigors of outer space? Will the latency from the space station to the ground (nearly a second, round trip) affect its effectiveness? How useful will it ultimately prove? These are all fascinating questions, and one which the ISS stress test will ultimately help to answer.