Unearthing Microsoft’s HoloLens feels a little like walking the decks of the Titanic. Four years ago, Microsoft’s augmented-reality headset ignited the imaginations of consumers and developers alike with its promise of lifelike animated sprites that could perch on real-world objects. Then... it sank.
It’s almost criminal that Microsoft’s original HoloLens demos never saw the light of day. Bending down to peer “inside” a coffee table into the Minecraft underworld was an utterly transformative experience. But at least Microsoft’s vision of using the HoloLens as a business tool apparently is alive and well.
As is the HoloLens. Because Microsoft never sold its device to consumers, PCWorld never formally reviewed it. But we have the original on hand. As Microsoft is expected to bring forth the next generation of HoloLens at Mobile World Congress (MWC) in Barcelona this weekend, we gave it another run-through, focusing on what made it best—and what could be better.
A consumer failure with corporate potential
As a consumer device, the HoloLens is dead. Only one Microsoft Store on the West Coast even has a HoloLens in stock for demonstration purposes. Microsoft’s remaining HoloLens supplies are sold through Insight, a solutions provider, Microsoft support reps said. On Microsoft’s site, the “Commercial Suite” version of the HoloLens costs a cool $5,000.
But commercial it was not, back in 2015. Removing it from its cloth-covered plastic shell three years later reveals a lovingly engineered piece of consumer hardware. The HoloLens perches fairly comfortably on my head, supported by a removeable nose piece and my ears. Clicking a dial adjusts the tightness of the fit. There’s an optional head strap, too, if you'd prefer taking its 1.28-pound weight on the crown of your head. I wore the HoloLens pretty solidly for about three hours in one stretch. While I was conscious of the weight, I never felt uncomfortable.
The HoloLens includes built-in speakers, which can be adjusted with a pair of buttons on one side. On the other, a similar pair of buttons controls the brightness of the display. An optional headphone jack is available too. Underneath the power button is a row of pinprick-sized holes, which light to indicate the battery level. (The HoloLens drains fast, and our unit provided about three to four hours of charge in fairly constant use.)
All that pales in comparison to the focal point of the HoloLens: the display. While virtual-reality devices like the first HTC Vive tethered you to the PC by way of a display and power cord, the augmented-reality HoloLens superimposes the display over the real world—understanding where physical surfaces are, and placing objects accordingly. You're free to roam wherever you'd like. It’s magical, when it works.
By now, the hardware underlying the HoloLens is fairly well understood, at least in broad strokes. In terms of resolution, though, the lack of a PC-powered display means that the HoloLens essentially provides 720p of resolution per eye. Combine that with the relatively limited field of view, and that’s where the experience gets complicated.
Want a brief look at what it's like to actually use the HoloLens? Here's a video using the HoloLens itself, taken with its built-in mixed-reality camera.
The first fix: Field of View (FOV)
HoloLens works its magic by suspending holograms, or what appears to be a three-dimensional object, virtually in front of you. There’s not that much difference between the image that a monitor paints on the glass in front of your eyes and the image that the HoloLens paints on each lens in front of you.
The difference is what HoloLens does next. It uses a front-facing camera to scan the room in front of you, interpreting what it sees as a three-dimensional mesh, then attaches the hologram to that mesh itself. Using the HoloLens usually means anchoring various windows (such as Settings, for example) to a nearby surface, then using any three-dimensional open space near you as your HoloLens playspace. The illusion of physically walking around the hologram, as well as being able to anchor it or have it interact with real-world surfaces, fools your eyes into thinking it’s real.
That illusion is especially effective if the hologram remains in the field of view—the rectangle that serves as the HoloLens screen. The FOV covers much of the width of a door at a few feet, or about 50 degrees. Anything outside of it, though, can’t be entirely seen, and this instantly reminds you of the limits the HoloLens FOV imposes.
Because the HoloLens “knows” where in virtual space a hologram exists, it’s possible to have one directly in front of you and another to the side or rear. It’s trying to turn and find the second hologram when frustration can kick in. (A setting allows you to trash all holograms loaded into your scene, even the ones you can't see.) But the first-generation HoloLens could never convincingly offer up the illusion of, say, a forest, as the HoloLens lacks the capability to swamp your peripheral vision.
The FOV problem isn’t just a visual issue. Interacting with holograms and apps typically requires “air clicking” a hologram or window, in much the same way you'd click a mouse. Triggering the Start menu requires a “bloom” hand gesture, like a flower opening. In both cases, though, the HoloLens must be able to detect your gesture. (There are two alternatives—voice control and a small Bluetooth “clicker”—but gestures are the most common way of interacting.)
Anyone who’s used the HoloLens, then, would rate the field-of-view as the primary area for improvement. If I were designing the next-generation HoloLens, however, I’d strike a compromise, offering users the option of dialing down the peripheral detail.
In the VR space, this is known as foveated rendering, concentrating the processing power on rendering detail where your eyes are looking. That’s probably a necessity for a new HoloLens, too—more on that in a minute. But I also appreciated the blank space at the edge of my vision, either to glance at my PC’s screen, a person, or something else. If Microsoft were to improve the HoloLens, I’d leave the option to “fade out” the periphery, providing a more grounded sense of place to the user.
The second fix: Resolution
It’s no secret that early VR users suffered from vertigo, caused by the latency between what the displays rendered and what the user “expected” to see. Though there’s still some noticeable render lag within the HoloLens, the fact that your vision tracks real-world objects helps prevent any nausea.
I still experienced discomfort. Dialing up the brightness of the display is a must. Even then, however, holograms can be low-resolution, and sometimes laggy when interacting with you. After an hour or two, my eyes ached from the strain. A reported processor spec bump must happen to ensure a next-gen HoloLens will be comfortable to use.
The resolution isn’t merely a visual issue, either. HoloLens maps your environment, applying a very coarse surface mesh over walls, desks, tables, and more. (Objects can be “anchored” to a surface like a desk, and certain games will actually use these surfaces as part of the experience.) A spartan workspace is easily accommodated. But as your environment moves further away from minimalism, HoloLens struggles to keep up.
My basement office, full of monitors and boxes and peripherals and lamps, would probably give organizational guru Marie Kondo the cold sweats. To compensate, HoloLens would occasionally draw a big black surface that would interrupt the scene. A next-gen HoloLens would be well served by increasing the number and density of the three-dimensional mesh of points it uses to creates a virtual space.
The third fix: UI
Certain elements of the HoloLens UI haven’t held up over time, including the “nose first” method of navigating. Too much navigation involves steering a cursor from point to point by slightly moving your head. It’s especially painful when you have to enter a Windows or Wi-Fi password: Manually pushing the cursor from letter to letter is a major pain.
Microsoft’s first-generation HoloLens works around this somewhat through voice recognition, where many simple tasks, such as opening the Settings menu, can be performed by merely saying a word, like “settings.” But modern VR equipment is beginning to include eye tracking, which can save time navigating from element to element. If Microsoft is developing a next-generation HoloLens, it would be a shame if eye tracking were excluded.
The HoloLens voice recognition works well, but even it can be unnecessarily confusing. Some commands can be performed simply by saying the command; others require you to invoke Cortana beforehand, which makes little sense. Still, I found I relied on voice more and more to avoid tediously steering the cursor.
Lifting your hand to constantly “air click” gets old, too. But scrolling is the worst, requiring you to air-click and drag through the minimal field of view. Microsoft’s optional “clicker” basically acknowledged that air navigation failed. But couldn’t Microsoft have tried something a bit different, such as “clicking” by audibly snapping your fingers?
The fourth fix: More apps
If you think the Windows app store looks empty, you should see the HoloLens aisle. At one point, UWP apps like Twitter were considered to be the future, traversing PC, phone, and HoloLens alike.
You can exhaust most of the best HoloLens apps in an afternoon. HoloTour offers genuinely interesting 360-degree recorded video tours of Machu Picchu and Rome, complete with a recorded tour guide. Step back, though, and they aren’t much different from a HoloLens-specific version of Myst, as you virtually explore hotspot after hotspot. Holo Galaxy isn’t much better, with a cursory 3D view of the solar system and other stellar points of interest.
RoboRaid, jaw-dropping in its time, creates virtual bugs and drones that literally crawl out from your HoloLens-mapped walls. You’ll need to use oral commands, your air-click skills, and the ability to hop nimbly about the room to avoid laser blasts. It’s still a lot of fun, but not too different from the “shooting gallery” games that tend to be bundled with VR gear.