Microsoft’s marketing machine has painted the HoloLens as sort of a quasi-magical device that can revolutionize the way we interact with computers. That’s still pretty true. But spend enough time digging through Microsoft’s online documentation, and you’ll find in reality that for all its $3,000 price, the the HoloLens is still limited in many ways.
Internally, Microsoft has known this for some time. But now that it’s begun circulating this information to developers, you need to begin learning about it, too—and the HoloLens has several significant weaknesses and strengths you might not have known about.
Granted, Microsoft’s HoloLens renders augmented reality—virtual objects superimposed over real-world objects, viewed through transparent glass— which is quite different than the virtual reality offered by the Rift, which plunges your eyeballs into a totally opaque, virtual theater. Still, dollars are dollars, whatever device you end up with.
1. The HoloLens is significantly lower-resolution
By now, the hardware underlying the HoloLens is fairly well understood, at least in broad strokes. At its heart is a Microsoft Holographic Processing Unit (the HPU 1.0), which Microsoft custom designed to offload all the holographic image processing. Apps are processed by an undisclosed 32-bit Intel chip on top of 2GB of RAM (with an additional 1GB of RAM for the HPU), plus 64GB of flash allocated for storage. There’s also a 2MP camera on the front for video recording. Microsoft has said the HoloLens will have between 2 and 3 hours of battery life, based on active use; though it’s designed to be untethered, you’ll also be able to use it while charging.
Here’s where it gets interesting. Microsoft says the HoloLens optics include a pair of HD 16:9 light engines, without specifying the resolution. On its site, though, Microsoft spells it out: The “default and maximum supported resolution” of the HoloLens is 720p or 1268×720 per eye, while the lowest supported resolution is just 360p (634×360). Microsoft also insists that developers maintain a minimum of only 60 frames per second when coding HoloLens applications. If the HoloLens is capturing video, the frame rate will drop even lower, to 30 frames per second.
Microsoft The Skype Holographic app for HoloLens allows you to move the virtual Skype window around physical space.
Both the Oculus Rift and HTC Vive offer a total resolution of 1080×1200 per eye and a 90Hz refresh rate. That’s much higher than the HoloLens on both counts.
It’s hard to say what this is going to mean in real life. Microsoft partner András Velvárt points out that the HoloLens is actually pushing 240 frames per second, allocating 60 FPS each to a red, green, blue, and a second green layer.
Microsoft’s documentation also goes into great detail identifying and trying to minimize sources of discomfort for the user, such as prioritizing a steady 60 frames per second to eliminate visual judder and make holograms feel stable. Terms like “jumpiness,” “swim,” and “drift” are all probably going to make their way into our conversation as we discuss augmented reality and virtual reality further.
2. Microsoft wants you to focus on “holographic density” instead
Ignore the number of pixels, though. When evaluating your Windows Holographic experience, Microsoft would prefer that you focus upon something called “holographic density,” or how bright and beautiful a hologram is. The “holographic resolution” of the HoloLens is 2.3 million “light points,” Microsoft says, with a “holographic density” of 2,500 radiants, or “light points per radian”.
Three lenses laid one over the other allow the HoloLens to combine red, green, and blue images into color holograms.
“The key to a great holographic experience is holograms that are light point rich, i.e. have a high holographic density, and are pinned or anchored to the world around you,” Microsoft says on a developer page that a Microsoft spokeswoman highlighted when asked for explanation on what those terms mean. “To achieve this, HoloLens has been designed for optimal holographic density of 2.5K radiants. The more radiants and light points there are, the brighter and richer the holograms become.”
It’s worth noting that the 2.3 million “light points” Microsoft is apparently calculating are more than the available pixels that the HoloLens actually renders, or about 913,000 pixels. Moreover, a hologram (such as an attacking robot) will take up a small portion of the screen. A PC’s 3D card needs to draw everything on the screen. The HoloLens, by contrast, just needs to draw holograms. Otherwise, the user will simply view the real world through its transparent lenses.
That might actually be a sneaky benefit of the HoloLens architecture. While the Rift needs to draw every pixel, the HoloLens many be able to reserve its processing power for where it’s needed most.
3. Holograms will appear only at certain distances
When I’ve used the HoloLens before, I’ve always noticed that holograms only appeared at a certain distance, and disappeared when they moved too close to my eyes. As it turns out, you’ll experience the same effect.
Microsoft believes the best holograms will appear to be about two meters away from the user because, in part, its HoloLens displays are fixed at that optical distance. Move a hologram too close or too far away, and the strain of trying to focus each eye becomes too great, and discomfort sets in. At an apparent distance of one meter, virtual objects rendered by the HoloLens will begin to fade out, and they’ll actually disappear as they close to 0.85 meters away.
Microsoft An image showing what it could look like for two designers collaborating on a single digital model using HoloLens.
Microsoft also explains why the HoloLens field of view has seemingly shrunk as Microsoft has fine-tuned the hardware.
“Putting content at 2.0m is also advantageous because the two displays are designed to fully overlap at this distance,” Microsoft says. “For images placed off this plane, as they move off the side of holographic frame they will disappear from one display while still being visible on the other.”
In other words, one eye will be able to see the hologram, while the other won’t. And that’s just going to look weird.
MICROSOFTHoloLens holograms will appear no closer than a meter away from the user. The “sweet spot” of hologram viewing is an object that appears to be about two meters away.
4. Memory restrictions will limit games and apps
There’s a reason why the Oculus Rift is connected to a PC: as an external display, it doesn’t have to worry about limitations like memory and storage. But the HoloLens is a standalone, mobile computer, and available storage and memory will be a real concern.
Microsoft Microsoft has a number of amazing games that are planned for HoloLens. But developers will have to operate under a number of constraints, as well.
For one thing, there’s a hard limit to the allocated size of HoloLens apps and games: 900 megabytes, a relatively generous amount of storage if you consider a HoloLens game to be a simple affair. But consider some of Microsoft’s earlier HoloLens demonstrations, where users moved around virtual spaces where weather apps sat alongside calendars, video, and Skype. That could be tricky. (Microsoft could steal a page from Windows Phone and “tombstone” apps that are not in use, but hasn’t indicated that it will do so.)
So yes, there will be some amazing HoloLens games that Microsoft hopes developers will use as examples, but others may be somewhat basic.
It seems possible that Microsoft might be able to get around some of these limitations by streaming data on the fly to the HoloLens, via its integrated 802.11ac connection. But don’t get your hopes up to play Call of Duty on the HoloLens—it ain’t gonna happen.
5. HoloLens will protect itself by killing your apps
Even if the HoloLens had the graphics horsepower to run an app like a first-person shooter—and let’s be honest, stalking DOOM imps (or your coworkers) in your office building would be incredible—it probably wouldn’t. That is to say, it might run the app for a minute or so, then kill it. Here’s why.
Unfortunately, Microsoft may have built a 32-bit apps processor into the HoloLens, but it’s also passively cooled. That means if its internal temperature spikes too high, the only recourse it has is to shut down the offending apps.
Microsoft says its “essential performance targets” call for the HoloLens to spend less than one minute in the orange and red “danger” areas, where it generates too much heat. (See the graphic, below.) And if that happens, the HoloLens protects itself. “If HoloLens exceeds its thermal capabilities, the foreground application will be shut down to allow the device to cool off,” the documentation says.
MICROSOFTThe key, according to Microsoft, is keeping the 1-minute moving average (the red line) beneath the orange region of high power consumption. If the HoloLens overheats, it will automatically begin shutting apps off.
6. You can record holograms, but only briefly
Microsoft allows apps to capture what the front-facing camera sees, and also what holograms you’re seeing. Microsoft calls this nifty feature mixed-reality capture, and you can launch it from the HoloLens device itself in one of two ways: either by telling Cortana, “Hey Cortana, take a picture [or video],” or by navigating to the main HoloLens menu.
Microsoft Here’s the main HoloLens menu, according to Microsoft. To take a photo, you’ll either need to click the button or simply tell Cortana to take a picture.
Either way, however, your recording time is limited to three minutes. In addition, when the HoloLens is recording the mixed-reality capture, the HoloLens refresh rate will drop to 30Hz to coincide with the recording, which will also be recorded at 30Hz. Both video and still pictures will be captured at 1408×792 resolution, Microsoft says.
7. You can control the HoloLens remotely
Microsoft has tailored a version of its Device Portal to the HoloLens as well. That may sound a bit dull, but Microsoft also uses the portal to provide a look into how the HoloLens views the world.
Microsoft You can manage your HoloLens remotely using the HoloLens Device Portal. Separate screens allow you to download video captured by the device, connect it to wireless networks, and measure its performance in real time.
Chances are that the Device Portal will probably survive the developer environment and make it to the consumer version. Within the app you can connect the HoloLens to a network, watch its performance (and temperature) fluctuate in real time, command it to re-scan a room, and even send it text, which will float in the air as a hologram, via the keyboard. Here’s an even niftier feature, below: 3DView, which allows you to see what the HoloLens sees, in sort of a vaguely Terminator-esque perspective.
Microsoft Here’s what the HoloLens “sees” with its 3D View. (Not much at the moment, apparently.)
It might not be enough to offset the limitations we outlined earlier, but it does go to show the level of polish that Microsoft has applied to the HoloLens experience.
Kudos to Microsoft for exposing this level of detail to the developer community at large, and not hiding it behind a password or other protection. By doing so, you can wade through Microsoft’s Windows Holographic pages and draw your own conclusions.
The point isn’t that HoloLens is going to be better or worse than the Rift or other virtual reality devices—it’s that it’s going to be different. Geeks have a decent understanding of how a PC and smartphone works. But the HoloLens will require a new vocabulary, and we’re just beginning to learn the language.
Clarification: A point that was originally in the story—that augmented reality is quite different than virtual reality—was accidentally edited out of the article. It has been reinserted as the third paragraph. The resolution of the HTC Vive and Oculus Rift has also been corrected.
For more on HoloLens and its virtual reality competition, check out the spirited discussion from a recent PCWorld Show below.