Virtually everyone now realizes that a good webcam matters.
As Microsoft rolls out the spring refresh of its Surface hardware—including the Surface Book 3, Surface Go 2, and associated peripherals—you might not be aware of one particular selling point: the Surface camera. Surface user-facing cameras capture 5MP still images and 1080p video, while the vast majority of competing laptops capture 1MP images and 720p video.
That might not have mattered as recently as February, when we reviewed the Surface Pro 7 Windows tablet. But in a world suddenly aware of what a ring light is, good, well-lit video is important. Webcams have become our window to the world. Zoom, Microsoft Teams, Google Meet, or WebEx are replacing restaurants, conference rooms, hotels, cafes and coffee shops as the new “places” where people connect.
Consumers have quickly realized what TV and streaming personalities have known for decades: Viewers must be able to see and hear you clearly. No one wants to be a webcam zombie, especially now. But even Microsoft’s own employees required a shove in the right direction.
Mark Hachman / IDG
This is our life now.
A legacy of superior cameras
Most people tend to forget Microsoft’s exceptionally strong legacy of camera work. The group that develops the Surface cameras is the same group that worked on Microsoft’s standalone Lifecam Webcam business. The camera was one of the iconic features of the Lumia 1020 and other members of the defunct Windows Phone lineup, which migrated to Microsoft in 2013 as part of the Nokia acquisition. Microsoft subsequently manufactured the Lumia 950 before shutting down the business.
Mark Hachman / IDG
Though Microsoft never technically manufactured the Lumia 1020, its massive image sensor and excellent color fidelity helped identify Microsoft’s phone platform with superb image quality.
Likewise, the original Microsoft Surface included a pair of 720p “Lifecams” on the front and rear when it debuted in 2013. With the Surface Pro 3, which represented the coming-of-age for the Surface Pro tablet line, Microsoft upgraded the user-facing camera quality to its current 5MP specification .
“It took a while to convince people that webcams—the environments that they’re used in—are really pretty low-light,” said Doug Beck, the director of engineering of A/V systems for the Microsoft Surface team.
Originally, Microsoft tested the Surface cameras by sending light meters home with employees. They were amazed, Beck said, to discover how dark many work-from-home environments were. Microsoft’s engineers set out to prioritize the signal-to-noise ratio (SNR) within the Surface imaging pipeline, marrying a bigger sensor with larger pixels to minimize the graininess, or noise, that would otherwise occur in low-light situations.
Fortunately, Microsoft had the space within the company’s Surface Book and Surface Pro tablet bezels to install a larger sensor. A larger silicon area allows the sensor to capture more light, in conjunction with the camera’s lens. Microsoft’s goal was to install “as big of an aperture lens that we can put in,” Beck said, which is a F2.0 aperture. Microsoft then put its engineers to work moving up and down the scaling pipeline, including the image signal processor, to improve SNR. Microsoft works with its third-party suppliers and provides active feedback to help design better denoisers and other improvements, he said.
Mark Hachman / IDG
Microsoft researchers and engineers work together to determine an accurate color spectrum that’s reflected in both the cameras as well as the Surface displays, like the Surface Pro 7.
The end of the process is to match the physical output of the camera pipeline to the physical signal that is output by the camera hardware to provide the best image quality possible, across multiple lighting conditions and scenarios, Beck said.
Blending art and science
“It’s just mentally straining when the audio quality is not good, and it’s mentally straining when the video quality isn’t good when you rely on it. Most people just stop looking at you, and you kind of lose that dimension of interactivity.”
Stevie Bathiche, Technical Fellow and an Applied Sciences Group lead at Microsoft, is telling me—over a webcam, of course—how he and his friends still hang out on Friday evenings, over Microsoft Teams. (Of course.) Being geeks, they can’t help but razz each other when someone signs on with a device that’s not up to snuff.
Mark Hachman / IDG
Remember how some laptop manufacturers thought that a keyboard-mounted webcam was a good idea? Microsoft mounts its Surface webcams at the top of the display.
Designing a better webcam, Bathiche said, means more than just the mechanics of delivering a high-definition signal across the Internet. It’s a fusion of art and science, and artistry enters when the science isn’t good enough.
Portraying an individual accurately is essential to representing their identity. Microsoft spent a lot of time just matching skintones. “We have some IP to match skin tones, across the variability of skin tones that are out there,” Beck said. “We do a really good job of matching those up and presenting them across the calibrated pipeline.”
Microsoft also has color scientists trained in the psychovisual aspects of the human visual system, Beck explained, blended with their background in color science. “We do objective testing to fall within certain contrast and color accuracies, but then the end of our process is really a subjective process,” Beck said. “They’ll make certain adjustments that sometimes don’t feel like they go with the science, but they provide the best possible outcome.”
The future is not more megapixels
Current Surface devices employ the 5MP user-facing camera as well as an 8MP rear camera. The latter is somewhat of a rarity among competing tablets and laptops. Microsoft, however, considers a rear-facing camera a necessity. Microsoft has improved its Camera app, combining it with its Office Lens technology to “read” documents and output what it sees to the Microsoft Whiteboard app. Right now, this is totally unique to the Surface line.
(In the first Surface RT, Microsoft tilted the rear camera so that it would be pointed straight back when the tablet was reclined with the kickstand. “We knew that people weren’t going to use it to take family photos,” Bathiche said. “That wasn’t its purpose. If it were, we would have done things differently. But we also felt that …hey, when you wanted to show someone something, or help me fix my DVD player, it was really nice to have a rear camera in these scenarios.”)
Microsoft
Microsoft’s rear-facing Surface cameras may not get a lot of attention, but they boast autofocus capabilities and can capture 8MP still images or 1080p video.
Will Microsoft be increasing the megapixels in successive generations? Actually, no. Though the Microsoft execs wouldn’t discuss details of the sensor itself, Bathiche said that it hasn’t changed from the prior generation, calling it the “sweet spot” of what laptop camera use can be.
“There’s a lot of interesting parallels between how we design our displays to how we design our cameras,” Bathiche said. In both, Microsoft never tried to design a product by trying to put a number that’s higher than our competitors, Bathiche said.
“That’s not how you produce a good product,” Bathiche added. “And you know, just like our display technologies, we put the right amount of pixels on the screen, so that you can’t see the pixels themselves, and that’s it. But beyond that, you don’t really need any more pixels, because that would be silly—you can’t see the difference between the higher number and the number we [already] give you.”
Mark Hachman / IDG
Microsoft’s new Camera app can “read” documents, such as this whiteboard.
That belief carries over into how Microsoft designs cameras. “If you add more pixels there, you’re just giving me a smaller pixel which collects less light, which makes noise higher, which makes it harder to manage thermals,” Bathiche said.
So would Microsoft ever include, say, a ring light on a Surface to better illuminate users? Again, Bathiche says that’s not the focus—Microsoft believes that the combination of a great sensor and great optics make up for that.
Where Microsoft is going, however, is the AI route—not surprising, given all of Microsoft’s recent work in the field. “For the future, the things we’re looking at are using way more sophisticated algorithms, things like deep neural networks, to essentially extract image features that are just way, way, way down in the noise floor,” Bathiche said.
Six months ago—heck, three months ago—a discussion of mobile camera quality would be synonymous with smartphones, not PCs. But we’re not generally on the go with our smartphones right now.
That’s left the PC, and its camera, front and center in our lives. “The beauty is that we’ve seen the PC really kind of be that new virtual place where the office is cool,” Bathiche said.
Updated at 2:36 PM with to clarify Bathiche’s title.