How augmented reality is redefining entertainment

Google received plenty of attention this past summer when it announced Google Glasses, possibly the highest-profile use of augmented reality (AR) yet. The glasses incorporate AR data to provide users with information directly on the eyeglass display. By now, you may be familiar with all the ways that Google Glasses could change the future, redefining our interactions with technology. What you may not be aware of are the many lesser-known AR projects in development.

Many smaller vendors are using AR in ways that will blow your mind. Although the apps and ideas have yet to garner the attention directed at Google Glasses, they are definitely worth checking out. Here are some examples of how augmented reality is being used and developed for mobile apps today, and how the technology is advancing at an astonishing rate.

AR’s mobile invasion

The term augmented reality was first coined in the 1990s, but the idea behind it is even older. It’s based on the concept of manipulating reality using technology and sending that information back to the user, who can then interact with AR-enabled apps to manipulate their surroundings on the screen in front of them. Unlike the AR of the past, today’s AR is centered less on the idea of wearing specialized headsets, and more on using cameras to distort reality. Now that every smartphone features a solid camera, AR has made significant strides in the mobile app market. Augmented reality can integrate aspects of the real world into the digital world using information sent from our smartphone’s cameras. An app can sense the geometry of objects we see in the real world and turn that geometry into a digital landscape.

AR-enabled mobile apps work by taking an image that you've snapped of your surroundings using your phone’s camera and superimposing that data on the image of the actual landscape. In effect, the apps turn your mobile device into a channel that combines virtual imagery with your actual surroundings. This allows you to interact with an altered view of your surroundings.

Real player: Gaming apps with amazing AR functions


A great example of such an app is Piclings, an iOS game that uses the iPhone’s camera to create level layouts for the game. The game recognizes the images taken by the camera, redefines them digitally, and incorporates them into the game world.

There are also games that are capable of integrating live camera feeds as well as still photos, such as Star Wars Arcade: Falcon Gunner. The iPhone’s camera captures what is in front of the player in real life, and then integrates it into the game. When players are looking at a landscape of mountains, they’ll be shooting down TIE fighters in the same background that actually exists in front of them.

Using AR apps for educational purposes

Although augmented reality has shown innovation in mobile entertainment, the technology has great potential beyond fun and games. A well-known app that demonstrates the best that AR has to offer is Google Sky Map. This Android app lets you point your smartphone camera to the sky and identify the stars, constellations, and planets above you. Details and scenery change in real time as you move your handset across the sky.

A similar app exists for iOS called Star Walk, which also tells you exactly what stars, constellations, and satellites you’re looking at as you point your device at the sky. The camera records both your position in relation to the sky and where you’re pointing. It then presents information to you about the cosmos. The app also has a feature called Time Machine that shows you what the sky looked like in the past or will look like in the future.

Such apps allow mobile devices to become educational tools and can aid students in a wide range of fields. Astronomy is a prime example, but think of the possibilities in fields such as anatomy and medicine. For example, you could use augmented reality to see where each body part is located, what it looks like under the skin, and how it functions.

Google Sky Maps

AR as a tool for navigation

Another great use for AR is as a navigation tool. An example of an app that takes advantage of AR in this way is Spyglass for iOS, which turns your iOS device into an outdoor toolkit with a wide range of features. Spyglass includes a milspec compass, a gyrocompass, maps, a GPS tracker, a speedometer, a sniper’s rangefinder, a sextant, a gyro horizon, an inclinometer, an angular calculator, and a 5X zoom camera.


However, what makes it such an interesting tool is the way it incorporates AR into its features. Spyglass can find, tag, and track multiple locations, bearings, and aspects of the sky in real time thanks to its 3D augmented reality; it also gives you automatic feedback depending on where you’re pointing your device’s camera. Such tools have incredible potential for navigation and travel, especially if the apps don’t need to rely on a data network or Wi-Fi connection to work. If a user’s coordinates and geographical information are stored within the app, the information can be provided via AR feedback rather than online connectivity.

Mobile AR browsers take AR to the streets

Some specialized mobile browsers offer augmented-reality capabilities, such as the Wikitude World Browser, which is available for Android, iOS, BlackBerry, and Windows Phone users. Wikitude overlays AR information within the app, displaying live information as the smartphone points to stores, hotels, scenery, and the locations around the user. The company behind Wikitude also created AR window, a tool geared for website developers that overlays information in ordinary webpages based on camera feeds.

These apps show the potential for AR to become an indispensable tool—especially as more development continues to provide more functionality. In addition to smartphones, tablets, headsets, and nearly any device with a camera can push the boundaries of AR technology.


AR apps and media interaction

Besides being an efficient tool for smartphone users and offering an innovative experience for gamers, AR can also use a camera to interact with other types of media, something we’ve previously seen in the Junaio Augmented Reality Web Browser.

The Junaio browser made headlines last year when German TV station ProSieben used it for a show called Galileo. Viewers with smartphones and the Junaio app could become active participants by pointing their smartphone cameras toward their TV screens in order to answer quiz questions. They could then compare their progress with that of other participants live through the app.

The company described it as the first interactive TV broadcast, and ratings shot up 14 percent thanks to the show’s use of AR. More than 40,000 unique users took part in the quiz, and Junaio became the number one app in the German App Store for three straight days.


Broadcasters have found other uses for AR besides quizzes and surveys; for some companies it can also be a great advertising tool. According to branding company Serviceplan, the SyFy TV channel is using AR for just this purpose: SyFy started an ad campaign where users point their smartphone cameras toward physical and outdoor display ads, and then see different images pop up on their screens.

Ads such as this can be made a lot more interesting with the help of digital technology and AR. They can also be made interactive with users taking pictures or using their devices to make choices related to the ads. There is great potential for AR to change the face of advertising.

SyFy TV ad

AR on the bigger screen

Although AR is currently being developed for a broad range of uses, it’s still in its infancy. This is particularly the case when it comes to AR on the big screen or in home entertainment centers. Imagine an AR app that shows you where the TV cameras don’t go, to give you a true behind-the-scenes view. Imagine movies or TV shows that use AR special effects to add content projected from the TV set to your living room (think of “The Ring”). For instance, AR cameras could register walls and other real objects and display content upon them for you to explore virtually. Another potential use of AR is to virtually incorporate your rooms into the show. For example, a scene from your favorite TV show could take place in your bedroom, using information sent via the mobile camera.

In the future, TV networks could integrate viewer feedback to create new TV episodes or change content. The show could change depending on popular demand from viewers voting using AR technology, or a movie could have different outcomes and endings depending on the choices viewers made through their AR-enabled devices. Users could dictate the future of content.

Apple could be in a unique position to truly embrace AR for TV consumption and home entertainment if the company actually released its oft-rumored connected TV. If and Apple TV set came with a fully functional OS/iOS, many of the above ideas could become commonplace—and those applications could even start incorporating stereo 3D for a truly mesmerizing experience.

Mind-reading headsets integrated with AR

Although AR as a term is generally reserved for apps that allow your device’s camera to alter reality or the on-screen image, it goes beyond this implementation. The future of AR may mean being able to go inside virtual worlds or scenery while your mind controls the augmented objects around you. There are already apps and headsets available that allow you to control apps and aspects of apps—such as a character's movement—with your mind alone.


Google isn’t the only player in town when it comes to AR headsets. Some competing headsets, or the ideas behind them, also have the potential to redefine entertainment. There are a growing number of apps available for mobile devices that provide a great experience and show the potential of augmented reality. By itself, AR is interesting stuff, but its true potential will be seen when it can be combined with other technologies. The adoption of AR technologies may move us closer to true virtual reality, where users can enter an entirely different world. Mobile apps today use AR in incredible ways, but we’re getting increasingly excited about what it will mean for the technology of tomorrow.

Subscribe to the Best of PCWorld Newsletter