Could artificial intelligence make devices easier to use? According to Samsung, it sure can, and that’s what it the company out to prove with its Bixby AI service.
Bixby is being loaded on the Galaxy S8 and S8+ smartphones, which were announced on Tuesday. Bixby is an agent that can help the smartphones talk, recommend, and remind, said Mok Oh, vice president of service strategy at Samsung.
The AI service is being positioned as a more intuitive way to use and interact with smartphones. For example, Bixby can help smartphones execute tasks with a voice command. It also brings cool features like image recognition and language translation on board the S8 smartphones.
Bixby isn’t intended to be a service like Amazon’s Alexa, Microsoft’s Cortana, or Apple’s Siri, though it does have some of their features. It’s also not a replacement for S Voice, which won’t be present on S8 and S8+ but will live on in Tizen devices.
Samsung plans to “Bixby-enable” other devices, such as appliances made by the company, though it is unclear when that will happen, said Werner Goertz, research director at Gartner.
There is a dedicated Bixby button on the smartphones. Samsung didn’t provide a chance to test Bixby and didn’t say when the AI service would come on the new smartphones. It could be available on the smartphones as early as April 21, when the S8 handsets ship, or be pushed out in a software update. Samsung has said the service will mature over time.
Once you long-press the Bixby button, here’s what the S8 handsets should be able to do.
Execute touch commands
Whatever you can do with touch, you can do with voice. That’s what Samsung is aiming for with Bixby.
Give the smartphone a command—like to take a selfie, launch the photo gallery app, or to make a phone call—and the device will be able to decipher it. You can speak out a command in multiple ways, and Bixby will understand it.
Image recognition
The Galaxy S8 smartphones will be able to take pictures, identify objects, and provide context to the images. Samsung provided the example of Bixby recognizing the Flatiron building in New York City and then providing recommendations on places to eat nearby.
This feature is also being linked closely to shopping—users can take an image of a product and post it on Pinterest, which is partnering with Samsung. Users will also be able to take picture of a product and check pricing, shop, or see recommendations of the products from sites of retail partners.
Language translation
Samsung provided an intriguing example of Bixby being able to take a picture of text and translate it into different languages. Samsung didn’t demonstrate how it worked, so there’s no clear sense yet of how this will happen. Bixby supports 52 languages, but the service isn’t based on Samsung’s homegrown technology. Instead, the company using a partner to provide this service. Samsung’s Oh didn’t provide the name of the partner and didn’t say if the company was also using a backend service like Google Translate for the capability.
Learn more about users over time
Over time, Bixby will learn about smartphone usage patterns and anticipate user needs. That feature will manifest in the form of the Samsung Galaxy S8 smartphones organizing and displaying “cards” based on anticipated actions.
For example, in the morning you may automatically see news and weather information, or Bixby may help fire up Uber to contact a cab to take you to work. In the evening, a reminder to set up the alarm may pop up. The goal is to contextualize use from commonly used apps in one single place, said Sriram Thodla, senior director at Samsung.
Which apps?
Bixby shines with the cameras in the S8 smartphones, bringing more context to images. But the bad news: Bixby won’t work with all apps. It is initially being integrated into a few apps like Photo Gallery, with wider app support coming later. Later on, Samsung will provide a software development kit, and developers will get to work with Bixby APIs (application programming interfaces).
The APIs could be helpful to retailers looking to integrate their stores into the Bixby recommendation engine. They will also help internet-of-things device makers that want to create devices for Samsung’s SmartThings device management system, which could get Bixby support later.