Microsoft has reached collaboration agreements with Twitter and Facebook to get their members’ public status updates and messages indexed and presented in useful ways on the Bing search engine.
Yusuf Mehdi, senior vice president of Microsoft’s Online Audience Business, made the announcement on stage at the Web 2.0 Summit in San Francisco.
The partnership with Twitter has it working with Microsoft to optimize how Bing crawls and indexes “tweets.” Microsoft in turn will apply search algorithms to the Twitter messages, so that Bing users will not only be able to see a real-time feed of “tweets” but also rank them by how relevant they are to their query, Mehdi said.
“This is a big deal we’ve been working on for a long time,” Mehdi said.
To rank “tweets” by relevance, a feature Microsoft calls “Best Match,” Bing will take into consideration a number of factors, such as who are the authors of the messages based on a “social relevance” score Bing will assign to them, Mehdi said.
Bing will also evaluate the message’s quality, noticing, for example, if it contains a link to an online article or Web page. It will also take into consideration how popular the message is by calculating how many times it has been “re-tweeted” by others.
In addition to providing links to Twitter messages, Bing will extract the URLs of the pages that the messages are making reference to, so that users can go directly to that source of the information.
When providing links to “tweets” that contain a shortened URL, Bing will put in parenthesis the main Web domain of the link, so that users know, before clicking, whether it’s a reputable site and thus avoid landing in a malicious phishing or malware-laden site.
Bing will also display a tag cloud of the most popular Twitter topics, so that users can click on and dive deeper into them.
The Twitter deal is nonexclusive, and hours later rival Google announced its own agreement with Twitter.
“The ‘tweets’ will be integrated universal-search style, ranked alongside the other [different types of] results, and you’ll be able to click on those results and go to a page that shows only ‘tweets’ and real-time updates,” said Marissa Mayer, Google’s vice president of search products and user experience, in an interview after her appearance on stage at the Web 2.0 Summit to announce the news. Users will also be able to restrict results to Twitter posts from the get-go using the engine’s filtering controls.
Google has so far been crawling Twitter posts on its own, but the subset of content available in this manner has been very small, she said. With this Twitter partnership, Google gets access to what Twitter calls its “firehose” API (application programming interface), which was recently released, she said.
“So we can do this really comprehensive coverage and indexing of ‘tweets’ integrated into Google search,” Mayer said.
Like Microsoft, Google has identified what Mayer calls “relevance signals” in Twitter posts, such as the links they contain and the people authoring them. The latter remains constant, so a Twitter author’s level of competency in a given topic can be analyzed and evaluated, she said.
However, Google is looking at social-networking content in a broader scale beyond Twitter. When asked if Google is interested in striking a deal like this with Facebook, Mayer said, “We’re interested in comprehensiveness, which is a fundamental element of search. We need to have all the answers in order to find the answers for people.”
Along those lines, she said that Google is working on a new search feature to let users view content in their Google search results that their friends have shared with them on social networks.
“We can use social networks and analyze them to improve search quality. When you’re signed in and have a Google Profile established, we’ll look at the different social networks you associate with, understand who your friends and connections are, and surface content written by them on your results pages,” Mayer said.
The Twitter search-result integration and the new social-networking results feature will be implemented “very soon,” Mayer said, declining to be more specific. Google’s announcement of the Twitter agreement states it will be ready “in the coming months.”
Still, it seems Bing is for now ahead of Google with an optimized search experience for Twitter that is already live.
Although Google remains by far the most popular search engine, Microsoft is making a big push to improve its position in this market, starting with Bing’s launch in May and the broad search deal with Yahoo, which is awaiting regulatory approval.
In addition to its core microblogging and social-networking features, Twitter has emerged as a repository of real-time testimonies on whatever is on people’s minds, such as news stories of global importance, celebrity gossip and hot-button issues. As such, being able to capture, analyze and make sense of Twitter’s stream of posts is seen as an important new area in the world of search engines.
“We’re super happy with the Twitter partnership,” said Qi Lu, president of Microsoft’s Online Services Division, who was also on stage being interviewed by conference moderator Tim O’Reilly. Lu declined to disclose financial details of the deal. He also said he wasn’t sure on its duration.
Neither Mehdi nor Lu said much about the Facebook arrangement, other than to indicate that it will be similar in nature to Twitter’s but that it will be implemented at a later date.
It will be interesting to see what shape the Facebook agreement takes, considering that Facebook allows individual members to make only basic profile information available via search engine results. Facebook has indicated it may let members make their profiles open to anyone on the Web, including their status updates, but that hasn’t happened yet.
Twitter, on the other hand, is a much more open service and most of its users make public their “tweets,” messages that can’t be longer than 140 characters.
Facebook expects its deal with Bing to become active early next year, a Facebook spokeswoman said via e-mail. Before Bing can make Facebook status updates searchable, Facebook has to first roll out to all users its new Publisher Privacy Control system, which is now being tested with a small group of users, the spokeswoman said.
“As you may recall, the new Publisher Privacy Control enables users to define who can see the content they publish on a per-post basis. For example, they may want to make some posts available to everyone, while restricting others to their friends and family,” she said.
Microsoft and Facebook have an existing partnership through which Microsoft provides Web search and search ads to Facebook.