The Smartphones You Can Rely On

We Americans pay a hefty slice of our monthly budget--$80, on average--for the convenience (and fun) of owning a smartphone. In total, U.S. consumers will spend $4.8 billion on smartphones and wireless services (voice and data messaging) in 2010, according to market research firm In-Stat.

Illustrations by Peter Arkle

With all that money being spent on mobile technology, you'd expect an abundance of critical research and analysis indicating which products and services work well for the money, and which don't. But little information exists. That's why PCWorld initiated this survey, asking owners of all kinds of smartphones to relate for their real-world experiences with buying, using, and troubleshooting the devices.

Perhaps the central finding of our survey is that smartphone owners have plenty of complaints, yet seem to willing to tolerate many of the shortcomings of their wireless service providers as well as the manufacturers of the phones. The survey asked the opinions of some 3500 smartphone owners during the month of October2009, and the results revealed mediocre or poor reliablity and ease of use in a number of major smartphone brands, and poor problem resolution and network service from some major wireless service providers. Surprisingly, however, smartphone owners registered noticably low levels of dissatisfaction about these deficiencies.

Note: The percentage numbers contained in our survey results carry a margin of error of 3.3 percent unless otherwise indicated.

Who's Hot, and Who's Not

The Apple iPhone, at least for now, is still the hottest and best-loved smartphone in use out. The carrier that sells and connects the device, AT&T, is, well, not so beloved. Verizon and Sprint are the best-liked wireless carriers in the eyes of the smartphone users we surveyed. (Note: Click on the thumbnail graphic to see the full-size image.)

The idea of pairing the best device with the best (compatible) network--Verizon--has been the subject of much speculation and debate over the past year, but Verizon's recent anti-iPhone advertising and the release of its "iPhone killer" Droid make that union less likely.

The good news is that both networks and devices are changing so fast that Apple's lockup with AT&T may not seem so important a year from now. Better devices running on faster networks are likely to enter the market in the coming months and years, especially if new competitors like the cable companies, WiMax providers, and dark horses like Google begin selling smartphones and wireless service.

Tech Support: Many Problems Go Unresolved

Our survey results show the level of smartphone users' satisfaction with service providers' technical support.
Smartphones are complex devices, and becoming more so. This increases users' reliance on service providers (and in some cases, handset manufacturers) to help keep the gadgets--little PCs, really--running and working properly. (Note: Click on the chart image to see our survey results on user satisfaction with technical support. And see "What Technical Support Means" below this section for more information on our measures of satisfaction in this area.)

Service providers are usually the first lines of support, shifting support issues over to the manufacturer of the phone only in severe cases. Service providers end up handling users' problems about 66 percent of the time, while smartphone manufacturers handle the problem only about 21 percent of the time, and electronics retailers do so only 1 percent of the time (the remaining 12 percent said "other" or "don't know").

Technical support must be judged on how well, and how fast, it helps users troubleshoot problems as they arise. The combined support systems (Web, phone, and in-store) of service providers and manufacturers fail to fully resolve smartphone technical breakdowns in a surprisingly high percentage of cases. Of the smartphone owners who reported having had to call tech support, 17 percent say their problem was never resolved, while another 21 percent say their problem was only "somewhat" (partly) resolved.

One carrier performed even worse than that: 19 percent of the Sprint customers who answered our survey said their problem was never resolved, while another 28 percent said their problem was only partly resolved.

Nevertheless, smartphone users report moderate satisfaction with the technical support they've received. On a 7-point scale (with 1 being "extremely dissatisfied" and 7 being "extremely satisfied"), readers ranked the four major service providers--AT&T, Sprint, T-Mobile, and Verizon--in the low 5 region, meaning they were "somewhat satisfied" with service provider tech support services. Specifically, Verizon and T-Mobile each scored 5.4 out of 7; Sprint and AT&T each scored 5.2.

What the Measurements Mean

The following describes the criteria we used in our survey to measure service providers' technical support.

Average phone hold time: Based on the average time smartphone owners waited on hold to speak to a phone support representative.

Phone service rating: Based on a cumulative score derived from smartphone owners' ratings of several aspects of their experience in phoning the company's technical support service. Among the factors considered were whether the information was easy to understand, and whether the support representative spoke clearly and knowledgeably.

Problem was never resolved: Based on the percentage of survey respondents who said the problem remained after they contacted technical support.

In-person service rating: Based on a cumulative score derived from phone owners' ratings of several aspects of technical support received at a service provider's retail location. Among the factors considered were the ease of getting a representative's attention in the store, and the knowledge, fairness, and attitude of the rep.

Overall service rating: Based on a cumulative score derived from smartphone owners' responses to a series of questions that focused on 11 specific aspects of their experience with the company's service department.

Next: Phone and In-Store Support

Satisfied With Phone Support

To access tech support, smartphone owners experiencing problems usually elect to call (as opposed to using Web-based or in-store support). And in general, smartphone users say they're satisfied with the quality of the phone support they get from service providers.

We asked users to rate the support departments on the quality of the information given, on the reps' communication skills and problem-solving abilities, and on other criteria. Smartphone customers from all four major carriers responded uniformly, and rather positively. Sprint customers gave its tech support an overall score of 5.6 out of 7, while AT&T, T-Mobile, and Verizon customers graded those companies' tech support a 5.7 out of 7.

Our study shows that, in 43 percent of cases, smartphone support issues are resolved through phone support alone, and, in almost 50 percent of cases, after contact with just one phone rep.

Javier D. Margo Jr., Rio Grande City, Texas
Still, many of our survey takers expressed frustration. "It took too long to solve the problem," says Sprint Palm Pre user Javier D. Margo Jr. "I spent five hours continuously either chatting online or speaking directly, often to two different reps (one from Sprint, one from Palm) at the same time."

"If they don't know the answer or how to solve the problem, [they should] take the person's number, find the solution to the issue, then call the person back," suggests one AT&T customer who asked that his name not be used. "That way," he continues, "the customer doesn't have to wait through transfers and holds while somebody goes to ask somebody else."

Sprint, Verizon Do Better on Hold Times

First impressions count, and often the first taste of service provider support that smartphone owners receive is the length of time they have to wait on hold to speak to a rep.

We saw some real differences in the hold times reported by customers of the four major wireless service providers. Sprint and Verizon seemed to excel here; their customers reported average hold times of just 4.4 minutes and 4.7 minutes, respectively. Meanwhile, AT&T and T-Mobile customers told us they waited on hold an average of 6.5 minutes and 6.6 minutes, respectively.

Smartphone owners as a whole have mixed opinions about the acceptability of hold times. About 49 percent of our survey takers said hold times were completely acceptable, while 33 percent called them somewhat acceptable, and about 18 percent considered them unacceptable.

In-Store Support: You May Have to Wait

Smartphone users are a bit more satisfied with the tech support they receive in service provider stores, compared with phone or online support. For 15 percent of smartphone problems, a visit to an AT&T, Sprint, T-Mobile, or Verizon store provided a resolution to the issue.

When asked about the attitude, knowledge, problem-solving skills, and timeliness of in-store reps, subscribers graded staffers in AT&T and Verizon stores fairly well at 5.7 and 5.3, respectively, on our 7-point scale.

Our results suggest that Sprint and Verizon may not have enough staff in stores to help customers quickly. Thirty-five percent of Sprint customers felt that they were kept waiting in the store too long before getting attention. About 28 percent of Verizon customers complained of long in-store wait times.

Next: Network Service

Sprint, Verizon Networks Are Best Liked

Without the networks that connect handsets to the Internet, smartphones are little better than paperweights. This network connection is where service providers make most of their profit, and customer perceptions of the performance of those networks is hugely important to service providers. (Click on the chart image to see our survey results on users' satisfaction with providers' network service.)

Big wireless companies make grand claims like "America's fastest" and "the nation's most dependable," based largely on their own research, and that provided by paid third parties. Since commissioned studies can lead to biased results, we turned to real-world wireless customers to learn about the value of the major wireless services. Specifically we asked smartphone owners to grade the speed and reliability of their Internet connection, and the quality of their voice calls.

We found that among the big four carriers, Sprint's customers are happiest overall with the performance of their network. In particular, Sprint customers like the speed and reliability with which the Sprint network connects them to the Internet. More than 70 percent of Sprint's customers said that they were "very satisfied" or "extremely satisfied" with the speed of their smartphone's internet connection. Sprint scored even better on the reliability of its network when connecting: Almost 80 percent of Sprint subscribers in the survey said that they were highly satisfied.

Verizon won the day on pure voice-call quality, with almost 85 percent of its subscribers saying they were highly satisfied with the sound and connection quality of calls. Sprint came in a close second, with almost 85 percent of its subscribers saying they were either very or extremely satisfied with the quality of voice service.

The AT&Tand T-Mobile networks didn't fair too badly in our survey either. Both scored well, between the 70th and 80th percentiles (along with Sprint and Verizon) in terms of overall satisfaction with network service.

AT&T and T-Mobile did stumble in a couple of key areas, however. Only 59 percent of T-Mobile subscribers described themselves as highly satisfied with the speed of their wireless Internet connection. T-Mobile's competitors all scored above the 65th percentile in this area.

AT&T fared poorly on voice-call quality. Though 74 percent of the company's subscribers in our survey said they were completely satisfied with call quality, AT&T's competitors each scored well above the 80th percentile on this measurement.

Some anecdotal evidence suggests that AT&T's voice network performance is highly variable from coverage area to coverage area, and even from time to time, as in the example provided by AT&T subscriber and iPhone user Tammy Zink:

Tammy Zink, Cape Coral, Florida
"A year ago, when I first got the phone, I was worried about AT&T's reputation, but AT&T's call service was fabulous--no dropped calls, great reception all the time. However, in past six months, almost every call is dropped, and reception is lousy. I now cringe when the phone rings or I have to make an important call."

Over the past two years, smartphones, and the iPhone in particular, have caused wireless Internet usage to rocket upwards. To better accommodate the demand, AT&T and other providers have been "tuning" their networks. As network resources are shifted to bolster one service, however, another service may suffer--in this case, voice service.

The findings of a new survey by Consumer Reports seem to agree with our results. Consumer Reports asked subscribers of the major wireless providers in 26 cities for their thoughts about their cell phone service, and found that AT&T subscribers were the "least satisfied" of all in 19 of those cities.

AT&T had the same general reaction to Consumer Reports' findings as it did to ours, saying neither study paints a true picture of U.S. wireless users' experience.

AT&T points to its own third-party research showing that its network is not only the fastest in the United States, but is also among the best regarding dropped calls. "In one of the most common measures of reliability--dropped calls--AT&T's national performance is within two-tenths of 1 percent of the highest score among major providers, with only 1.32 percent of calls dropped nationally," says AT&T spokesperson Jenny Bridges in an e-mail to PCWorld. She adds: "That translates to a difference of less than 2 calls out of 1000."

So far, AT&T has suffered most from the network-taxing effects of escalating data service usage, but it may be Verizon's turn in 2010. With the release of its impressive Droid phone (and others), Verizon might find its wireless customers demanding unprecedented amounts of Web connectivity.

Next: Ease of Use

Ease of Use: Comparing Apples to Lemons?

According to our survey takers, Apple's iPhone remains in a league of its own when it comes to ease of use. Part of the vast appeal of the device is its design--inside and out. Aesthetically pleasing, the iPhone's user interface looks simple and intuitive, helping owners access its many functions. (Click on the chart image to see our survey results on the ease of use of smartphones from various major manufacturers.)

We asked smartphone owners ten questions about the ease of use their handsets, including how satisfied they were with the ease of using nine of the most important functions--things like browsing the Internet, syncing data, sharing files, taking pictures, and playing music and video.

The iPhone scored higher than its peers in all but one of those areas (users gave the iPhone only an "average" score on the quality of the photos and videos it shoots). When asked about their overall satisfaction with their phone's ease of use, 82 percent of iPhone owners said they were "very" or "extremely" satisfied with their device.

Apple Will Be Challenged

But this state of affairs won't last. A new wave of Google Android-powered phones, such as Verizon's Droid, will almost certainly rival the iPhone in ease of use. At the time of our survey, however, the new Android phones were not in use in high enough numbers to affect our results. Next year's survey will likely tell a different story.

"According to the feedback I've gotten from companies that design user interfaces for smartphones, the consensus is that the iPhone interface has gotten a little dated," IDG mobile device technology analyst Will Stofega says.

With Android, Stofega adds, Google "has done a very nice job on its interface, and I expect it to be very popular, depending on how it's implemented."

And Android phones won't be the only ones gunning for Apple's ease-of-use throne in the coming months. Stofega points out that Nokia is pushing hard to develop its own Symbian operating system to meet and surpass the ease-of-use standard that the iPhone set. New phones using this updated and touchscreen-friendly operating system should begin appearing during 2010.

But today, Apple simply dominates its competitors in design. Apple's closest rival in this area was Nokia, which earned just one higher-than-average score--that for the ease of use of the cameras in its smartphones. Nokia placed second in "overall reliability," but with a paltry 59 percent of owners reporting high satisfaction with ease of use.

Among those fairing most poorly with survey takers in ease of use were HTC, Motorola, RIM (BlackBerry), and Samsung. Asked about the overall ease of use of their devices, 55 percent of BlackBerry users were completely satisfied, while just 45 percent of HTC owners, 43 percent of Samsung users, and 42 percent of Motorola phone users were completely satisfied.

Samsung owners gave their phones worse than average marks in all ease-of-use categories save two--both having to do with the video cameras in the phones. (Samsung chose not to release any comments on the results of our survey.)

Motorola users expressed frustration with the ease of setting up their phones, syncing data with computers, browsing the Internet, and locating and playing back music and video. Motorola phones also received below-average marks on the sound and video quality they produce on playback.

Common complaints among all users included difficult -to-use keyboards, slow-to-respond touchscreens, clumsy file management, hard-to-use cameras, poor sound quality, and difficult or slow syncing. Others complained of user interfaces that are hard to customize, cameras that take pictures when not asked to, operating systems that don't multitask, and browsers that won't play Flash content.

Next: Reliability

Reliability: Operating Systems Fail the Most

The more technology you pack into a device, the more things can break. And smartphones must be the poster child for carrying a dizzying number of bells and whistles inside the shell--microphones, speakers, touchscreens, cameras, an operating system, applications, an accelerometer, compasses . . . the list goes on. (Click on the chart image to see our survey results on device reliability from major manufacturers.)

With all that stuff in there, you would expect high rates of component and software failures. And you would be right. In our survey, 31 percent of smartphone owners reported one or more significant problems with their device before it was two years old.

And readers told us that when something fails in a smartphone, in about 35 percent of cases it's the operating system.

"My biggest complaint is that the response is slow and the interface is clunky!" says Sprint HTC Touch Pro user John Abercrombie. "Also, sometimes it 'locks up' the OS and the only way out is to remove and reinsert the battery."

Sprint customer and Palm Treo user Duane Calvin says his phone's OS "now reboots itself several times a day; I have no idea when it has shut down."

Of the major smartphone brands, HTC and LG phones had higher rates of OS failure than their peers. Of HTC owners who reported a problem, 44 percent said the culprit was the operating system software. Thirty-nine percent of problems with LG devices could be blamed on the OS.

After the OS, breakdowns in miscellaneous features such as BlueTooth connections or GPS radios and compasses were the cause of 24 percent of problems, surveyed smartphone owners said.

These failures, survey takers told us, were especially prevalent in HTC and Samsung smartphones. In HTC devices, a feature failure was the problem in 29 percent of cases; in Samsung phones, in 30 percent of total problems reported. Meanwhile, owners of Apple, LG and Palm phones reported lower-than-average numbers of feature breakdowns.

Battery issues accounted for about 11 percent of reported problems. Only HTC, LG, and Samsung had significantly higher incidences of battery problems than the average of all handset manufacturers. Apple and Palm owners reported marginally less battery trouble.

We also asked about severe problems that rendered phones impossible to use. Among all those who reported problems (regardless of the type of phone they use), about 13 percent said their problem was "severe" enough to flatline the phone.

LG owners reported fewer severe problems. LG phones developed just as many problems as other phones, but only 8 percent of those problems rendered the phone impossible to use. Palm users reported a higher incidence of severe problems, at 19 percent--or six points above the average.

Still, 64 percent of smartphone owners say they are "very" or "extremely" satisfied with the reliability of their devices.

Seventy-four percent of iPhone users said they were highly satisfied overall with the reliability of their phone--the highest score of any smartphone. Samsung owners, meanwhile, reported the lowest rates of overall satisfaction, at only 55 percent.

For Now, They're Still New Toys

Smartphones are a relatively new technology. The first iPhone, for example, hit the market only a few years ago--on June 29, 2007, to be exact. The devices, and the networks that connect them, have certainly improved since then. But as our survey results show, in many ways, they're still a work in progress.

Users report high numbers of technical breakdowns in their devices, a poor record of completely fixing those problems by tech support departments, difficult-to-use phones, and slow and/or unreliable wireless network coverage. Yet, as our survey also shows, satisfaction levels remain strangely high among users in those same areas.

Why is that? It may be that consumers are still a bit dazzled by the allways-on, connect-anywhere technology that smartphones are. So dazzled, perhaps, that they're willing to overlook a few basic shortcomings in the devices, as well as in the companies that make, sell, and support them--even as users pay a high premium every month to own one.

IDC's Will Stofega offers the classic example of this. The iPhone's OS does not allow you to cut and paste text or images--from an online news article to an e-mail, for example. This, of course, would never be tolerated in another type of computer. Are iPhone users caught in Steve Jobs' "reality distortion field"? Maybe so. But our survey suggests that that distortion field isn't the sole property of just one smartphone maker, but, perhaps, of all of them.

In short, smartphone owners should expect more. Here's hoping that, as smartphones become less novel, expectations will rise--and consumers will begin to see the same reliability and service levels that they routinely demand from other devices.

For comprehensive coverage of the Android ecosystem, visit Greenbot.com.

Subscribe to the Best of PCWorld Newsletter

Comments