Why You Shouldn't Trust the uSamp Verizon iPhone Survey

Saturday’s are usually slow days in the tech news world, but thanks to a new survey (PDF) from online research firm uSamp that argues that 44% of Verizon Android users and 26% of AT&T iPhone users will wait in line to get a Verizon iPhone on the first day it goes on sale, we have something fun to chat about.

Here are the most interesting results of the survey: 44% of Verizon Android users are very likely (19%) or somewhat likely (25%) to buy an iPhone on February 10. About 26% of AT&T iPhone owners are either very likely (8%) or somewhat likely (18%) to get in line for a Verizon iPhone on day one.

Now, given the huge demand for Verizon iPhones, there can be little doubt that it will be a huge hit for both Verizon and Apple. I’m not doubting that at all. What I do doubt, however, are the survey results from uSamp. Here are some of the problems I see with this survey and the way it’s currently portrayed:

[list]

How does uSamp recruit panelists? That one was a bit harder to find out, but as far as I can see, panelists are recruited through Opinion Place – a service owned by the same company as uSamp that pays panelists in Paypal credits, Amazon gift cards and American Airlines frequent flier miles. Basically, these people take surveys to make a buck – not exactly a trustworthy methodology.

If you read all the way to the end of the survey, you see that a lot of people have “second thoughts about switching.” Once asked about these second thoughts, the number of switchers from AT&T to Verizon suddenly drops from 26% to 15%. Same for Verizon RIM and Android customers. The number of potential switchers drops from 54% to 25% after the survey asks if they still plan to switch, no matter the cost of switching, concerns about network speeds and the possibility that lots of iPhones could slow down Verizon’s network. This once again makes me wonder of the panelists who took this survey.

Related to this: I never trust a survey that asks consumers about future purchase decisions (see the ChangeWave iPhone survey from January for an example and more explanation). Saying ‘yes’ about buying something in the future is very easy. Doing it is a lot harder (especially if early termination fees play a role).

as far as I can see, this is only the second survey uSamp’s published under its own name. That doesn’t have to be a negative, but one could argue that uSamp doesn’t have a lot of experience in running its own surveys. The first survey – about consumer sentiments ahead of the holidays – seems quite reasonable, but maybe those questions were more in line with topics the paid uSamp panelists knew something about…

[/list]
I’m happy to be proven wrong here. As I said above, I’m not doubting that the Verizon iPhone will be a huge hit and that lots of people will switch. I just think this survey isn’t the best way to prove that.

uSamp provides online survey respondents to almost every well known market research firm worldwide. In the same way that people are paid to sit in focus group rooms or sit through telephone surveys, online surveys also reward people for their valuable time. Online sampling, has been proven to be statistically accurate and representative of populations. uSamp has no vested interest in Apple or Verizon. In a short time, the actual sales figures will tell the truth.

uSamp surveyed 116 Verizon Android subscribers. 44% said they were "very likely" or "somewhat likely" to switch to an iPhone. When pressed about second thoughts due to a range of issues including cost the number who "still plan to switching" dropped to 21%. For a sample size of 116 the margin of error is about +/- 10 % points. That means as few as 11% or as many as 31% may actually switch.

That's about the same as the churn.

All that is assuming that uSamp's survey methods are sound.

5pts

The thing is, this survey is not an outlier, it jibes with other surveys which for years have had similar results. This is exactly the kind of rebalancing of the US market that everyone is expecting. Surveys have been saying for years that AT&T would lose 25% of their iPhone users right away. Surveys have been saying for years that half of Verizon's smartphone users wanted an iPhone but made do with whatever Verizon had in order to stay on Verizon. These surveys were key to the argument that Verizon's network is better. People were on AT&T just to get iPhone. People were passing on iPhone just to get Verizon.

When Verizon themselves asked their own customers "who wants an iPhone?" that number was larger than all of their smartphone customers combined. There is huge demand for iPhone on Verizon. There is 4 years of pent-up demand.

On AT&T, iPhone:Android is 15:1, while on Verizon, iPhone:Android is 0:millions. A ton of Android users are going to go to iPhone to even that out. Not saying Verizon will get to 15:1, but they are going to get to 2:1 or 4:1 very quickly.

iPhone being available only on AT&T distorted the US market in 2 ways:

• AT&T sold way more iPhones than they "should" have

• Verizon sold way more Android phones than they "should" have

So we should expect to see exactly what this uSamp survey predicts: iPhone users moving from AT&T to Verizon, and Verizon users moving from Android to iPhone. 25% and 50% respectively are eminently reasonable numbers.

You're saying you doubt two studies, one from a new group and one from ChangeWave, which you blow off as having a "notoriously unreliable" track record.

That's not really the case though. ChangeWave has been plotting out future demand from customer surveys rather accurately. I know, because I cover their reports.

I've also covered patently false reports of "surveys" that were clearly intended to draw attention to a dog, such as Microsoft's ABI study that showed people a picture of the Zune and asked if they might buy one, and then turned around and reported some large percentage of iPod users were going to switch back in 2007.

Well guess what? That phony survey had zero impact, despite being throw around by the media as fact. Nobody bought the Zune just because they'd heard people would be switching from the iPod.

So my question for you is, what falsehood are you working so hard to overturn here? And can't you do a better job than saying "I think this outfit is not legit because I haven't heard of them and I don't know how they collected their data, and, while I have no proof otherwise, I think my hunch is more accurate than their study."

What are you adding here, better data? Better process? More noise? Yeah, that's about it. We now have the collective opinions of 700+1. I don't know who you are, but you haven't convinced me that your opinion is 700 times as important or relevant as people on the street.

5pts

In the long run, I totally agree. Lots of people will switch But uSamp's data says all these people will line up on day one. That doesn't feel right. That would be more AT&T users lining up for the VZ iPhone than people who lined up for the iPhone 4 launch...

5pts

Well, a few thoughts:

a) I didn't say that I think ChangeWave is notoriously unreliable - I said surveys that ask consumers for future purchase decisions are unreliable for the reasons mentioned in that post and here.

b) re: 700+1 opinions: my opinion about the VZ iPhone isn't relevant here and plays no role in this post at all. No hunch involved.

c) I think I explain how uSamp gets it's data and why I don't trust it.