Friday, March 27, 2009

Looking at the Q1 2009 Survey Results--Time for a Step Back

After looking at last quarter's survey results, I started to get concerned about the validity of our high "customer satisfaction score". Although our responses have been rather consistent, the drop in satisfaction rate between Q3 2008 and Q1 2009 got me a little worried--mainly because it was a 4% drop.I started to re-examine the data and it appears as though there are many factors that should be considered when looking at the “drop” in “satisfaction”.

The first flag was what appears to be a drop in survey participation over the past few quarters. During the first installment of the survey we had 30,272 respondents. That number has dropped by nearly 50%, with 15,823 respondents in Q4 of 2008 and 15,682 respondents in Q1 2009.

Here are a few possible explanations for this “apparent” drop in participation over time:

1) The novelty of the first survey encouraged more participation.

-Even though we have 100 million people seeing the “What’s New” page with every update and we only allowed the survey to appear to 10% of that population, there is a self-selecting bias in any opt-in survey.
-The further we move away from Firefox 3 launch, the lower the enthusiasm for the product. I'm not saying that people have stopped loving Firefox 3, I'm just saying that the further we get away from the launch, the less likely people are to be willing to take a survey to tell us they love the product. (Its’ awesomeness, in essence, has become the status quo).

2) The first survey was released with 3.02 and 3.03 (released a day later), which may have inadvertently increased the number of people who saw the survey link, and consequently, may have artificially doubled our participation with the survey’s first release.

I think option #2 is really the only one that makes sense in this context. It’s easy to see how (self-selecting) individuals would have taken the survey twice and thus inflated our initial “customer satisfaction score”.And since the past two surveys have been consistent in participation and responses, I think that it may be worth completely throwing out the data from the first survey.

Other concerns about the survey came from looking at the responses to certain questions.

For example: "How long have you been using Firefox?"

About 50% of respondents to this question say they have used Firefox for more than two years. That response rate doesn't quite make sense to me if we are trying to get responses from a cross-section of all of our users. Those numbers would make more sense coming from our user-base or long term, loyal, customers. Other survey questions that were similarly concerning: "Have you installed an Add-On?" (50% said yes), "On average, how much time do you spend on the web?" (2-5 hrs a day). The responses to both of these questions again seem to indicate that a large number of the respondents to our survey were part of our core fan-base as opposed to part of a more general cross-section of users. (You can take a look at the Comscore data Dave Bottoms gathered last year if you want to know about the average Internet user.)

Another red flag was the number of people who started but did not complete the third installment of the survey. The survey released in December ’08 had a total of 37 “abandoned” surveys. The most recent installment of the survey was “abandoned” 5419 times. Why the huge jump in abandonment rate? I think it comes down to two changes we made to the survey:

1) We increased the number of questions from 8 to 10.

2) We reformatted the survey so that the questions were split equally on two pages.

I think both of these changes, though well intentioned, really took us away from our main goal of this program—to create a simple, short survey that would efficiently allow us to get a better picture of all of our customers.

To review, here are the takeaways:

1) Q32008 Results and “Satisfaction Score” are probably inflated and should be disregarded.

2) The surveys tend to be taken mostly by long-term, loyal Firefox users. We need to find a way to get responses from a more diverse cross-section of Firefox users.

3) We need to pay attention to the design of our survey to ensure we don’t scare users off with the number of questions/layout of the survey.

So, what next? I think our next steps are to step back and try and get a better understanding of what the data we have collected in the past nine months *really* means. That also means trying to figure out what we have previously called our "Customer Satisfaction Score" is really measuring. Is it a customer loyalty score? Is it an enthusiasm score? At this point, I'm not really sure, mainly because I don't know the extent of the biases in our data.

It would be great to hear your thoughts and suggestions: what can we do differently? How do we get a better sample of all types of Firefox users? How can we try to minimize bias?

Clothing has always been a thing that has been given a great importance by human beings. It displays the attitude that the people exhibit. ed hardy clothesed hardy shirtsLots of brands have been existent in the clothing industry and a famous one among them is the Ed hardy clothing brand.ed hardy jacketsed hardy hoodiesThe brand got its name from the famous American tattoo artist Ed Hardy. He was a very famous tattoo artist and has published many books on tattooing techniques. ed hardy bootsed hardy polo shirtsBut his tattooing turned into a brand by the efforts of a company called Christian Audiger. This company was a very famous and very powerful company in the field of clothing. ed hardy shoesed hardy jeansThey felt that it would be appropriate to create a brand called Ed hardy and use Hardy's art as the main selling point for the brand. This venture had turned out to be a very successful one and Ed hardy clothing is one of the most famous brands in the clothing industry.ed hardy outerweared hardy long sleeve shirtsThe brand became very famous because it was worn by many famous celebrities like Madonna, Britney spears and also Sylvester Stallone. ed hardy bagsed hardy winter bootsThe brand has clothes for men, women, and kids. They have also diversified their business by having a lot of accessories to support their clothing business. ed hardy love kills slowly shirtsed hardy love kills slowly shoesed hardy love kills slowly bootsfor men they have a variety of products such as active wear, denim, outwear, swim trucks, sweaters, t-shirts, tops etc. ed hardy trousersed hardy mensthe accessories include things such as belts, caps, scarves, shoes, socks, jeweler, sunglasses, ties and even wallets.The art works is a unique combination of American and Japanese cultures. The unique selling proposition of the brand is the way in which the company has used the art works of Ed hardy.ed hardy womens ed hardy t shirtsed hardy sunglasses

Nice work and thanks!RunningAdidas currently manufactures several running shoesNike shoes, including the adiStar Control 5, the adiStar RideCheap nike shoesDiscount nike shoesthe Supernova Sequence and the Supernova Cushion 7, among others. Nike shox r4nike shox shoesAdidas also uses kangaroo leather to make their more expensive shoes. Association footballOne of the main focuses of Adidas is football kit and associated equipment. puma catcheap sport shoesAdidas also provides apparel and equipment for all teams in Major League Soccer. Adidas remain a major company in the supply of team kits for international football teams. cheap nike shoxcheap nike maxAdidas also makes referee kits that are used in international competition and by many countries and leagues in the world. In the United States, referees wear the Adidas kits in MLS matches even though the primary referee supplier is Official Sports. nike tn dollarnike running shoesThe company has been an innovator in the area of footwear for the sport with notable examples including development of the Copa Mondial moulded boot on firm dry pitches for forty years. nike air max tnpuma shoesAdidas became renowned for advancing the "Predator" boot design.This design featured a ribbed rubber structure for the upper leather of the shoe, used to accent the movement of the ball.discount puma shoespuma mens shoesThe Predator also features the Craig Johnston invented "Traxion" sole. As the development and popularity of Football continued Adidas played a leading role in shaping the style of the play itself. puma running shoespuma shoesFIFA, the sports governing body, commissioned specially designed footballs for use in its own World Cup tournaments to favour more attacking play. ghd hair straighteners mk4hair straightnersghd iv styler hair straightener ghd hair straightnerscheap ghd hair straighteners

I would like to thank you for sharing your thoughts and time into the stuff you post!! Thumbs upNow it will be very cold in winter, in order to keep warm yourself, moncler company has provide jackets for you, so it is better to own moncler jackets yourselif as soon as possible. Eeveryone love fashion clothing, polo ralph lauren is very popular all over world, that is my dream to get one.We know Ray Ban by America soldier, all of them wear ray ban sunglasses when they walk on the road, it is fashionable for you. Most of people like to wear jeans, it is very modern when wearing true religion jeans in the street.

For this motive to provide every product to the customer, and to accomplish the all needs of the people we introduce an authentic and valuable E-marketing place where mushroom of beautiful and well equipped houses and accommodations are provided by the so worthy and trusty E-marketing place which is geboon.com .see Free Classified Ads