Tag Archives: survey

Hey there! Sorry for the long delay since my previous post; it’s been a busy past few weeks. Since coming back from the summit, my internship has been full of twists and turns, leaving me little time to write.

I’ve been assigned to work on a whole lot of different things. To give equal exposure to all of them, I’ll write several posts to update you on my work these past few weeks.

One of my big projects thus far has been designing an industry-wide web developer survey. Some of you might have seen one of my previous blog entries about the results of the survey we distributed this past March. The findings, while interesting, don’t paint a complete picture of the state of the open web and the developer tools you all like to use. Thus far the surveys we’ve released – the one previously mentioned and one that was released last November – have been distributed via Mozilla channels and have only really inquired about Mozilla platform technologies. We hope to change that.

We hope to get a wider snapshot of the web development community, not just for us, but for you. A panorama, if you will.

The survey will be released in a few weeks – sometime before the end of August – so check back frequently for the opportunity to take the survey and help us paint the picture of the open web. Thus far I’ve iterated through several drafts of the survey and talked to several market research consultants to determine how to best distribute and design the survey. It will be ready soon!

On a more interesting note, I’ve also been reaching out to infographic designers. Although the data obtained from the survey will certainly help us, we want it to help you, too. We want to present our data and conclusions in an aesthetically awesome way. Right now, I’m talking over our goals and the possibilities of an end design with several artists. They all have some pretty good stuff going on; they’ve won awards and made some pretty interesting designs. Stay tuned!

Because we expect such a large audience to be taking the survey, there has been a lot of pressure to get every word and every question right. I never realized just how much goes into designing a questionnaire: figuring out the average time spent to read and answer each question, the best way to structure a question, the most efficient reconciliation of details and the big picture, etc. Given the impact we expect this survey to have on our engagement efforts, the survey must determine with exact precision the information that we’re looking for. Such a statistical conquest, I have discovered, isn’t as easy as it looks.

More blog posts about the other things I’m working on will be coming soon! Thanks for reading.

In this post we’ll share the results of our latest survey and provide some data and insights from all the great feedback we have received. We hope this will help us better understand developers’ needs and continue to build out the Mozilla Developer Network to better engage with them.

Recap of the November 2009 Survey

As a refresher, here is a summary of the initial November 2009 survey. The main takeaways were as follows:

Firefox is the most common browser used first by developers for testing their work, primarily because of the many developer tools and Add-ons available for Firefox and Mozilla’s strong support for web standards.

Firebug stood out as the most popular developer tool and many developers agreed that it is “absolutely essential for development.”

Results from the March 2010 Survey

Our second developer survey received 2,267 responses over two months from late March to early June.

For this survey there were several common questions from the previous one – about browser preferences – so that we could measure satisfaction over time and follow trends in browser adoption following the release of Firefox 3.6.
We also responded to the overwhelmingly enthusiastic praise of Firebug by asking for feedback on the most recent release. By obtaining more specific feedback on the latest version of Firebug, we hope to address any remaining issues to make it an even more effective and attractive tool for developers.

Developer Browser Preferences

We did not see much change in browser preferences from November 2009 to March 2010. Even considering a new iteration of Firefox and increasing competition from other browsers, the data still reflect general browser market share trends: the latest versions of Firefox and Chrome gained at the expense of older versions of IE.

The following data show the primary browsers developers use to test their websites:

Which browser(s) do you test your web sites against to make sure they are compatible?

November 2009

Tested Against

Percent

Firefox 3.5

82.03%

IE 8

74.63%

IE 7

68.23%

Chrome

60.23%

March 2010

Tested Against

Percent

Firefox 3.6

80.17%

IE 8

70.23%

Chrome

62.37%

IE 7

56.18%

The numbers are unlikely to shift drastically because regardless of developers’ personal preferences, the top browsers listed above are the ones most commonly used by regular users, as demonstrated by browser market share data. Thus it makes sense that developers test against these browsers most often.

Firefox isn’t just the most popular browser for developing, according to the data below; it’s also popular for personal usage. It should be noted, however, that this survey was distributed primarily though Mozilla-related channels, so there might be some bias in the results; we address this problem later in the report.

Note also that the vast majority of developers were quick on the uptake switching from 3.5 to 3.6.

Have you tried Firefox 3.6?

Upgrading to Firefox 3.6

Although the vast majority of developers switched from 3.5 to 3.6, there were still some who did not. In fact, roughly 10% of the March survey respondents still used 3.5. The primary reasons for the continued usage on 3.5, however, had little to do with personal preference. Roughly half of those who still used 3.5 did not even know that 3.6 was released, while the other half responded that they didn’t have time to update.

One interesting note from the results was that a lot of developers still using Firefox 3.5 were on Linux distros that did not yet have the Firefox 3.6 package available at the time they took the survey. Given that 23% of respondents develop on the Linux platform, that could explain why so many had not made the jump.

What operating system do you mainly use for your development environment?

Unfortunately the data lacked specific details on why developers did not know about 3.6 or why they did not find time to update to the next version. With that said, regardless of the reasons, it is clear we should focus on facilitating the update process for future iterations of Firefox. We will start by promoting early betas and announcing new Firefox releases on the MDN website.

Feedback on Firebug 1.5

We all know that Firebug is important to developers, so we wanted to focus on getting Firebug-related feedback for this survey. Comments about Firebug echoed those from the November survey: the consensus is that “Firebug is the single most important tool in web development.” Indeed Firebug is considered such a critical tool in the developer’s arsenal that several of you asked us to package Firebug as a built-in component of Firefox instead of just as an Add-on; some users call the duo the “triple F”: Firefox and Firebug.

While the 1.5 release was a huge improvement from previous versions, developers weren’t shy about sharing their frustrations and feature requests. Many noted occasional stability issues, such as when Firebug is finding elements in document trees or taking certain Ajax requests. Additionally, we received several suggestions for improving the user interface, such as incorporating more Firefox-native design elements, increasing the size of the “Deactivate Firebug” button, and removing elements like the “Quick Info” and “CSS Overview” boxes.

One data point that stood out was that about 22% of developers had not tried Firebug 1.5 at time they took the survey. Considering that it was released in January and the survey was open from March to June, we would have expected that the adoption of such an important tool to be higher than what the data suggests. This presents an opportunity to better communicate new releases and raise awareness among developers to make sure they are up-to-date on the latest development happening with Firebug.

Have you tried Firebug 1.5?

For those of you who are interested in learning more about documentation, features, and related news, visit the Firebug website. Along with regular blog posts from the team, the site will host community forums for discussions with community and Firebug team members and offer ways to get involved if you want to contribute to the project. Firebug 1.6 development is well underway and we’ll continue to see it evolve to meet web developers’ needs.

Next Steps

Thus far your feedback via our first two surveys has been invaluable as we strive to build a better open Web through the Mozilla Developer Network. We now have a decent snapshot of the type of development you do, the technologies and tools you use, and the browsers you prefer for both work and personal use. The insights we have gained will help us refine the MDN roadmap and guide the programs we develop over of the next year to better engage with developers and build out the MDN website to meet your needs.

Our next step is to move beyond these initial data to get a larger snapshot of the developer community. We will start by designing an informal survey to get a grasp of what developers here at Mozilla and within our immediate community have to say about the tools and resources they find the most valuable for their work. We plan to follow that up with more research on the web developer community as a whole by distributing an industry-wide survey.

The goal for Q3 is to design a survey that can be distributed to a more diverse and unbiased global developer community. We plan to launch the new survey later this year and will look to you to help us reach the right mix of developers. Ultimately our goal is to have the clearest possible picture of the global web developer community and how it views the current “state of the Web”.