We shipped two chemspills over the weekend: Firefox 36.0.3 and 36.0.4 (plus Firefox ESR 31.5.2 and 31.5.3)

Firefox 37 is scheduled to be released on March 31. That’s next week! For Firefox 37 Beta/Release, please report any keyboard related issues on OS X such as the keyboard stopping to work and requiring a browser restart.

MemShrink (njn)

jemalloc3 has been disabled again on trunk due to Windows crashes. The heap allocator is a fundamental thing and changing it in a complex program like a browser is hard.

Thanks to Jim Blandy, jemalloc (the old version currently used in Firefox releases, not jemalloc3) is now used as the allocator for the JavaScript shell, instead of the system allocator (bug 1134039). This makes the shell configuration closer to the browser configuration, which will make arewefastyet.com results more representative. But we don’t yet have data on whether those results have changed significantly due to some OS X problems (bug 1146267). It’s worth repeating: changing the heap allocator is hard.

about:memory now has a “resident-peak” measurement (bug 1145007) on Unix (Linux, OSX, *BSD), which measures peak physical memory usage for the process. It may be useful for detecting short-lived spikes in memory usage.

The new Reader View feature was causing high memory usage on desktop Nightly, but has been fixed in bug 1139678.

Media

In Firefox 37, the Media team is shipping the MSE API (Media Source Extensions, bug 778617) to improve YouTube’s HTML5 video playback on Windows. YouTube will now default to HTML5 video instead of Flash on Windows. MSE for OS X is being tested in Aurora 38.

Shumway

The Shumway team continues to focus on improving startup performance and Flash ad rendering.

Only two weeks and two beta builds until Firefox Beta 37 is released. The Tracking Firefox dashboard shows tracked bugs for the Aurora and Beta releases.

OrangeFactor

RyanVM sends a major shout out to Tim Taubert for being on an orange fixing tear last week.

MemShrink

Mike Hommey enabled jemalloc3 (bug 762449) on Nightly. It’s configured to not ride the trains for now. There are a number of perf regressions and crashes that Mike is diligently working through.

Media

The media team is close to shipping Media Source Extensions (MSE bug 1083588) for YouTube. MSE improves HTML5 video performance as YouTube transitions its default video player from Flash to HTML5 video. The team plans to ship MSE on Windows in Firefox 37, OS X in Firefox 38, and then Linux. Be on the look out for any YouTube bugs, such as videos that won’t load or get stuck rebuffering.

Performance

Telemetry and FHR measurements were unified into a single system on Nightly 39 (bug 1069869). The old FHR will still be around for a couple of releases. The unified Telemetry client-side is still being stabilized, so Telemetry from Nightly 39 isn’t reliable yet. Be very careful when interpreting Nightly 39 data!

Aaron Klotz created a tool for visualizing Windows attached input queues, as detailed on his blog. Attached input queues can cause odd problems like bug 1105386 where a page won’t load or render if the mouse is not moving.

With bug 1128768, we can now gather information about the amount of jank caused by different types of Flash content. This will be used to quantify the benefit of targeting Shumway at different types of Flash content.

Per-compartment CPU accounting (bug 674779) should land in the next week or two. Use it to report add-ons that use too much CPU or CPOWs (e.g. bug 1136923).

Shumway

The team’s current focus is playing Flash ads in Shumway instead of Flash. Using Shumway will reduce the number of Flash plugin instantiations, which is correlated with Flash deadlocks, and reduce page jank. Flash ads have also exploited Flash security bugs to install malware. Shumway should protect against many of those exploits.

Shumway now plays IMDb trailer videos on Nightly (bug 1137433) and substantial progress is being made on a verifiable sandboxed. An implementation of AS3’s meta-object protocol that should be pretty faithful to Flash’s an a new Shumway interpreter should land soon. This enables proper handling of SecurityDomains, a crucial Flash security feature.

The next merge date is March 30, just four weeks away. Firefox 37 and 38 releases have been moved up one week to avoid conflicts with holidays. Firefox 38 will be the “spring thing” Firefox release and the next ESR, retiring ESR 31.

A new Windows-only “address-space” memory reporter (bug 1134030) landed which may help debug Windows graphics and video memory usage. If you have other ideas on how to measure Windows graphics and video memory usage, contact Nicholas Nethercote.

IndexedDB performance work will also land soon: bug 866846 will enable SQLite’s WAL journal and bug 1112702 will change transactions to be non-durable. These SQLite options favor performance over durability like Chrome and IE do. They do not increase the risk of database corruption.

“Electrolysis” (or “e10s” for short) is the project name for Mozilla’s multi-process Firefox. Sandboxing tabs into multiple processes will improve security and UI responsiveness. Firefox currently sandboxes plugins like Flash into a separate process, but sandboxing web content is more difficult because Firefox’s third-party add-ons were not designed for multiple processes. IE and Chrome use multiple processes today, but Google didn’t need to worry about add-on compatibility when designing Chrome’s multi-process sandbox because they didn’t have any. :)

And that’s where our Firefox Nightly testers come in! We can’t test every Firefox add-on ourselves. We’re asking for your help testing your favorite add-ons in Firefox Nightly’s multi-process mode. We’re tracking tested add-ons, those that work and those that need to be fixed, on the website arewee10syet.com (“Are We e10s Yet?”). Mozilla is hosting a QMO Testday on Friday August 1 where Mozilla QA and e10s developers will be available in Mozilla’s #testday IRC channel to answer questions.

Install the add-on you would like to test. See arewee10syet.com for some suggestions.

e10s is disabled by default. Confirm that the add-on works as expected in Firefox Nightly before enabling e10s. You might find Firefox Nightly bugs that are not e10s’ fault. :)

Now enable e10s by opening the about:config page and changing the browser.tabs.remote.autostart preference to true.

Restart Firefox Nightly. When e10s is enabled, Firefox’s tab titles will be underlined. Tabs for special pages, like your home page or the new tab page, are not underlined, but tabs for most websites should be underlined.

Confirm that the add-on still works as expected with e10s.

To disable e10s, reset the browser.tabs.remote.autostart preference to false and restart Firefox.

Some e10s problems you might find include Firefox crashing or hanging. Add-ons that modify web page content, like Greasemonkey or AdBlock Plus, might appear to do nothing. But many add-ons will just work.

If the add-on works as expected, click the “it works” link on arewee10syet.com for that add-on or just email me so we can update our list of compatible add-ons.

If the add-on does not work as expected, click the add-on’s “Report bug” link on arewe10syet.com to file a bug report on Bugzilla. Please include the add-on’s name and version, steps to reproduce the problem, a description of what you expected to happen, and what actually happened. If Firefox crashed, include the most recent crash report IDs from about:crashes. If Firefox didn’t crash, copying the log messages from Firefox’s Browser Console (Tools menu > Web Developer menu > Browser Console menu item; not Web Console) to the bug might include useful debugging information.

To start the week, we reviewed Mozilla’s 2014 organizational goals. If a Mozilla team is working on projects that do not advance the organization’s stated goals, then something is out of sync. The goals where the JS team can most effectively contribute are “Scale Firefox OS” (sell 10M Firefox OS phones) and “Get Firefox on a Growth Trajectory” (increase total users and hours of usage). Knowing that Mozilla’s plans to sell 10M Firefox OS phones helps us prioritize optimizations for Tarako (the $25 Firefox OS phone) over larger devices like Firefox OS tablets, TVs, or dishwashers.

Security was a hot topic after Mozilla’s recent beating in Pwn2Own 2014. Christian Holler (“decoder”) and Gary Kwong gave presentations on OOM and Windows fuzzing, respectively. Bill McCloskey discussed the current status of Electrolysis (e10s), Firefox’s multiprocess browser that will reduce UI jank and implement sandboxing of security exploits. e10s is currently available for testing in the Nightly channel; just select “File > New e10s Window” to open a new e10s window. (This works out of the box on OS X today, but requires an OMTC pref change on Windows and Linux.)

Thinker Lee and Ting-Yuan Huang, from Mozilla’s Firefox OS team in Taipei, presented some of the challenges they’ve faced with Tarako, a Firefox OS phone with only 128 MB RAM. They’re using zram to compress unused memory pages instead of paging them to flash storage. Thinker and Ting-Yuan had suggestions for tuning SpiderMonkey’s GC to avoid problems where the GC runs in background apps or inadvertently touches compressed zram pages.

Till Schneidereit lead a brainstorming session about improving SpiderMonkey’s embedding API. Ideas included promoting SpiderMonkey as a scripting language solution for game engines (like 0 A.D.) or revisiting SpiderNode, a 2012 experiment to link Node.js with SpiderMonkey instead of V8. SpiderNode might be interesting for our testing or to Node developers that would like to use SpiderMonkey’s more extensive support for ES6 features or remote debugging tools. ES6 on the server doesn’t have the browser compatibility limitations that front-end web development does. The meeting notes and further discussion continued on the SpiderMonkey mailing list. New Mozilla contributor Sarat Adiraj soon posted his patches to revive SpiderNode in bug 1005411.

For the work week’s finale, Mozilla’s GC developers Terrence Cole, Steve Fink, and Jon Coppeard landed their generational garbage collector (GGC), a major redesign of SpiderMonkey’s GC. GGC will improve JS performance and lay the foundation for implementing a compacting GC to reduce JS memory usage later this year. GGC is riding the trains and should ship in Firefox 31 (July 2014).

Web analytics software often tracks people using a “fingerprint” of their browsers’ unique characteristics. Bumper stickers are a good analogy. If you see a blue Volkswagen with the same uncommon bumper stickers as a blue Volkswagen you saw yesterday, there is a very good chance it is the same person driving the same car. If you don’t want people to recognize your car, you can remove your bumper stickers or display the same bumper stickers seen on many other cars. The list of plugins and fonts installed on your computer are like bumper stickers on your browser.

I landed a fix for bug 757726 so Firefox 28 will “cloak” uncommon plugin names from navigator.plugins[] enumeration. This change does not disable any plugins; it just hides some plugin names.

If you find that a website no longer recognize your installed plugin when running Firefox 28, this is likely a side effect of bug 757726. Please file a new bug blocking bug 757726 so we can fix our whitelist of uncloaked plugin names or have a web compatibility evangelist reach out to the website author to fix their code.

This code change will reduce browser uniqueness by “cloaking” uncommon plugin names from navigator.plugins[] enumeration. If a website does not use the “Adobe Acrobat NPAPI Plug-in, Version 11.0.02″ plugin, why does it need to know that the “Adobe Acrobat NPAPI Plug-in, Version 11.0.02″ plugin is installed? If a website does need to know whether the plugin is installed or meets minimum version requirements, it can still check navigator.plugins[“Adobe Acrobat NPAPI Plug-in, Version 11.0.02″] or navigator.mimeTypes[“application/vnd.fdf”].enabledPlugin (to workaround problem plugins that short-sightedly include version numbers in their names).

navigator.plugins[“Unity Player”].name // get cloaked plugin by name
“Unity Player”

In theory, all plugin names could be cloaked because web content can query navigator.plugins[] by plugin name. Unfortunately, we could not cloak all plugin names because many popular websites check for Flash or QuickTime by enumerating navigator.plugins[] and comparing plugin names one by one, instead of just asking for navigator.plugins[“Shockwave Flash”] by name. These websites should be fixed.

The policy of which plugin names are uncloaked can be changed in the about:config pref “plugins.enumerable_names”. The pref’s value is a comma-separated list of plugin name prefixes (so the prefix “QuickTime” will match both “QuickTime Plug-in 6.4″ and “QuickTime Plug-in 7.7.3″). The default pref cloaks all plugin names except Flash, Shockwave (Director), Java, and QuickTime. To cloak all plugin names, set the pref to the empty string “”. To cloak no plugin names, set the pref to magic value “*”.

I started hacking on this patch in my spare time 13 months ago. I finally found some time to complete it. :)

Most Mozilla developers use Mercurial (hg) patch queues to organize patches and revise them based on feedback from Bugzilla code reviews. I had not found a similar git workflow that was nearly as convenient until Stacked Git. Suggestions to use git rebase --interactive and temporary branches were not as nimble as being able to push and pop patches on a branch.

Stacked Git (stg) is a git porcelain script that adds support for hg-style patch queues. stg’s patch queues augment, not replace, git’s feature branches. When you start a patch queue for a branch, stg creates a corresponding “my_branch.stgit” branch. Your patch queue operations are recorded in the “my_branch.stgit” branch, but your “my_branch” has commits for each of your applied patches. OSX users can use Homebrew to install Stacked Git: `brew install stgit`

Firefox for Android has expanded its HTML5 video capabilities to include H.264 video playback. Web developers have been using Adobe Flash to play H.264 video on Firefox for Android, but Adobe no longer supports Flash for Android. Mozilla needed a new solution, so Firefox now uses Android’s “Stagefright” library to access hardware video decoders. The challenges posed by H.264 patents and royalties have been documented elsewhere.

Supported devices

Firefox currently supports H.264 playback on any device running Android 4.1 (Jelly Bean) and any Samsung device running Android 4.0 (Ice Cream Sandwich). We have temporarily blocked non-Samsung devices running Ice Cream Sandwich until we can fix or workaround some bugs. Support for Gingerbread and Honeycomb devices is planned for a later release (Bug 787228).

Testing H.264

If your device is not supported yet, you can manually enable H.264 for testing. Enter about:config in Firefox for Android’s address bar, then search for “stagefright”. Toggle the “stagefright.force-enabled” preference to true. H.264 should work on most Ice Cream Sandwich devices, but Gingerbread and Honeycomb devices will probably crash.

If Firefox does not recognize your hardware decoder, it will use a safer (but slower) software decoder. Daring users can manually enable hardware decoding. Enter about:config as described above and search for “stagefright”. To force hardware video decoding, change the “media.stagefright.omxcodec.flags” preference to 16. The default value is 0, which will try the hardware decoder and fall back to the software decoder if there are problems (Bug 797225). The most likely problems you will encounter are videos with green lines or crashes.

Giving feedback/reporting bugs

If you find any video bugs, please file a bug report here so we can fix it! Please include your device model, Android OS version, the URL of the video, and any about:config preferences you have changed. Log files collected from aLogcat or adb logcat are also very helpful.

This post is a whirlwind tour of Boot To Gecko‘s virtual keyboard implementation. I take no credit for the design. These are just my notes from a recent expedition into Boot To Gecko (B2G).

B2G has a microkernel-like architecture. B2G apps are written in HTML+CSS+JS, each app running in an isolated “content” process. They communicate with a centralized Gecko process that is trusted to interface with hardware, such as drawing on screen or dispatching events to app processes.

B2G’s virtual keyboard consists of two halves: an unprivileged keyboard app and a privileged keyboard component in the Gecko process.

The keyboard app is a regular app and runs in its own content process. The keyboard app draws keys on screen and converts touch events into key events. In the future, users might be able to install third-party keyboard apps. The important files of the keyboard app are MozKeyboard.js, keyboard.js, and latin.js.

The Gecko process’ keyboard component routes key events from the keyboard app to the other apps. The important files of Gecko’s keyboard component are Keyboard.jsm and forms.js.

Take a deep breath before continuing on to the example. :)

Say a user is running the Contacts app and would like to add a new contact. On the “Add Contact” screen, the user taps the “Name” text field. Gecko’s forms.js receives a focus event for the text field and sends a “Forms:Input” message. Gecko’s Keyboard.jsm component receives the “Forms:Input” message and sends a “Keyboard:FocusChange” message to the keyboard app’s MozKeyboard.js. Control now passes from the Gecko process to the keyboard app process.

In the keyboard app process, MozKeyboard.js receives the “Keyboard:FocusChange” message and calls keyboard.js’ onfocuschange() callback, which calls showKeyboard() to draw a keyboard on screen and load an input method for the user’s language. The default input method is latin.js.

For example, latin.js spawns a SuggestionsWorker thread and sends a “predict” message with the key codes that the user has typed since the last word break. The SuggestionsWorker searches a dictionary for possible matches and asynchronously returns a “predictions” message containing (up to) three predicted words for keyboard.js to display on screen.

Keyboard.jsm receives the key event from the keyboard app and sends a “Forms:Input:Value” message to forms.js. forms.js dispatches a DOM input event to the Contact app’s “Name” text field.

To help prioritize my work on Firefox for Android’s keyboard and IME bugs, I created this list of popular Android keyboard apps. The Google Play Store does not divulge much information about other company’s apps, but it does reveal rough upper and lower bounds on the number of “Installs” over the last 30 days. For comparison, I have included numbers for popular Android browsers: Chrome, Dolphin, Firefox, and Opera.

I also compiled the number of 4 and 5 star ratings for these apps. I assume that people who like an app enough to write a review and give it 4 or 5 stars are likely to remain active users.

btw, this list does not include the Swype keyboard because it’s not available in the Play Store. Swype is only available bundled on a phone or from the company’s beta program.

Open questions:

Why is the GO Keyboard so popular? It supports many languages and themes, but I would imagine that a keyboard designed for the nuances of a particular language would be more popular.

Why does the Dolphin browser have so few beta users compared to Firefox and Opera? About 10% of Firefox and Opera users are using beta versions, but only 1% of Dolphin users.

Is there a fair way to aggregate numbers for apps that have multiple versions? Many apps have free and paid or stable and beta versions. These populations likely overlap. For example, many users are likely to have installed the trial version of an app before paying for it. Beta users may keep the stable version installed in case they are blocked by a beta bug.