This extension was originally introduced for Firebug back in 2008 but has not been maintained for several years. Many of its existing users asked us to bring the feature back, and the Firebug Working Group got the opportunity to build it again from scratch using the Remote Debugging Protocol that’s built into Firefox. This enabled us to support new features like multiprocess Firefox and remote debugging. The extension also integrates with Firebug 3 (aka Firebug.next), but Firebug isn’t required.

We had two goals in mind when building the extension:

Make the Pixel Perfect feature available again

Show how to build a real world extension on top of the Remote Debugging Protocol and other Firefox APIs

This post is about the feature, but if you are an extension developer interested in learning how to build developer or designer tools using the latest Firefox APIs, you might want to read about its internal architecture.

Getting started with Pixel Perfect 2

The latest Pixel Perfect 2 (PP2) release can be installed from addons.mozilla.org (you need to have at least Firefox 36 installed). After installation, you should see a new item in the main Firefox toolbar, with a button on the left and an arrow on the right.

Clicking the button opens PP2, and clicking the arrow shows a menu with links to online resources.

If you have Firebug 3 (currently alpha) installed, you can also open PP2 from the Style Editor panel.

The PP2 UI consists of one floating window that is used to add and remove layers. This is how it looks after installation.

Add the first layer using the Add Layer button. Click the button and pick an image file from your hard drive.

The new layer should be visible inside the floating popup window as well as in the page.

You can change its properties, such as location and opacity. You can also drag the layer in the page.

The screenshot shows a page with the text Form Editor: Contact Us and one layer, shown with a blue dashed border, that shows how the page should look. Now you can use the Developer Toolbox to tweak the page to pixel perfection!

Jan ‘Honza’ Odvarko

]]>https://hacks.mozilla.org/2015/03/pixel-perfect-2-extension-for-firefox-developer-tools/feed/0What do you want from your DevTools?https://hacks.mozilla.org/2015/03/what-do-you-want-from-your-devtools/
https://hacks.mozilla.org/2015/03/what-do-you-want-from-your-devtools/#commentsThu, 26 Mar 2015 14:47:43 +0000https://hacks.mozilla.org/?p=28536Every tool starts with an idea: “if I had this, then I could do that faster, better, or more easily.” The Firefox Developer Tools are built atop hundreds of those ideas, and many of them come from people in the web development community just like you.

Good ideas don’t have to be big or even particularly innovative as long as they help you accomplish your goals more effectively. For instance, all of the following features also came directly from feedback on our UserVoice forum:

Though somewhat small in scope, each of those ideas made the DevTools more efficient, more helpful, and more usable. And that’s our mission: we make Firefox for you, and we want you to know that we’re listening. If you have ideas for the Firefox Developer Tools, submit them on UserVoice. If you don’t have ideas, vote on the ones that are already there. The right people will see it, and it will help us build the best tools for you.

If you want to go a step further and contribute to the tools yourself, we’re always happy to help people get involved. After all, Mozilla was founded on the open source ideal of empowering communities to collaborate and improve the Internet for all of us.

]]>https://hacks.mozilla.org/2015/03/what-do-you-want-from-your-devtools/feed/16WebRTC in Firefox 38: Multistream and renegotiationhttps://hacks.mozilla.org/2015/03/webrtc-in-firefox-38-multistream-and-renegotiation/
https://hacks.mozilla.org/2015/03/webrtc-in-firefox-38-multistream-and-renegotiation/#commentsWed, 25 Mar 2015 14:28:24 +0000https://hacks.mozilla.org/?p=28501Building on the JSEP (Javascript Session Establishment Protocol) engine rewrite introduced in 37, Firefox 38 now has support for multistream (multiple tracks of the same type in a single PeerConnection), and renegotiation (multiple offer/answer exchanges in a single PeerConnection). As usual with such things, there are caveats and limitations, but the functionality seems to be pretty solid.

Multistream and renegotiation features

Why are these things useful, you ask? For example, now you can handle a group video call with a single PeerConnection (multistream), and do things like add/remove these streams on the fly (renegotiation). You can also add screensharing to an existing video call without needing a separate PeerConnection. Here are some advantages of this new functionality:

Simplifies your job as an app-writer

Requires fewer rounds of ICE (Interactive Connectivity Establishment – the protocol for establishing connection between the browsers), and reduces call establishment time

Requires fewer ports, both on the browser and on TURN relays (if using bundle, which is enabled by default)

Now, there are very few WebRTC services that use multistream (the way it is currently specified, see below) or renegotiation. This means that real-world testing of these features is extremely limited, and there will probably be bugs. If you are working with these features, and are having difficulty, do not hesitate to ask questions in IRC at irc.mozilla.org on #media, since this helps us find these bugs.

Also, it is important to note that Google Chrome’s current implementation of multistream is not going to be interoperable; this is because Chrome has not yet implemented the specification for multistream (called “unified plan” – check on their progress in the Google Chromium Bug tracker). Instead they are still using an older Google proposal (called “plan B”). These two approaches are mutually incompatible.

On a related note, if you maintain or use a WebRTC gateway that supports multistream, odds are good that it uses “plan B” as well, and will need to be updated. This is a good time to start implementing unified plan support. (Check the Appendix below for examples.)

Building a simple WebRTC video call page

So let’s start with a concrete example. We are going to build a simple WebRTC video call page that allows the user to add screen sharing during the call. As we are going to dive deep quickly you might want to check out our earlier Hacks article, WebRTC and the Early API, to learn the basics.

First we need two PeerConnections:

pc1 =new mozRTCPeerConnection();
pc2 =new mozRTCPeerConnection();

Then we request access to camera and microphone and attach the resulting stream to the first PeerConnection:

To keep things simple we want to be able to run the call just on one machine. But most computers today don’t have two cameras and/or microphones available. And just having a one-way call is not very exciting. So let’s use a built-in testing feature of Firefox for the other direction:

Note: You’ll want to call this part from within the success callback of the first getUserMedia() call so that you don’t have to track with boolean flags if both getUserMedia() calls succeeded before you proceed to the next step.
Firefox also has a built-in fake audio source (which you can turn on like this {audio: true, fake: true}). But listening to an 8kHz tone is not as pleasant as looking at the changing color of the fake video source.

For this example we take a shortcut: Instead of passing the signaling message through an actual signaling relay, we simply pass the information into both PeerConnections as they are both locally available on the same page. Refer to our previous hacks article WebRTC and the Early API for a solution which actually uses FireBase as relay instead to connect two browsers.

The one remaining piece is to connect the remote videos once we receive them.

pc1.onaddstream=function(obj){
pc1video.mozSrcObject= obj.stream;}

Add a similar clone of this for our PeerConnection 2. Keep in mind that these callback functions are super trivial — they assume we only ever receive a single stream and only have a single video player to connect it. The example will get a little more complicated once we add the screen sharing.

With this we should be able to establish a simple call with audio and video from the real devices getting sent from PeerConnection 1 to PeerConnection 2 and in the opposite direction a fake video stream that shows slowly changing colors.

Implementing screen sharing

Now let’s get to the real meat and add screen sharing to the already established call.

You need to append your domain to the user preference media.getusermedia.screensharing.allowed_domains in about:config to whitelist it for screen sharing.

For the screenConstraints you can also use ‘window‘ or ‘application‘ instead of ‘screen‘ if you want to share less than the whole screen.
We are using getTracks() here to fetch and store the video track out of the stream we get from the getUserMedia call, because we need to remember the track later when we want to be able to remove screen sharing from the call. Alternatively, in this case you could use the addStream() function used before to add new streams to a PeerConnection. But the addTrack() function gives you more flexibility if you want to handle video and audio tracks differently, for instance. In that case, you can fetch these tracks separately via the getAudioTracks() and getVideoTracks() functions instead of using the getTracks() function.

Once you add a stream or track to an established PeerConnection this needs to be signaled to the other side of the connection. To kick that off, the onnegotiationneeded callback will be invoked. So your callback should be setup before adding a track or stream. The beauty here — from this point on we can simply re-use our signaling call chain. So the resulting screen share function looks like this:

The important thing to note here: With multistream and renegotiation onaddstream can and will be called multiple times. In our little example onaddstream is called the first time we establish the connection and PeerConnection 2 starts receiving the audio and video from the real devices. And then it is called a second time when the video stream from the screen sharing is added.
We are simply assuming here that the screen share will have no audio track in it to distinguish the two cases. There are probably cleaner ways to do this.

Please refer to the Appendix for a bit more detail on what happens here under the hood.

As the user probably does not want to share his/her screen until the end of the call let’s add a function to remove it as well.

We are holding on to a reference to the original stream to be able to call stop() on it to release the getUserMedia permission we got from the user. The addTrack() call in our screenShare() function returned us an RTCRtpSender object, which we are storing so we can hand it to the removeTrack() function.

If you are going to build something which allows both ends of the call to add screen share, a more realistic scenario than our demo, you will need to handle special cases. For example, multiple users might accidentally try to add another stream (e.g. the screen share) exactly at the same time and you may end up with a new corner-case for renegotiation called “glare.” This is what happens when both ends of the WebRTC session decide to send new offers at the same time. We do not yet support the “rollback” session description type that can be used to recover from glare (see Jsep draft and the Firefox bug). Probably the best interim solution to prevent glare is to announce via your signaling channel that the user did something which is going to kick off another round of renegotiation. Then, wait for the okay from the far end before you call createOffer() locally.

Note that each track gets its own m-section, denoted by the msid attribute.

As you can see from the BUNDLE attribute, Firefox offers to put the new video stream, with its different msid value, into the same bundled transport. That means if the answerer agrees we can start sending the video stream over the already established transport. We don’t have to go through another ICE and DTLS round. And in case of TURN servers we save another relay resource.

Note that there is only one video m-section, with two different msids, which are part of the ssrc attributes rather than in their own a lines (these are called “source-level” attributes).

]]>https://hacks.mozilla.org/2015/03/webrtc-in-firefox-38-multistream-and-renegotiation/feed/1Understanding the CSS box model for inline elementshttps://hacks.mozilla.org/2015/03/understanding-inline-box-model/
https://hacks.mozilla.org/2015/03/understanding-inline-box-model/#commentsTue, 24 Mar 2015 15:26:44 +0000https://hacks.mozilla.org/?p=28401In a web page, every element is rendered as a rectangular box. The box model describes how the element’s content, padding, border, and margin determine the space occupied by the element and its relation to other elements in the page.

Depending on the element’s display property, its box may be one of two types: a block box or an inline box. The box model is applied differently to these two types. In this article we’ll see how the box model is applied to inline boxes, and how a new feature in the Firefox Developer Tools can help you visualize the box model for inline elements.

Inline boxes and line boxes

Inline boxes are laid out horizontally in a box called a line box:

If there isn’t enough horizontal space to fit all elements into a single line, another line box is created under the first one. A single inline element may then be split across lines:

The box model for inline boxes

When an inline box is split across more than one line, it’s still logically a single box. This means that any horizontal padding, border, or margin is only applied to the start of the first line occupied by the box, and the end of the last line.

For example, in the screenshot below, the highlighted span is split across 2 lines. Its horizontal padding doesn’t apply to the end of the first line or the beginning of the second line:

Also, any vertical padding, border, or margin applied to an element will not push away elements above or below it:

However, note that vertical padding and border are still applied, and the padding still pushes out the border:

If you need to adjust the height between lines, use line-height instead.

Inspecting inline elements with devtools

When debugging layout issues, it’s important to have tools that give you the complete picture. One of these tools is the highlighter, which all browsers include in their devtools. You can use it to select elements for closer inspection, but it also gives you information about layout.

In the example above, the highlighter in Firefox 39 is used to highlight an inline element split across several lines.

The highlighter displays guides to help you align elements, gives the complete dimensions of the node, and displays an overlay showing the box model.

From Firefox 39 onwards, the box model overlay for split inline elements shows each individual line occupied by the element. In this example the content region is displayed in light blue and is split over four lines. A right padding is also defined for the node, and the highlighter shows the padding region in purple.

Highlighting each individual line box is important for understanding how the box model is applied to inline elements, and also helps you check the space between lines and the vertical alignment of inline boxes.

]]>https://hacks.mozilla.org/2015/03/understanding-inline-box-model/feed/1How to make a browser app for Firefox OShttps://hacks.mozilla.org/2015/03/how-to-make-a-browser-app-for-firefox-os/
https://hacks.mozilla.org/2015/03/how-to-make-a-browser-app-for-firefox-os/#commentsFri, 20 Mar 2015 17:47:44 +0000https://hacks.mozilla.org/?p=28417Firefox OS is an operating system built on top of the Firefox web browser engine, which is called Gecko. A browser app on Firefox OS provides a user interface written with HTML5 technology and manages web page browsing using the Browser API. It also manages tabbing, browsing history, bookmarks, and so on depending on the implementation.

While Firefox OS already includes a browser, you can use the browser API to create your own browser or add browser functionality to your app. This article shows you how to build a browser app for Firefox OS devices. Following the steps, you’ll get a basic Firefox OS browser app with an address bar and back/forward buttons to browse web pages.

Prerequisites: WebIDE

The WebIDE is available with Firefox 34 or later. A Firefox OS device is not required to develop, build or test a browser app. By using the WebIDE, we can easily bootstrap a web app, make HTML/CSS/JS modifications and run the app on one of the Firefox OS simulators. To open the WebIDE within Firefox, select Tools > Web Developer > WebIDE from the top menu:

Create an app from the template

First, bootstrap a web app using the WebIDE empty template. The app type needs to be privileged so the browser permission can be set. This permission is required in order to use the Browser API. The Browser API provides additional methods and events to manage iframes.

Below is a screenshot of what the app should look like. Next we’ll start to add code to make a simple browser app.

Edit the manifest.webapp

Let’s start to make some code changes. The template has already set the app type to privileged in the manifest.webapp.

"type":"privileged",

To allow our app to use the Browser API, we need to specify the ‘browser‘ permission.

"permissions":{"browser":{}},

HTML structure

The browser app we’re building will have a toolbar on top and a div as the iframe container which displays ‘Hello myBrowser!’ by default. The browser iframe will be created later using JavaScript. We could have added the iframe to the HTML, but in this example a new browser iframe will be created dynamically.

Manage the browser iframe

The browser app has to handle mozbrowser events which are accessible when the app has the browser permission. In order to separate the UI and mozbrowser event handling and preserve the flexibility of supporting multiple tabs in the future, we have added a tab.js file. Following is the code in tab.js which creates an iframe with the ‘mozbrowser‘ attribute to enable the use of the Browser API.

The mozallowfullscreen attribute enables the embedded web page of the iframe to use fullscreen mode. A web page can request fullscreen mode by calling Element.mozRequestFullscreen().

The ‘remote‘ attribute separates the embedded iframe into another child process. This is needed for security reasons to prevent malicious web sites from compromising the browser app. Currently, the attribute does nothing in this sample browser app because Nested OOP has not been implemented. See Bug 1020135 - (nested-oop) [meta] Allow nested oop <iframe mozbrowser>.

Next the ‘Tab‘ object constructor is added to tab.js. This constructor handles browser iframe creation and browser event handling. When a Tab is constructed, it creates a browser iframe and attaches the mozbrowser event listeners to it:

There are additional mozbrowser events that are not used in this simple browser app. The events used in the sample provide access to useful information about the iframe, and our browser app uses them to provide web page details, like the title, URL, loading progress, contextmenu, etc.

For example, to display the page title on the toolbar, we use the mozbrowsertitlechange event to retrieve the title of the web page. Once the title is updated, a CustomEvent'tab:titlechange' with the Tab itself as the detail is dispatched to notify other components to update the title.

Browse a web page on submit event

Using the address bar as a search bar is a common and useful practice among browsers. When a user submits the input, first we check if it is a valid URL. If not, the input is regarded as a search term and it will be appended to a search engine URI. Note that the validation of the URL can be handled in multiple ways. The sample app uses an input element with its type attribute set to URL. A URL object could also have been used with the URL string passed as the first parameter to the constructor. A DOM exception will be thrown if the URL is not valid. This exception can then be caught and the URL string assumed to be a search term.

Code can then be added to the main app.js to handle the submit button. This code checks to see if a currently active Tab object exists, and either loads the URL or reloads the URL if it has not changed. If one does not exist, it is created and the URL is loaded.

Enable/disable back and forward buttons

Finally the Browser API can be used to check if the page can go backward or forward in the navigation history. The iframe.getCanGoBack method returns a DOMRequest object which indicates if the page can go backward in its onsuccess callback. We use a Promise to wrap this function for easier access. This code is added to the tab.js file.

Run the app

One of the Firefox OS simulators can now be used to test the sample browser app.

On the top right of the WebIDE, click Select Runtime to choose a Firefox OS phone or simulator:

Once a Firefox OS runtime is selected and launched, you can install and run the app by clicking the play button within the WebIDE. See the WebIDE documentation for more information on building, debugging, and deploying Firefox OS apps.

Final thoughts

There’s more functionality that can be added to this simple little browser to make it really useful including bookmarks, settings, search suggestions, and sharing. Feel free to expand on this code and have fun with it. Let us know what you think and be sure to spread the word about Firefox OS!

Additional information

]]>https://hacks.mozilla.org/2015/03/how-to-make-a-browser-app-for-firefox-os/feed/2Using the Firefox DevTools to Debug fetch() on GitHubhttps://hacks.mozilla.org/2015/03/using-the-firefox-devtools-to-debug-fetch-on-github/
https://hacks.mozilla.org/2015/03/using-the-firefox-devtools-to-debug-fetch-on-github/#commentsThu, 19 Mar 2015 18:45:19 +0000https://hacks.mozilla.org/?p=28413Firefox Nightly recently added preliminary support for Fetch, a modern, Promise-based replacement for XMLHttpRequest (XHR). Our initial work supported most of the Fetch Specification, but not quite all of it. Specifically, when Fetch first appeared in Nightly, we hadn’t yet implemented serializing and de-serializing of FormData objects.

GitHub was already using Fetch in production with a home-grown polyfill, and required support for serializing FormData in order to upload images to GitHub Issues. Thus, when our early, incomplete implementation of Fetch landed in Nightly, the GitHub polyfill stepped out of the way, and image uploads from Firefox broke.

In the 15-minute video below, Dan Callahan shows a real-world instance of using the Firefox Developer Tools to help find, file, and fix Bug 1143857: “Fetch does not serialize FormData body; breaks GitHub.” This isn’t a canned presentation, but rather a comprehensive, practical demonstration of actually debugging minified JavaScript and broken event handlers using the Firefox DevTools, reporting a Gecko bug in Bugzilla, and ultimately testing a patched build of Firefox.

There are three kinds of lies: lies, damned lies, and statistics – Mark Twain

Deciding what to track (all the things)

When you are adding analytics to a system you should try to log everything. At some point in the future if you need to pull information out of a system it’s much better to have every piece of information to hand, rather than realising that you need some data that you don’t yet track. Here are some guidelines and suggestion for collecting and analysing information about how people are interacting with your website or app.

Grouping your stats as a best practice

Most analytics platforms allow you to tag an event with metadata. This lets you analyse stats against each other and makes it easier to compare elements in a user interaction.

For example, if you are logging clicks on a menu, you could track each menu item differently e.g.:

The analytics above mean you now have a total of all menu presses, and you can find the most/least popular of the menu items with no extra effort.

Optimising your funnel

A conversion funnel is a term of art derived from a consumer marketing model. The metaphor of the funnel describes the flow of steps a user goes through as they engage more deeply with your software. Imagine you want to know how many users clicked log in and then paid at the checkout? If you track events such as “Checkout complete” and “User logged in” you can then ask your analytics platform what percentage of users did both within a certain time frame (a day for instance).

Imagine the answer comes out to be 10%, this tells you useful information about the behaviour of your users (bear in mind this funnel is not order sensitive, i.e., it does not matter in which order the events happen in login -> cart -> pay or cart -> login -> pay). Thus, you can start to optimise parts of your app and use this value to determine whether or not you are converting more of your users to make a purchase or otherwise engage more deeply.

Deciding what to measure

Depending on your business, different stats will have different levels of importance. Here are some common stats of interest to developers of apps or online services:

Number of sessions

The total number of sessions on your product (the user opening your product, using it, then closing it = 1 session)

Session length

How long each session lasts (can be mode, mean, median)

Retention

How many people come back to your product having used it before (there are a variety of metrics such as rolling retention, 30 day retention etc)

MAU

Monthly active users: how may users use the app once a month

DAU

Daily active users: how may users use the app once a day

ARPU

Average revenue per user: how much money you make per person

ATV

Average transaction value: how much money you make per sale

CAC

Customer acquisition cost: how much it costs to get one extra user (normally specified by the channel for getting them)

CLV

Customer lifetime value: total profit made from a user (usually projected)

Churn

The number of people who leave your product in a given time (usually given as a percentage of total user base)

Cycle time

The the time it takes for one user to refer another

Choosing an analytics tool or platform

There are plenty of analytics providers, listed below are some of the best known and most widely used:

Analytics for your Firefox App

You can use any of the above providers above with Firefox OS, but remember when you paste a script into your code they generally are protocol agnostic: they start //myjs.com/analytics.js and you need to choose either http: or https: — https://myjs.com/analytics.js (This is required only if you are making a packaged app.)

Let us know how it goes.

]]>https://hacks.mozilla.org/2015/03/an-analytics-primer-for-developers/feed/4Optimising SVG imageshttps://hacks.mozilla.org/2015/03/optimising-svg-images/
https://hacks.mozilla.org/2015/03/optimising-svg-images/#commentsWed, 11 Mar 2015 15:56:19 +0000https://hacks.mozilla.org/?p=28281SVG is a vector image format based on XML. It has great advantages, most notably it is lightweight. Since SVG is a text format, it can be viewed and modified using a simple text editor, and applying GZIP compression produces excellent results.

It’s critical for a website to provide assets that are as lightweight as possible, especially on mobile where bandwidth can be very limited. You want to optimise your SVG files to have your app load and display as quickly as possible.

This article will show how to use dedicated tools to optimise SVG images. You will also learn how the markup works so you can go the extra mile to produce the lightest possible images.

Introducing svgo

Optimising SVG is very similar to minifying CSS or other text-based formats such as JavaScript or HTML. It is mainly about removing useless whitespace and redundant characters.

The tool I recommend to reduce the size of SVG images is svgo. It is written for node.js. To install it, just do:

$ npm install-g svgo

In its basic form, you’ll use a command line like this:

$ svgo --input img/graph.svg --output img/optimised-graph.svg

Please make sure to specify an --output parameter if you want to keep the original image. Otherwise svgo will replace it with the optimised version.

svgo will apply several changes to the original file—stripping out useless comments, tags, and attributes, reducing the precision of numbers in path definitions, or sorting attributes for better GZIP compression.

This works with no surprises for simple images. However, in more complex cases, the image manipulation can result in a garbled file.

svgo plugins

svgo is very modular thanks to a plugin-based architecture.

When optimising complex images, I’ve noticed that the main issues are caused by two svgo plugins:

convertPathData

mergePaths

Deactivating these will ensure you get a correct result in most cases:

$ svgo --disable=convertPathData --disable=mergePaths -i img/a.svg

convertPathData will convert the path data using relative and shorthand notations. Unfortunately, some environments won’t fully recognise this syntax and you’ll get something like:

Screenshot of Gnome Image Viewer displaying an original SVG image (left) and a version optimised via svgo on (right)

Please note that the optimised image will display correctly in all browsers. So you may still want to use this plugin.

The other plugin that can cause you trouble—mergePaths—will merge together shapes of the same style to reduce the number of <path> tags in the source. However, this might create issues if two paths overlap.

In the image on the right, please note the rendering differences around the character’s neck and hand, also note the Twitter logo. The outline view shows 3 overlapping paths that make up the character’s head.

My suggestion is to first try svgo with all plugins activated, then if anything is wrong, deactivate the two mentioned above.

If the result is still very different from your original image, then you’ll have to deactivate the plugins one by one to detect the one which causes the issue. Here is a list of svgo plugins.

Optimising even further

svgo is a great tool, but in some specific cases, you’ll want to compress your SVG images even further. To do so, you have to dig into the file format and do some manual optimisations.

In these cases, my favourite tool is Inkscape: it is free, open source and available on most platforms.

If you want to use the mergePaths plugin of svgo, you must combine overlapping paths yourself. Here’s how to do it:

Open your image in Inkscape and identify the path with the same style (fill and stroke). Select them all (maintain shift pressed for multiple selection). Click on the Path menu and select Union. You’re done—all three paths have been merged into a single one.

The 3 different paths that create the character’s head are merged, as shown by the outline view on the right.

Repeat this operation for all paths of the same style that are overlapping and then you’re ready to use svgo again, keeping the mergePaths plugin.

There are all sorts of different optimisations you can apply manually:

Convert strokes to paths so they can be merged with paths of similar style.

Cut paths manually to avoid using clip-path.

Exclude an underneath path from a overlapping path and merge with a similar path to avoid layer issues. (In the image above, see the character’s hair—the side hair path is under his head, but the top hair is above it—so you can’t merge the 3 hair paths as is.)

Final considerations

These manual optimisations can take a lot of time for meagre results, so think twice before starting!

A good rule of thumb when optimising SVG images is to make sure the final file has only one path per style (same fill and stroke style) and uses no <g> tags to group path to objects.

In Firefox OS, we use an icon font, gaia-icons, generated from SVG glyphs. I noticed that optimising them resulted in a significantly lighter font file, with no visual differences.

Whether you use SVG for embedding images on an app or to create a font file, always remember to optimise. It will make your users happier!

]]>https://hacks.mozilla.org/2015/03/optimising-svg-images/feed/5This API is so Fetching!https://hacks.mozilla.org/2015/03/this-api-is-so-fetching/
https://hacks.mozilla.org/2015/03/this-api-is-so-fetching/#commentsTue, 10 Mar 2015 15:05:41 +0000https://hacks.mozilla.org/?p=28260For more than a decade the Web has used XMLHttpRequest (XHR) to achieve asynchronous requests in JavaScript. While very useful, XHR is not a very nice API. It suffers from lack of separation of concerns. The input, output and state are all managed by interacting with one object, and state is tracked using events. Also, the event-based model doesn’t play well with JavaScript’s recent focus on Promise- and generator-based asynchronous programming.

The Fetch API intends to fix most of these problems. It does this by introducing the same primitives to JS that are used in the HTTP protocol. In addition, it introduces a utility function fetch() that succinctly captures the intention of retrieving a resource from the network.

The Fetch specification, which defines the API, nails down the semantics of a user agent fetching a resource. This, combined with ServiceWorkers, is an attempt to:

As of this writing, the Fetch API is available in Firefox 39 (currently Nightly) and Chrome 42 (currently dev). Github has a Fetch polyfill.

Feature detection

Fetch API support can be detected by checking for Headers,Request, Response or fetch on the window or worker scope.

Simple fetching

The most useful, high-level part of the Fetch API is the fetch() function. In its simplest form it takes a URL and returns a promise that resolves to the response. The response is captured as a Response object.

The fetch() function’s arguments are the same as those passed to theRequest() constructor, so you may directly pass arbitrarily complex requests to fetch() as discussed below.

Headers

Fetch introduces 3 interfaces. These are Headers, Request andResponse. They map directly to the underlying HTTP concepts, but have
certain visibility filters in place for privacy and security reasons, such as
supporting CORS rules and ensuring cookies aren’t readable by third parties.

Some of these operations are only useful in ServiceWorkers, but they provide
a much nicer API to Headers.

Since Headers can be sent in requests, or received in responses, and have various limitations about what information can and should be mutable, Headers objects have a guard property. This is not exposed to the Web, but it affects which mutation operations are allowed on the Headers object.
Possible values are:

“none”: default.

“request”: guard for a Headers object obtained from a Request (Request.headers).

“request-no-cors”: guard for a Headers object obtained from a Request created
with mode “no-cors”.

“immutable”: Mostly used for ServiceWorkers, renders a Headers object
read-only.

The details of how each guard affects the behaviors of the Headers object are
in the specification. For example, you may not append or set a “request” guarded Headers’ “Content-Length” header. Similarly, inserting “Set-Cookie” into a Response header is not allowed so that ServiceWorkers may not set cookies via synthesized Responses.

All of the Headers methods throw TypeError if name is not a valid HTTP Header name. The mutation operations will throw TypeError if there is an immutable guard. Otherwise they fail silently. For example:

The Request’s mode is used to determine if cross-origin requests lead to valid responses, and which properties on the response are readable. Legal mode values are "same-origin", "no-cors" (default) and "cors".

The "same-origin" mode is simple, if a request is made to another origin with this mode set, the result is simply an error. You could use this to ensure that
a request is always being made to your origin.

The "no-cors" mode captures what the web platform does by default for scripts you import from CDNs, images hosted on other domains, and so on. First, it prevents the method from being anything other than “HEAD”, “GET” or “POST”. Second, if any ServiceWorkers intercept these requests, they may not add or override any headers except for these. Third, JavaScript may not access any properties of the resulting Response. This ensures that ServiceWorkers do not affect the semantics of the Web and prevents security and privacy issues that could arise from leaking data across domains.

"cors" mode is what you’ll usually use to make known cross-origin requests to access various APIs offered by other vendors. These are expected to adhere to
the CORS protocol. Only a limited set of headers is exposed in the Response, but the body is readable. For example, you could get a list of Flickr’s most interesting photos today like this:

You may not read out the “Date” header since Flickr does not allow it viaAccess-Control-Expose-Headers.

response.headers.get("Date");// null

The credentials enumeration determines if cookies for the other domain are
sent to cross-origin requests. This is similar to XHR’s withCredentials
flag, but tri-valued as "omit" (default), "same-origin" and "include".

The Request object will also give the ability to offer caching hints to the user-agent. This is currently undergoing some security review. Firefox exposes the attribute, but it has no effect.

Requests have two read-only attributes that are relevant to ServiceWorkers
intercepting them. There is the string referrer, which is set by the UA to be
the referrer of the Request. This may be an empty string. The other iscontext which is a rather large enumeration defining what sort of resource is being fetched. This could be “image” if the request is from an tag in the controlled document, “worker” if it is an attempt to load a worker script, and so on. When used with the fetch() function, it is “fetch”.

Response

Response instances are returned by calls to fetch(). They can also be created by JS, but this is only useful in ServiceWorkers.

We have already seen some attributes of Response when we looked at fetch(). The most obvious candidates are status, an integer (default value 200) and statusText (default value “OK”), which correspond to the HTTP status code and reason. The ok attribute is just a shorthand for checking that status is in the range 200-299 inclusive.

headers is the Response’s Headers object, with guard “response”. The url attribute reflects the URL of the corresponding request.

Response also has a type, which is “basic”, “cors”, “default”, “error” or
“opaque”.

"basic": normal, same origin response, with all headers exposed except
“Set-Cookie” and “Set-Cookie2″.

"error": network error. No useful information describing the error is available. The Response’s status is 0, headers are empty and immutable. This is the type for a Response obtained from Response.error().

As you can see, Response has a two argument constructor, where both arguments are optional. The first argument is a body initializer, and the second is a dictionary to set the status, statusText and headers.

Both Request and Response (and by extension the fetch() function), will try to intelligently determine the content type. Request will also automatically set a “Content-Type” header if none is set in the dictionary.

Streams and cloning

It is important to realise that Request and Response bodies can only be read once! Both interfaces have a boolean attribute bodyUsed to determine if it is safe to read or not.

This decision allows easing the transition to an eventual stream-based Fetch API. The intention is to let applications consume data as it arrives, allowing for JavaScript to deal with larger files like videos, and perform things like compression and editing on the fly.

Often, you’ll want access to the body multiple times. For example, you can use the upcoming Cache API to store Requests and Responses for offline use, and Cache requires bodies to be available for reading.

So how do you read out the body multiple times within such constraints? The API provides a clone() method on the two interfaces. This will return a clone of the object, with a ‘new’ body. clone() MUST be called before the body of the corresponding object has been used. That is, clone() first, read later.

Future improvements

Along with the transition to streams, Fetch will eventually have the ability to abort running fetch()es and some way to report the progress of a fetch. These are provided by XHR, but are a little tricky to fit in the Promise-based nature of the Fetch API.

The author would like to thank Andrea Marchesini, Anne van Kesteren and Ben
Kelly for helping with the specification and implementation.

]]>https://hacks.mozilla.org/2015/03/this-api-is-so-fetching/feed/9Ruby support in Firefox Developer Edition 38https://hacks.mozilla.org/2015/03/ruby-support-in-firefox-developer-edition-38/
https://hacks.mozilla.org/2015/03/ruby-support-in-firefox-developer-edition-38/#commentsThu, 05 Mar 2015 14:00:57 +0000https://hacks.mozilla.org/?p=27884It was a long-time request from East Asian users, especially Japanese users, to have ruby support in the browser.

Formerly, because of the lack of native ruby support in Firefox, users had to install add-ons like HTML Ruby to make ruby work. However, in Firefox Developer Edition 38, CSS Ruby has been enabled by default, which also brings the support of HTML5 ruby tags.

Introduction

What is ruby? In short, ruby is an extra text, which is usually small, attached to the main text for indicating the pronunciation or meaning of the corresponding characters. This kind of annotation is widely used in Japanese publications. It is also common in Chinese for books for children, educational publications, and dictionaries.

Basic Usage

Basically, the ruby support consists of four main tags: <ruby>, <rb>, <rt>, and <rp>. <ruby> is the tag that wraps the whole ruby structure, <rb> is used to mark the text in the normal line, <rt> is for the annotation, and <rp> is a tag which is hidden by default. With the four tags, the result above can be achieved from the following code:

Since <rb> and <rt> can be auto-closed by themselves, we don’t bother to add code to close those tags manually.

As shown in the image, the duplicate parts as well as <rp>s are hidden automatically. But why should we add content which is hidden by default?

The answer is: it is more natural in semantics, and it helps conversion to the inline form, which is a more general form accepted by more software. For example, this allows the page to have a decent effect on a browser with no ruby support. It also enables the user agent to generate well-formed plain text when you want to copy text with ruby (though this feature hasn’t yet been landed on Firefox).

In addition, the extra content makes it possible to provide inline style of the annotation without changing the document. You would only need to add a rule to your stylesheet:

ruby, rb, rt, rp {display:inline;font-size:inherit;}

Actually, if you don’t have those requirements, only <ruby> and <rt> are necessary. For simplest cases, e.g. a single ideographic character, you can use code like:

<ruby>咲<rt>Saki</rt></ruby>

Advanced Support

Aside from the basic usage of ruby, Firefox now provides support for more advanced cases.

By default, if the width of an annotation does not match its base text, the shorter text will be justified as shown in the example above. However, this behavior can be controlled via the ruby-align property. Aside from the default value (space-around), it can also make the content align to both sides (space-between), centered (center), or aligned to the start side (start).

Multiple levels of annotations are also supported via tag <rtc> which is the container of <rt>s. Every <rtc> represents one level of annotation, but if you leave out the <rtc>, the browser will do some cleanup to wrap consecutive <rt>s in a single anonymous <rtc>, forming one level.

You can use ruby-position to place the given level of annotation on the side you want. For the examples above, if you want to put the second level under the main line, you can apply ruby-position: under; to the <rtc> tag. Currently, only under and the default value over is supported.

(Note: The CSS Working Group is considering a change to the default value of ruby-position, so that annotations become double-sided by default. This change is likely to happen in a future version of Firefox.)

In the end, an advanced example of ruby combining everything introduced above:

]]>https://hacks.mozilla.org/2015/03/ruby-support-in-firefox-developer-edition-38/feed/3Announcing the MDN Fellowship Programhttps://hacks.mozilla.org/2015/03/mdnfellowshiplaunch/
https://hacks.mozilla.org/2015/03/mdnfellowshiplaunch/#commentsWed, 04 Mar 2015 16:03:09 +0000https://hacks.mozilla.org/?p=28014For nearly a decade, the Mozilla Developer Network (MDN) has been a vital source of technical information for millions of web and mobile developers. And while each month hundreds of developers actively contribute to MDN, we know there are many more with deep expertise in the Web who aren’t participating—yet. Certainly MDN and the Web would benefit from their knowledge and skill, and we’re piloting a program to provide benefits for them as well.

What it is

The MDN Fellowship pilot is a seven-week part-time (5-10 hours per week) education and leadership program pairing advanced web and mobile developers with engineering and educational experts at Mozilla to work on significant, influential web projects. Fellows will contribute by developing apps, API descriptions, and curriculum on MDN for their project. They will also receive hands-on coaching from project mentors, as well as others at Mozilla who will provide training and guidance on curriculum development, so that their work can be taught to others.

An overview of projects

Here’s a look at some of the technical topics and the associated project tasks we’ve identified for our first group of MDN fellows.

ServiceWorkers essentially act as proxy servers that sit between web applications, the browser, and (when available) the network. They are key to the success of web apps, enabling the creation of effective offline experiences and allow access to push notifications and background sync APIs.What you’ll work on: You’ll write a demonstration web app (new or existing) to showcase Service Worker functionality and provide detailed API descriptions.

WebGL is the latest incarnation of the OpenGL family of real-time rendering immediate mode graphics APIs. This year WebGL is getting some cool new features with the publication of standardization efforts around the WebGL 2.0 spec.What you’ll do: Develop a curriculum on MDN for teaching the WebGL APIs developers new to graphics programming.

Web app performance.App performance is impacted by many factors, including serving content, rendering, and interactivity. Finding and addressing performance bottlenecks depends on tooling the browser networking and rendering but also (and often more importantly) user perception.What you’ll do. Develop a curriculum on MDN for teaching developers to master performance tooling and to develop web apps with performance as a feature.

TestTheWebForward. Mozilla participates in an important W3C initiative, TestTheWebForward, a community-driven effort for open web platform testing.What you’ll do: Review various existing technical specifications to identify gaps between the documentation and current situation. Refine existing tests to adapt to this cross-browser test harness.

MDN curriculum development. MDN serves as a trusted resource for millions of developers. In 2015, MDN will expand the scope of this content by developing Content Kits: key learning materials including code samples, video screencasts and demos, and more.

What you’ll do: Act as lead curator for technical curriculum addressing a key web technology, developing code samples, videos, interactive exercises and other essential educational components. You may propose your own subject area (examples: virtual reality on the web, network security, CSS, etc.) or we will work with you to match you based on your subject area knowledge and Mozilla priorities.

“I’m looking forward to working with a Fellow to make low-level real-time graphics more approachable,” says WebGL project mentor Nick Desaulniers. More information, including project mentors and the type of skills & experience required for each project, can be found on the MDN Fellowship page.

How to apply

If any of these projects sounds interesting, check out the website and apply by April 1. We’ll announce the fellows in May and start the program in June by bringing fellows and mentors together to get everyone familiarized with their projects and one another. Then, for 6 weeks, you’ll work directly with your mentor on your project from your home base. You’ll also receive ongoing feedback and coaching to set you up for success on teaching what you’re building to larger groups of developers.

Here’s a timeline:

Now – April 1: Apply!

April: Finalist candidates will be interviewed.

May: Fellows are announced.

June (dates & location TBA): Orientation at a Mozilla space.

June 29 – August 3: Work on your projects + regular team calls to receive coaching on your work.

August 11-12: Graduation

This program is for you if…

You code, document, and ship with confidence, and now you’d like to start sharing your skills with a broader community. Maybe you want a way out of the walled garden and you’d like to contribute to Mozilla’s mission of keeping the web open for all. Perhaps you want to be effective at sharing your expertise with a wider audience.

Consider this program if you want an opportunity to:

Amplify the impact of your technical expertise by contributing to influential, significant projects at Mozilla.

]]>https://hacks.mozilla.org/2015/03/mdnfellowshiplaunch/feed/18asm.js Speedups Everywherehttps://hacks.mozilla.org/2015/03/asm-speedups-everywhere/
https://hacks.mozilla.org/2015/03/asm-speedups-everywhere/#commentsTue, 03 Mar 2015 16:41:16 +0000https://hacks.mozilla.org/?p=28055asm.js is an easy-to-optimize subset of JavaScript. It runs in all browsers without plugins, and is a good target for porting C/C++ codebases such as game engines – which have in fact been the biggest adopters of this approach, for example Unity 3D and Unreal Engine.

Obviously, developers porting games using asm.js would like them to run well across all browsers. However, each browser has different performance characteristics, because each has a different JavaScript engine, different graphics implementation, and so forth. In this post, we’ll focus on JavaScript execution speed and see the significant progress towards fast asm.js execution that has been happening across the board. Let’s go over each of the four major browsers now.

Chrome

Already in 2013, Google released Octane 2.0, a new version of their primary JavaScript benchmark suite, which contained a new asm.js benchmark, zlib. Benchmarks define what browsers optimize: things that matter are included in benchmarks, and browsers then compete to achieve the best scores. Therefore, adding an asm.js benchmark to Octane clearly signaled Google’s belief that asm.js content is important to optimize for.

A further major development happened more recently, when Google landed TurboFan, a new work-in-progress optimizing compiler for Chrome’s JavaScript engine, v8. TurboFan has a “sea of nodes” architecture (which is new in the JavaScript space, and has been used very successfully elsewhere, for example in the Java server virtual machine), and aims to reach even higher speeds than CrankShaft, the first optimizing compiler for v8.

While TurboFan is not yet ready to be enabled on all JavaScript content, as of Chrome 41 it is enabled on asm.js. Getting the benefits of TurboFan early on asm.js shows the importance of optimizing asm.js for the Chrome team. And the benefits can be quite substantial: For example, TurboFan speeds up Emscripten‘s zlib benchmark by 13%, and fasta by 24%.

Safari

During the last year, Safari’s JavaScript Engine, JavaScriptCore, introduced a new JIT (Just In Time compiler) called FTL. FTL stands for “Fourth Tier LLVM,” as it adds a fourth level of optimization above the three previously-existing ones, and it is based on LLVM, a powerful open source compiler framework. This is exciting because LLVM is a top-tier general-purpose compiler, with many years of optimizations put into it, and Safari gets to reuse all those efforts. As shown in the blogposts linked to earlier, the speedups that FTL provides can be very substantial.

Another interesting development from Apple this year was the introduction of a new JavaScript benchmark, JetStream. JetStream contains several asm.js benchmarks, an indication that Apple believes asm.js content is important to optimize for, just as when Google added an asm.js benchmark to Octane.

Internet Explorer

The JavaScript engine inside Internet Explorer is named Chakra. Last year, the Chakra team blogged about a suite of optimizations coming to IE in Windows 10 and pointed to significant improvements in the scores on asm.js workloads in Octane and JetStream. This is yet another example of how having asm.js workloads in common benchmarks drives measurement and optimization.

The big news, however, is the recent announcement by the Chakra team that they are working on adding specific asm.js optimizations, to arrive in Windows 10 together with the other optimizations mentioned earlier. These optimizations haven’t made it to the Preview channel yet, so we can’t measure and report on them here. However, we can speculate on the improvements based on the initial impact of landing asm.js optimizations in Firefox. As shown in this benchmark comparisons slide containing measurements from right after the landing, asm.js optimizations immediately brought Firefox to around 2x slower than native performance (from 5-12x native before). Why should these wins translate to Chakra? Because, as explained in our previous post, the asm.js spec provides a predictable way to validate asm.js code and generate high-quality code based on the results.

So, here’s looking forward to good asm.js performance in Windows 10!

Firefox

As we mentioned before, the initial landing of asm.js optimizations in Firefox generally put Firefox within 2x of native in terms of raw throughput. By the end of 2013, we were able to report that the gap had shrunk to around 1.5x native – which is close to the amount of variability that different native compilers have between each other anyhow, so comparisons to “native speed” start to be less meaningful.

At a high-level, this progress comes from two kinds of improvements: compiler backend optimizations and new JavaScript features. In the area of compiler backend optimizations, there has been a stream of tiny wins (specific to particular code patterns or hardware) making it difficult to point to any one thing. Two significant improvements stand out, though:

Along with backend optimization work, two new JavaScript features have been incorporated into asm.js which unlock new performance capabilities in the hardware. The first feature, Math.fround, may look simple but it enables the compiler backend to generate single-precision floating-point arithmetic when used carefully in JS. As described in this post, the switch can result in anywhere from a 5% – 60% speedup, depending on the workload. The second feature is much bigger: SIMD.js. This is still a stage 1 proposal for ES7 so the new SIMD operations and the associated asm.js extensions are only available in Firefox Nightly. Initial results are promising though.

Separate from all these throughput optimizations, there have also been a set of load time optimizations in Firefox: off-main-thread and parallel compilation of asm.js code as well as caching of the compiled machine code. As described in this post, these optimizations significantly improve the experience of starting a Unity- or Epic-sized asm.js application. Existing asm.js workloads in the benchmarks mentioned above do not test this aspect of asm.js performance so we put together a new benchmark suite named Massive that does. Looking at Firefox’s Massive score over time, we can see the load-time optimizations contributing to a more than 6x improvement (more details in the Hacks post introducing the Massive benchmark).

The Bottom Line

What is most important, in the end, are not the underlying implementation details, nor even specific performance numbers on this benchmark or that. What really matters is that applications run well. The best way to check that is to actually run real-world games! A nice example of an asm.js-using game is Dead Trigger 2, a Unity 3D game:

The video shows the game running on Firefox, but as it uses only standard web APIs, it should work in any browser. We tried it now, and it renders quite smoothly on Firefox, Chrome and Safari. We are looking forward to testing it on the next Preview version of Internet Explorer as well.

As with Unity, the developers of Cloud Raiders were able to compile their existing C++ codebase (using Emscripten) to run on the web without relying on plugins. The result runs well in all four of the major browsers.

In conclusion, asm.js performance has made great strides over the last year. There is still room for improvement – sometimes performance is not perfect, or a particular API is missing, in one browser or another – but all major browsers are working to make sure that asm.js runs quickly. We can see that by looking at the benchmarks they are optimizing on, which contain asm.js, and in the new improvements they are implementing in their JavaScript engines, which are often motivated by asm.js. As a result, games that not long ago would have required plugins are quickly getting to the point where they can run well without them, in modern browsers across the web.

]]>https://hacks.mozilla.org/2015/03/asm-speedups-everywhere/feed/10Firefox Developer Edition 38: 64-bits and morehttps://hacks.mozilla.org/2015/03/firefox-developer-edition-38-64-bits-and-more/
https://hacks.mozilla.org/2015/03/firefox-developer-edition-38-64-bits-and-more/#commentsMon, 02 Mar 2015 16:14:23 +0000https://hacks.mozilla.org/?p=28100In celebration of the 10th anniversary of Firefox, we unveiled Firefox Developer Edition, the first browser created specifically for developers. At that time, we also announced plans to ship a 64-bit version of Firefox. Today we’re happy to announce the next phase of that plan: 64-bit builds for Firefox Developer Edition are now available on Windows, adding to the already supported platforms of OS X and Linux.

A 64-bit build is a major step toward giving users rich, desktop-quality app experiences in the browser. Let’s also take a look at at some of the other features that make this a release worth noting. If you haven’t downloaded the Developer Edition browser yet, it’s a fine time to give it a try. Here’s why:

Unreal demo in Win 64-bit Developer Edition

Run larger applications

A 32-bit browser is limited to 4GB of address space. That address space is further whittled down by fragmentation issues. Meanwhile, web applications are getting bigger and bigger. Browser-based games that deliver performant, native-like gameplay, such as those built with Epic Games’ Unreal Engine, are often much larger than we expect from traditional web applications. These games ship with large assets that must be stored in memory so they can be synchronously loaded.

For some of the largest of these apps, a 64-bit browser means the difference between whether or not a game will run. For example, when porting to asm.js it’s recommended to keep heap size to 512mb in a 32-bit browser. That goes up to 2GB in a 64-bit version of Firefox.

Emscripten helps port C and C++ code to run on the Web and deliver native-like performance. For an in-depth look at how assets are stored and accessed using a variety of methods in asm.js/emscripten built applications, read Alon Zakai’s post on Synchronous Execution and Filesystem Access in Emscripten.

Gain faster execution and increased security

64-bit Firefox just goes faster. We get access to new hardware registers and instructions to speed up JavaScript code.

For asm.js code, the increased address space also lets us use hardware memory protection to safely remove bounds checks from asm.js heap accesses. The gains are pretty dramatic: 8%-17% on the asmjs-apps-*-throughput tests as reported on arewefastyet.com.

The larger 64-bit address space also improves the effectiveness of ASLR (address space layout randomization), making it more difficult for web content to exploit the browser.

Firefox Developer Edition additions and improvements

Beyond the new 64-bit capabilities, the Firefox 38 Developer Edition release implements many new features, as it does every 6 weeks when it is updated. Some of these are described below. For all the details and associated bugs in progress, you’ll want to visit the release notes.

The new version of Firefox Developer Edition fixes these issues. We now support adding multiple media streams (camera, screen sharing, audio stream) to the same mozRTCPeerConnection within a WebRTC conversation. This allows the developer to call the addStream method for each additional stream, which in turn triggers the onAddStream event for the clients.

Renegotiation allows streams to be modified during a conversation, for example sharing a screen stream during a conversation. This is now possible without re-creating a session.

The BroadcastChannel API

The BroadcastChannel API allows simple messaging between browser contexts with the same user agent and origin is now available. Here’s more detail and some ideas for how to use the BroadcastChannel API in Firefox 38.

Support for KeyboardEvent.code

KeyboardEvent.code is now enabled by default. The code attribute give a developer the ability to determine which physical key is pressed without keyboard layout or keyboard state modifications.

KeyboardEvent code attribute

For more examples of uses cases see the motivation section of the UI Events Specification (formerly DOM Level 3 Events).

XHR logging

The Network Monitor already displays a great deal of information on XMLHttpRequests, but often the console is used to debug code along with network requests. In the latest Developer Edition of Firefox, the console now supports filtering XMLHttpRequests within console logging.

Network Monitor XHR Request

XHR logging in console

Let us know what you think

Many additional improvements are available in this version. Download it now. Tell a friend.

]]>https://hacks.mozilla.org/2015/03/firefox-developer-edition-38-64-bits-and-more/feed/32Birdsongs, Musique Concrète, and the Web Audio APIhttps://hacks.mozilla.org/2015/02/birdsongs-musique-concrete-and-the-web-audio-api/
https://hacks.mozilla.org/2015/02/birdsongs-musique-concrete-and-the-web-audio-api/#commentsThu, 26 Feb 2015 23:12:34 +0000https://hacks.mozilla.org/?p=28009In January 2015, my friend and collaborator Brian Belet and I presented Oiseaux de Même — an audio soundscape app created from recordings of birds — at the first Web Audio Conference. In this post I’d like to describe my experience of implementing this app using the Web Audio API, Twitter Bootstrap, Node.js, and REST APIs.

Screenshot showing Birds of a Feather, a soundscape created with field recordings of birds that are being seen in your vicinity.

What is it? Musique Concrète and citizen science

We wanted to create a web-based Musique Concrète, building an artistic sound experience by processing field recordings. We decided to use xeno-canto — a library of over 200,000 recordings of 9,000 different bird species — as our source of recordings. Almost all the recordings are licensed under Creative Commons by their generous recordists. We select recordings from this library based on data from eBird, a database of tens of millions of bird sightings contributed by bird watchers everywhere. By using the Geolocation API to retrieve eBird sightings near to the listeners’ location, our soundscape can consist of recordings of bird species that bird watchers have reported recently near the listener — each user gets a personalized soundscape that changes daily.

Use of the Web Audio API

We use the browser’s Web Audio API to play back the sounds from xeno-canto. The Web Audio API allows developers to play back, record, analyze, and process sound by creating AudioNodes that are connected together, like an old modular synthesizer.

Our soundscape is implemented using four AudioBuffer nodes, each of which plays a field recording in a loop. These loops are placed in a stereo field using Panner nodes, and mixed together before being sent to the listener’s speakers or headphones.

Controls

After all the sounds have loaded and begin playing, we offer users several controls for manipulating the sounds as they play:

The Pan button randomizes the spatial location of the sound in 3D space.

The Rate button randomizes the playback rate.

The Reverse button reverses the direction of sound playback.

Finally, the Share button lets you capture the state of the soundscape and save that snapshot for later.

The controls described above are implemented as typical JavaScript event handlers. When the Pan button is pressed, for example, we run this handler:

// sets the X,Y,Z position of the Panner to random values between -1 and +1
BirdSongPlayer.prototype.randomizePanner=function(){this.resetLastActionTime();// NOTE: x = -1 is LEFTthis.panPosition={ x:2*Math.random()-1, y:2*Math.random()-1, z:2*Math.random()-1}this.panner.setPosition(this.panPosition.x,this.panPosition.y,this.panPosition.z);}

Some parts of the Web Audio API are write-only

I had a few minor issues where I had to work around shortcomings in the Web Audio API. Other authors have already documented similar experiences; I’ll summarize mine briefly here:

Can’t read Panner position: In the event handler for the Share button, I want to retrieve and store the current Audio Buffer playback rate and Panner position. However, the current Panner node does not allow retrieval of the position after setting it. Hence, I store the new Panner position in an instance variable in addition to calling setPosition().

This has had a minimal impact on my code so far. My longer-term concern is that I’d rather store the position in the Panner and retrieve it from there, instead of storing a copy elsewhere. In my experience, multiple copies of the same information becomes a readability and maintainability problem as code grows bigger and more complex.

Can’t read AudioBuffer’s playbackRate: The Rate button described above calls linearRampToValueAtTime() on the playbackRate AudioParam. As far as I can tell, AudioParams don’t let me retrieve their values after calling linearRampToValueAtTime(), so I’m obliged to keep a duplicate copy of this value in my JS object.

Can’t read AudioBuffer playback position: I’d like to show the user the current playback position for each of my sound loops, but the API doesn’t provide this information. Could I compute it myself? Unfortunately, after a few iterations of ramping an AudioBuffer’s playbackRate between random values, it is very difficult to compute the current playback position within the buffer. Unlike some API users, I don’t need a highly accurate position, I just want to visualize for my users when the current sound loop restarts.

Debugging with the Web Audio inspector

Firefox’s Web Audio inspector shows how Audio Nodes are connected to one another.

I had great success using Firefox’s Web Audio inspector to watch my Audio Nodes being created and interconnected as my code runs.

In the screenshot above, you can see the four AudioBufferSources, each feeding through a GainNode and PannerNode before being summed by an AudioDestination. Note that each recording is also connected to an AnalyzerNode; the Analyzers are used to create the scrolling amplitude graphs for each loop.

Visualizing sound loops

As the soundscape evolves, users often want to know which bird species is responsible for a particular sound they hear in the mix. We use a scrolling visualization for each loop that shows instantaneous amplitude, creating distinctive shapes you can correlate with what you’re hearing. The visualization uses the Analyzer node to perform a fast Fourier transform (FFT) on the sound, which yields the amplitude of the sound at every frequency. We compute the average of all those amplitudes, and then draw that amplitude at the right edge of a Canvas. As the contents of the Canvas shift sideways on every animation frame, the result is a horizontally scrolling amplitude graph.

What’s next

I’m continuing to work on cleaning up my JavaScript code for this project. I have several user interface improvements suggested by my Mozillia colleagues that I’d like to try. And Prof. Belet and I are considering what other sources of geotagged sounds we can use to make more soundscapes with. In the meantime, please try Oiseaux de Même for yourself and let us know what you think!

]]>https://hacks.mozilla.org/2015/02/birdsongs-musique-concrete-and-the-web-audio-api/feed/6WebRTC requires Perfect Forward Secrecy (PFS) starting in Firefox 38https://hacks.mozilla.org/2015/02/webrtc-requires-perfect-forward-secrecy-pfs-starting-in-firefox-38/
https://hacks.mozilla.org/2015/02/webrtc-requires-perfect-forward-secrecy-pfs-starting-in-firefox-38/#commentsWed, 25 Feb 2015 15:19:59 +0000https://hacks.mozilla.org/?p=27959Today, we are announcing that Firefox 38 will take further measures to secure users’ communications by removing support in WebRTC for all DTLS cipher suites that do not support forward secrecy. For developers: if you have a WebRTC application or server that doesn’t support PFS ciphers, you will need to update your code.

Forward secrecy, also known as Perfect Forward Secrecy (PFS), is a feature of a cryptographic protocol that limits the damage of a key compromise: “This means that the compromise of one [session] cannot lead to the compromise of others, and also that there is not a single secret value which can lead to the compromise of multiple [sessions]”.

The PFS suites in TLS and DTLS use an ephemeral Diffie-Hellman key exchange (DHE) or elliptic-curve Diffie-Hellman (ECDHE) to create a different shared secret key for each session. The WebRTC security architecture recommends that PFS suites be preferred for WebRTC.

Due to bug 102794, Firefox is unable to act as a server for DHE cipher suites. We plan to add complete DHE support, but until then we recommend the use of the ECDHE cipher suites.

Existing users of the webrtc.org codebase who are using OpenSSL and derivatives such as BoringSSL need to update to enable ECDHE ciphers. This bug contains more details.

If you have a WebRTC application or server that doesn’t support PFS ciphers, you should be working on getting that resolved ASAP. Firefox 38 is scheduled for Beta the week of March 30th, and a general release is planned for Tuesday, May 12th.

]]>https://hacks.mozilla.org/2015/02/webrtc-requires-perfect-forward-secrecy-pfs-starting-in-firefox-38/feed/2Synchronous Execution and Filesystem Access in Emscriptenhttps://hacks.mozilla.org/2015/02/synchronous-execution-and-filesystem-access-in-emscripten/
https://hacks.mozilla.org/2015/02/synchronous-execution-and-filesystem-access-in-emscripten/#commentsTue, 24 Feb 2015 15:39:22 +0000https://hacks.mozilla.org/?p=27815Emscripten helps port C and C++ code to run on the Web. When doing such porting, we have to work around limitations of the web platform, one of which is that code must be asynchronous: you can’t have long-running code on the Web, it must be split up into events, because other important things – rendering, input, etc. – can’t happen while your code is running. But, it is common to have C and C++ code that is synchronous! This post will review how Emscripten helps handle this problem, using a variety of methods. We’ll look at preloading a virtual filesystem as well as a recently-added option to execute your compiled code in a special interpreter. We’ll also get the chance to play some Doom!

First, let’s take a more concrete look at the problem. Consider, for example,

FILE *f =fopen("data.txt","rb");fread(buffer,100,1, f);fclose(f);

This C code opens a file and reads from it synchronously. Now, in the browser we don’t have local filesystem access (content is sandboxed, for security), so when reading a file, we might be issuing a remote request to a server, or loading from IndexedDB – both of which are asynchronous! How, then, does anything get ported at all? Let’s go over three approaches to handling this problem.

1. Preloading to Emscripten’s virtual filesystem

The first tool Emscripten has is a virtual in-memory filesystem, implemented in JavaScript (credit goes to inolen for most of the code), which can be pre-populated before the program runs. If you know which files will be accessed, you can preload them (using emcc’s –preload-file option), and when the code executes, copies of the files are already in memory, ready for synchronous access.

On small to medium amounts of data, this is a simple and useful technique. The compiled code doesn’t know it’s using a virtual filesystem, everything looks normal and synchronous to it. Things just work. However, with large amounts of data, it can be too expensive to preload it all into memory. You might only need each file for a short time – for example, if you load it into a WebGL shader, and then forget about it on the CPU side – but if it’s all preloaded, you have to hold it all in memory at once. Also, the Emscripten virtual filesystem works hard to be as POSIX-compliant as it can, supporting things like permissions, mmap, etc., which add overhead that might be unnecessary in some applications.

How much of a problem this is depends not just on the amount of data you load, but also the browser and the operating system. For example, on a 32-bit browser you are generally limited to 4GB of virtual address space, and fragmentation can be a problem. For these reasons, 64-bit browsers can sometimes succeed in running applications that need a lot of memory while 32-bit browsers fail (or fail some of the time). To some extent you can try to work around memory fragmentation problems by splitting up your data into separate asset bundles, by running Emscripten’s file packager separately several times, instead of using –preload-file once for everything. Each bundle is a combination of JavaScript that you load on your page, and a binary file with the data of all the files you packaged in that asset bundle, so in this way you get multiple smaller files rather than one big one. You can also run the file packager with –no-heap-copy, which will keep the downloaded asset bundle data in separate typed arrays instead of copying them into your program’s memory. However, even at best, these things can only help some of the time with memory fragmentation, in an unpredictable manner.

Preloading all the data is therefore not always a viable solution: With large amounts of data, we might not have enough memory, or fragmentation might be a problem. Also, we might not know ahead of time which files we will need. And in general, even if preloading works for a project, we would still like to avoid it so that we can use as little memory as possible, as things generally run faster that way. That’s why we need the 2 other approaches to handling the problem of synchronous code, which we will discuss now.

And if you have synchronous code doing something other than filesystem access, for example rendering, Emscripten provides a generic API to do an asynchronous callback (emscripten_async_call). For the common case of a main loop which should be called once per frame from the browser’s event loop, Emscripten has a main loop API (emscripten_set_main_loop, etc.).

Concretely, an fread() would be replaced with something like

emscripten_async_wget_data("filename.txt",0, onLoad, onError);

where the first parameter is the filename on the remote server, then an optional void* argument (that will be passed to the callbacks), then callbacks on load and on error. The tricky thing is that the code that should execute right after the fread() would need to be in the onLoad callback – that’s where the refactoring comes in. Sometimes this is easy to do, but it might not be.

Refactoring code to be asynchronous is generally the optimal thing to do. It makes your application use the APIs that are available on the Web in the way they are intended to be used. However, it does require changes to your project, and may require that the entire thing be designed in an event-friendly manner, which can be difficult if it wasn’t already structured that way. For these reasons, Emscripten has one more approach that can help you here.

The Emterpreter is a fairly new option in Emscripten that was initially developed for startup-time reasons. It compiles your code into a binary bytecode, and ships it with a little interpreter (written in JavaScript, of course), in which the code can be executed. Code running in an interpreter is “manually executed” by us, so we can control it more easily than normal JavaScript, and we can add the capability to pause and resume, which is what we need to turn synchronous code into asynchronous code. Emterpreter-Async, the Emterpreter plus support for running synchronous code asynchronously, was therefore fairly easy to add on top of the existing Emterpreter option.

The idea of an automatic transformation from synchronous to asynchronous code was experimented with by Lu Wang during his internship over the summer of 2014: the Asyncify option. Asyncify rewrites code at the LLVM level to support pausing and resuming execution: you write synchronous code, and the compiler rewrites it to run asynchronously. Returning to the fread() example from before, Asyncify would automatically break up the function around that call, and put the code after the call into a callback function – basically, it does what we suggested you do manually in the “Refactor code to be asynchronous” section above. This can work surprisingly well: For example, Lu ported vim, a large application with a lot of synchronous code in it, to the Web. And it works! However, we hit significant limitations in terms of increased code size because of how Asyncify restructures your code.

The Emterpreter’s async support avoids the code size problem that Asyncify hit because it is an interpreter running bytecode: The bytecode is always the same size (in fact, smaller than asm.js), and we can manipulate control flow on it manually in the interpreter, without instrumenting the code.

Of course, running in an interpreter can be quite slow, and this one is no exception – speed can be significantly slower than usual. Therefore, this is not a mode in which you want to run most of your code. But, the Emterpreter gives you the option to decide which parts of your codebase are interpreted and which are not, and this is crucial to productive use of this option, as we will now see.

Let’s make this concrete by showing the option in practice on the Doom codebase. Here is a normal port of Doom (specifically Boon:, the Doom code with Freedoom open art assets). That link is just Doom compiled with Emscripten, not using synchronous code or the Emterpreter at all, yet. It looks like the game works in that link – do we even need anything else? It turns out that we need synchronous execution in two places in Doom: First, for filesystem access. Since Doom is from 1993, the size of the game is quite small compared to today’s hardware. We can preload all of the data files and things just work (that’s what happens in that link). So far, so good!

The second problem, though, is trickier: For the most part Doom renders a whole frame in each iteration of the main loop (which we can call from the browser’s event loop one at a time), however it also does some visual effects using synchronous code. Those effects are not shown in that first link – Doom fans may have noticed something was missing! :)

Here is a build with the Emterpreter-Async option enabled. This runs the entire application as bytecode in the interpreter, and it’s quite slow, as expected. Ignoring speed for now, you might notice that when you start a game, there is a “wipe” effect right before you begin to play, that wasn’t in the previous build. It looks kind of like a descending wave. Here’s a screenshot:

That effect is written synchronously (note the screen update and sleep). The result is that in the initial port of the game, the wipe effect code is executed, but the JavaScript frame doesn’t end yet so no rendering happens. For this reason, we don’t see the wipe in the first build! But we do see it in the second, because we enabled the Emterpreter-Async option, which supports synchronous code.

The second build is slow. What can we do? The Emterpreter lets you decide which code runs normally, as full-speed asm.js, and which is interpreted. We want to run only what we absolutely must run in the interpreter, and everything else in asm.js, so things are as fast as possible. For purposes of synchronous code, the code we must interpret is anything that is on the stack during a synchronous operation. To understand what that means, imagine that the callstack currently looks like this:

main()=> D_DoomMain()=> D_Display()=> D_Wipe()=> I_uSleep()

and the last of those does a call to sleep. Then the Emterpreter turns this synchronous operation into an asynchronous operation by saving where execution is right now in the current method (this is easy using the interpreter’s program counter, as well as since all local variables are already stored in a stack on a global typed array), then doing the same for the methods calling it, and while doing so to exit them all (which is also easy, each call to the interpreter is a call to a JavaScript method, which just returns). After that, we can do a setTimeout() for when we want to resume. So far, we have saved what we were doing, stopped, set an asynchronous callback for some time in the future, and we can then return control to the browser’s event loop, so it can render and so forth.

When the asynchronous callback fires sometime later, we reverse the first part of the process: We call into the interpreter for main(), jump to the right position in it, then continue to do so for the rest of the call stack – basically, recreating the call stack exactly as it was before. At this point we can resume execution in the interpreter, and it is as if we never left: synchronous execution has been turned asynchronous.

That means that if D_Wipe() does a synchronous operation, it must be interpreted, and anything that can call it as well, and so forth, recursively. The good news is that often such code tends to be small and doesn’t need to be fast: it’s typically event-loop handling code, and not code actually doing hard work. Talking abstractly, it’s common to see callstacks like these in games:

main()=> MainLoop()=> RunTasks()=> PhysicsTask()=> HardWork()

and

main()=> MainLoop()=> RunTasks()=> IOTask()=> LoadFile()

Assuming LoadFile() does a synchronous read of a file, it must be interpreted. As we mentioned above, this means everything that can be on the stack together with it must also be interpreted: main(), MainLoop(), RunTasks(), and IOTask() – but not any of the physics methods. In other words, if you never have physics and networking on the stack at the same time (a network event calling something that ends up calling physics, or a physics event that somehow decides to do a network request all of a sudden), then you can run networking in the interpreter, and physics at full speed. This is the case in Doom, and also other real-world codebases (and even in ones that are tricky, as in Em-DOSBox which has recursion in a crucial method, sometimes a solution can be found).

Here is a build of Doom with that optimization enabled – it only interprets what we absolutely must interpret. It runs at about the same speed as the original, optimized build and it also has the wipe effect fully working. Also, the wipe effect is nice and smooth, which it wasn’t before: even though the wipe method itself must be interpreted – because it calls sleep() – the rendering code it calls in between sleeping can run at full speed, as that rendering code is never on the stack while sleeping!

To have synchronous code working properly while the project stays at full speed, it is crucial to run exactly the right methods in the interpreter. Here is a list of the methods we need in Doom (in the ‘whitelist’ option there) – only 15 out of 1,425, or ~1%. To help you find a list for your project, the Emterpreter provides both static and dynamic tools, see the docs for more details.

Conclusion

Emscripten is often used to port code that contains synchronous portions, but long-running synchronous code is not possible on the Web. As described in this article, there are three approaches to handling that situation:

If the synchronous code just does file access, then preloading everything is a simple solution.

However, if there is a great amount of data, or you don’t know what you’ll need ahead of time, this might not work well. Another option is to refactor your code to be asynchronous.

If that isn’t an option either, perhaps because the refactoring is too extensive, then Emscripten now offers the Emterpreter option to run parts of your codebase in an interpreter which does support synchronous execution.

Together, these approaches provide a range of options for handling synchronous code, and in particular the common case of synchronous filesystem access.

It’s been a while since we said anything on Hacks about the Web Audio API. However, with Firefox 37/38 hitting our Developer Edition/Nightly browser channels, there are some interesting new features to talk about!

This article presents you with some new Web Audio tricks to watch out for, such as the new StereoPannerNode, promise-based methods, and more.

Simple stereo panning

Firefox 37 introduces the StereoPannerNode interface, which allows you to add a stereo panning effect to an audio source simply and easily. It takes a single property: pan—an a-rate AudioParam that can accept numeric values between -1.0 (full left channel pan) and 1.0 (full right channel pan).

But don’t we already have a PannerNode?

You may have already used the older PannerNode interface, which allows you to position sounds in 3D. Connecting a sound source to a PannerNode causes it to be “spatialised”, meaning that it is placed into a 3D space and you can then specify the position of the listener inside. The browser then figures out how to make the sources sound, applying panning and doppler shift effects, and other nice 3D “artifacts” if the sounds are moving over time, etc:

var audioContext =new AudioContext();var pannerNode = audioContext.createPanner();// The listener is 100 units to the right of the 3D origin
audioContext.listener.setPosition(100,0,0);// The panner is in the 3D origin
pannerNode.setPosition(0,0,0);

This works well with WebGL-based games as both environments use similar units for positioning—an array of x, y, z values. So you could easily update the position, orientation, and velocity of the PannerNodes as you update the position of the entities in your 3D scene.

But what if you are just building a conventional music player where the songs are already stereo tracks, and you actually don’t care at all about 3D? You have to go through a more complicated setup process than should be necessary, and it can also be computationally more expensive. With the increased usage of mobile devices, every operation you don’t perform is a bit more battery life you save, and users of your website will love you for it.

Enter StereoPannerNode

StereoPannerNode is a much better solution for simple stereo use cases, as described above. You don’t need to care about the listener’s position; you just need to connect source nodes that you want to spatialise to a StereoPannerNode instance, then use the pan parameter.

To use a stereo panner, first create a StereoPannerNode using createStereoPanner(), and then connect it to your audio source. For example:

Also, since pan is an a-rate AudioParam you can design nice smooth curves using parameter automation, and the values will be updated per sample. Trying to do this kind of change over time would sound weird and unnatural if you updated the value over multiple requestAnimationFrame calls. And you can’t automate PannerNode positions either.

For example, this is how you could set up a panning transition from left to right that lasts two seconds:

The browser will take care of updating the pan value for you. And now, as of recently, you can also visualise these curves using the Firefox Devtools Web Audio Editor.

Detecting when StereoPannerNode is available

It might be the case that the Web Audio implementation you’re using has not implemented this type of node yet. (At the time of this writing, it is supported in Firefox 37 and Chrome 42 only.) If you try to use StereoPannerNode in these cases, you’re going to generate a beautiful undefined is not a function error instead.

To make sure StereoPannerNodes are available, just check whether the createStereoPanner() method exists in your AudioContext:

Changes to the default PannerNode panning algorithm

The default panning algorithm type used in PannerNodes used to be HRTF, which is a high quality algorithm that rendered its output using a convolution with human-based data (thus it’s very realistic). However it is also very computationally expensive, requiring the processing to be run in additional threads to ensure smooth playback.

Authors often don’t require such a high level of quality and just need something that is good enough, so the default PannerNode.type is now equalpower, which is much cheaper to compute. If you want to go back to using the high quality panning algorithm instead, you just need to change the type:

pannerNodeInstance.type='HRTF';

Incidentally, a PannerNode using type = 'equalpower' results in the same algorithm that StereoPannerNode uses.

Promise-based code

In both cases the differences don’t seem major, but if you’re composing the results of promises sequentially or if you’re waiting on an event to complete before calling several other methods, promises are really helpful to avoid callback hell.

Detecting support for Promise-based methods

Again, you don’t want to get the dreaded undefined is not a function error message if the browser you’re running your code on doesn’t support these new versions of the methods.

A quick way to check for support: look at the returned type of these calls. If they return a Promise, we’re in luck. If they don’t, we have to keep using the old methods:

Audio workers

Although the spec has not been finalised and they are not implemented in any browser yet, it is also worth giving a mention to Audio Workers, which —you’ve guessed it— are a specialised type of web worker for use by Web Audio code.

Audio Workers will replace the almost-obsolete ScriptProcessorNode. Originally, this was the way to run your own custom nodes inside the audio graph, but they actually run on the main thread causing all sorts of problems, from audio glitches (if the main thread becomes stalled) to unresponsive UI code (if the ScriptProcessorNodes aren’t fast enough to process their data).

The biggest feature of audio workers is that they run in their own separate thread, just like any other Worker. This ensures that audio processing is prioritised and we avoid sound glitches, which human ears are very sensitive to.

There is an ongoing discussion on the w3c web audio list; if you are interested in this and other Web Audio developments, you should go check it out.

The characters that indicate items in a list are called counters — they can be bullets or numbers. They are defined using the list-style-type CSS property. CSS1 introduced a list of predefined styles to be used as counter markers. The initial list was then slightly extended with addition of more predefined counter styles in CSS2.1. Even with 14 predefined counter styles, it still failed to address use cases from around the world.

Now, the CSS Counter Styles Level 3 specification, which reached Candidate Recommendation last week, adds new predefined counter styles to the existing list that should address most common counter use cases.

In addition to the new predefined styles, the spec also offers an open-ended solution for the needs of worldwide typography by introducing the @counter-style at-rule — which lets us define custom counter styles or even extend existing ones — and the symbols() function. The latter is a shorthand for defining inline styles and is useful when the fine-grained control that @counter-style offers is not needed.

This article provides a guide to using the new counter features of CSS Level 3.

@counter-style

A @counter-style at-rule is identified by a name and defined with a set of descriptors. It takes a numerical counter value and converts it into a string or image representation. A custom counter style definition looks like this:

system – the system descriptor lets the author specify an algorithm to convert the numerical counter value to a basic string representation. For example, the cyclic system cycles repeatedly through the list of symbols provided to create counter representations.

negative – the negative descriptor provides the ability to specify additional symbols, such as a negative sign, to be appended/prepended to a counter representation if the counter value is negative. If only one symbol is specified, it will be added in front of the marker representation. A second value, if specified (as in the example below), will be added after the marker.

The above counter style definition will wrap negative markers in a pair of parentheses, instead of preceding them with the minus sign.

prefix – specifies a symbol that will be prepended to the final counter representation.

suffix – specifies a symbol that will be appended to the final counter representation.The default value of the suffix descriptor is a full stop followed by a space. If you need to replace the full stop “.” with a parenthesis, you can specify the suffix descriptor:

range – range lets the author define a range of values over which a counter style is applicable. If a counter value falls outside the specified range, then the specified or default fallback style is used to represent that particular counter value, for example:

The above rule will apply the style only for counter values starting from 4 to 8. The fallback style, decimal, will be used for the rest of the markers.

pad – as the name suggests, the pad descriptor lets the author specify a padding length when the counter representations need to be of a minimum length. For example, if you want the counters to start at 01 and go through 02, 03, 04, etc., then the following pad descriptor should be used:

(For a better way to achieve the same result, but without specifying the counter symbols, see the section below about extending existing systems.)

fallback – the fallback specifies a style to fall back to if the specified system is unable to construct a counter representation or if a particular counter value falls outside the specified range.

symbols and additive-symbols – specify the symbols that will be used to construct the counter representation. The symbols descriptor is used in most cases other than when the system specified is ‘additive.’ While symbols specifies individual symbols, the additive-symbols descriptor is composed of what are known as additive tuples – , which each consist of a symbol and a non-negative weight.Symbols specified can be strings, images in the form url(image-name) or identifiers. The below example uses images as symbols.

speak-as – specifies how a counter value should be spoken by speech synthesizers such as screen readers. For example, the author can specify a marker symbol to be read out as a word, a number, or as an audio cue for a bullet point. The example below reads out the counter values as numbers.

You can then use a named style with list-style-type or with the counters() function as you normally would use a predefined counter style.

ul {list-style-type: circled-digits;}

On a supported browser, the counter style above will render lists with markers like this:

The system descriptor

The system descriptor takes one of the predefined algorithms, which describe how to convert a numerical counter value to its representation. The system descriptor can have values that are cyclic, numeric, alphabetic, symbolic, additive, fixed and extends.

If the system specified is cyclic, it will cycle through the list of symbols provided. Once the end of the list is reached, it will loop back to the beginning and start over. The fixed system is similar, but once the list of symbols is exhausted, it will resort to the fallback style to represent the rest of the markers. The symbolic, numeric and alphabetic systems are similar to the above two, but with their own subtle differences.

The additive system requires that the additive-symbols descriptor be specified instead of the symbols descriptor. additive-symbols takes a list of additive tuples. Each additive tuple consists of a symbol and its numerical weight. The additive system is used to represent “sign-value” numbering systems such as the Roman numerals.

We can rewrite the Roman numerals as a @counter-style rule using the additive system like this:

Extending existing or custom counter styles

The extends system lets authors create custom counter styles based on existing ones. Authors can use the algorithm of another counter style, but alter its other aspects. If a @counter-style rule using the extends system has any unspecified descriptors, their values will be taken from the extended counter style specified.

For example, if you want to define a new style which is similar to decimal, but has a string “Chapter ” in front of all the marker values, then rather than creating an entirely new style you can use the extends system to extend the decimal style like this:

Or, if you need the decimal style to start numbers from 01, 02, 03.. instead of 1, 2, 3.., you can simply specify the pad descriptor:

@counter-style decimal-new {
system: extends decimal;
pad:2"0";}

A counter style using the extends system should not specify the symbols and additive-symbols descriptors.

See the reference page on MDN for details and usage examples of all system values.

A shorthand – The symbols() function

While a @counter-style at-rule lets you tailor a custom counter style, often such elaborate control is not needed. That is where the symbols() function comes in. symbols() lets you define inline counter styles as function properties. Since the counter styles defined in this way are nameless (anonymous), they can not be reused elsewhere in the stylesheet.

An example anonymous counter style looks like this:

ul {list-style-type: symbols(fixedurl(one.svg)url(two.svg));}

Even more predefined counter styles

CSS Lists 3 vastly extends the number of predefined styles available. Along with the predefined styles such as decimal, disc, circle and square that existed before, additional styles such as hebrew, thai, gujarati, disclosure-open/close, etc. are also added to the predefined list. The full list of predefined styles enabled by the spec can be seen here.

Browser support

Support for @counter-style and the symbols() function along with the new predefined styles landed in Firefox 33. Google Chrome hasn’t implemented @counter-styles or the symbols() function yet, but most of the new numeric and alphabetic predefined styles have been implemented. There is no support in IE so far. Support has already landed in the latest releases of Firefox for Android and in the upcoming Firefox OS 2.1, which has support for @counter-style. Support for symbols() will land in Firefox OS 2.2.

Examples

Check out this demo page and see if @counter-style is supported in your favorite browsers.

]]>https://hacks.mozilla.org/2015/02/introducing-counter-styles/feed/3Exploring object-fithttps://hacks.mozilla.org/2015/02/exploring-object-fit/
https://hacks.mozilla.org/2015/02/exploring-object-fit/#commentsTue, 10 Feb 2015 15:25:44 +0000https://hacks.mozilla.org/?p=27733On web documents, a common problem concerns the display of different sized images (or videos) in the same place. Perhaps you are writing a dynamic gallery app that accepts user submissions. You can’t guarantee that everyone will upload images of exactly the same aspect ratio, so what do you do?

Letting the aspect ratio distort to fit the containing replaced element nearly always looks horrible. And doing some kind of dynamic cropping or resizing on the fly each time might be more work than you have the capability to implement. (For instance, maybe you’re working on a CMS and don’t have permission to edit anything except the page content.)

These elements have decent support across modern browsers (with the exception of IE). In this article we’ll look at a few examples of how they can be used.

Note: object-fit works with SVG content, but the same effect can also be achieved by setting the preserveAspectRatio="" attribute in the SVG itself.

How do object-fit and object-position work?

You can successfully apply object-fit to any replaced element, for example:

img {height:100px;width:100px;
object-fit: contain;}

The five possible values of object-fit are as follows:

contain: The content (e.g. the image) will be resized so that it is fully displayed with intrinsic aspect ratio preserved, but still fits inside the dimensions set for the element.

fill: The content will expand to exactly fill the dimensions set for the element, even if this does break its aspect ratio.

cover: Preserves the aspect ratio of the content, but alters the width and height so that the content completely covers the element. The smaller of the two is made to fit the element exactly, and the larger overflows the element and is cropped.

none: Completely ignores any height or weight set on the element, and just uses the replaced element content’s intrinsic dimensions.

scale-down: The content is sized as if none or contain were specified, whichever would result in a smaller replaced element size.

object-position works in exactly the same way as background-position does for background images; for example:

The effects of the different object-fit values

The following code examples show the effects of the different object-fit values.

Letterboxing images with object-fit: contain

Sometimes referred to as letter-boxing, there are times when you will want to preserve the aspect ratio of the images on a page, but get them to fit inside the same area. For example, you might have a content management system that allows you to upload products on an e-commerce site or images for an image gallery, with lots of different content authors. They may upload images in roughly the right size, but the dimensions are not always exact, and you want to fit each image into the same amount of space.

Having images with the aspect ratio shifted usually looks horrible, so you can letterbox them instead with object-fit: contain (object-fit: contain example):

Cropping images with object-fit:cover

A different solution is to maintain aspect ratio, but crop each image to the same size so it completely envelops the <img> element, with any overflow being hidden. This can be done easily with object-fit:cover (object-fit: cover example):

.cover{
object-fit: cover;}

Overriding a video’s aspect ratio with object-fit: fill

Going in the opposite direction, it is also possible to take a video and force it to change aspect ratio. Maybe some of your content editor’s videos have a broken aspect ratio, and you want to fix them all on the fly, in one easy fell swoop?

It would look terrible: the video would appear letter-boxed, since the <video> element always tries to maintain the source file’s intrinsic aspect ratio. We could fix this by applying object-fit: fill (object-fit: fill example):

.fill{
object-fit: fill;}

This overrides the video’s intrinsic aspect ratio, forcing it to completely fill the <video> element so it displays correctly.

Interesting transition effects

Combining object-fit and object-position with CSS transitions can lead to some pretty interesting effects for image or video galleries. For example:

Only a small part of the image is shown, and the element grows to reveal more of the image when it is focused/hovered (object-fit: none example).

This is because by setting object-fit: none on the <img>, we cause the content to completely ignore any width and height set earlier, and spill out of the sides of the element. We then use overflow: hidden to crop anything that spills out. A transition is then used to smoothly increase the size of the <img> element when it’s hovered/focused, which reveals more of the image.

Gallery example

To show a slightly more applied usage of object-fit, we have created a gallery example:

The 16 images are loaded via XHR, and inserted into the images as ObjectURLs.

Each image in turn is given an onclick handler so that when clicked the images appear full size, filling the screen (the main image, initially set to display: none; in the CSS is given a class of blowup, which makes it display and fill the whole screen; the main image’s src is then set to the same object URL as the thumb that was clicked).

Note: the thumbnails have all been given tabindex="0" to make them focusable by tabbing (you can make anything appear in the page’s tab order by setting on tabindex="0" on it), and the onclick handler that makes the full size images appear has been doubled up with an onfocus handler to provide basic keyboard accessibility:

The thumbnails: These have object-fit: cover set on them so that all image thumbs will appear at the same size, at the proper aspect ratio, but cropped different amounts. This looks pretty decent, and creates a nice effect when you resize the window.

The main image: This has object-fit: contain and object-position: center set on it, so that it will appear in full, at the correct aspect ratio and as big as it can be.

]]>https://hacks.mozilla.org/2015/02/exploring-object-fit/feed/15Embedding an HTTP Web Server in Firefox OShttps://hacks.mozilla.org/2015/02/embedding-an-http-web-server-in-firefox-os/
https://hacks.mozilla.org/2015/02/embedding-an-http-web-server-in-firefox-os/#commentsMon, 09 Feb 2015 14:41:58 +0000https://hacks.mozilla.org/?p=27752Nearing the end of last year, Mozilla employees were gathered together for a week of collaboration and planning. During that week, a group was formed to envision what the future of Firefox OS might be surrounding a more P2P-focused Web. In particular, we’ve been looking at harnessing technologies to collectively enable offline P2P connections such as Bluetooth, NFC and WiFi Direct.

Since these technologies only provide a means to communicate between devices, it became immediately clear that we would also need a protocol for apps to send and receive data. I quickly realized that we already have a standard protocol for transmitting data in web apps that we could leverage – HTTP.

By utilizing HTTP, we would already have everything we’d need for apps to send and receive data on the client side, but we would still need a web server running in the browser to enable offline P2P communications. While this type of HTTP server functionality might be best suited as part of a standardized WebAPI to be baked into Gecko, we actually already have everything we need in Firefox OS to implement this in JavaScript today!

navigator.mozTCPSocket

Packaged apps have access to both raw TCP and UDP network sockets, but since we’re dealing with HTTP, we only need to work with TCP sockets. Access to the TCPSocket API is exposed through navigator.mozTCPSocket which is currently only exposed to “privileged” packaged apps with the tcp-socket permission:

"type":"privileged","permissions":{"tcp-socket":{}},

In order to respond to incoming HTTP requests, we need to create a new TCPSocket that listens on a known port such as 8080:

var socket = navigator.mozTCPSocket.listen(8080);

When an incoming HTTP request is received, the TCPSocket needs to handle the request through the onconnect handler. The onconnect handler will receive a TCPSocket object used to service the request. The TCPSocket you receive will then call its own ondata handler each time additional HTTP request data is received:

Typically, an HTTP request will result in a single calling of the ondata handler. However, in cases where the HTTP request payload is very large, such as for file uploads, the ondata handler will be triggered each time the buffer is filled, until the entire request payload is delivered.

In order to respond to the HTTP request, we must send data to the TCPSocket we received from the onconnect handler:

The above example sends a proper HTTP response with “Hello World!” in the body. Valid HTTP responses must contain a status line which consists of the HTTP version HTTP/1.1, the response code 200 and the response reason OK terminated by a CR+LF \r\n character sequence. Immediately following the status line are the HTTP headers, one per line, separated by a CR+LF character sequence. After the headers, an additional CR+LF character sequence is required to separate the headers from the body of the HTTP response.

FxOS Web Server

Now, it is likely that we will want to go beyond simple static “Hello World!” responses to do things like parsing the URL path and extracting parameters from the HTTP request in order to respond with dynamic content. It just so happens that I’ve already implemented a basic-featured HTTP server library that you can include in your own Firefox OS apps!

FxOS Web Server can parse all parts of the HTTP request for various content types including application/x-www-form-urlencoded and multipart/form-data. It can also gracefully handle large HTTP requests for file uploads and can send large binary responses for serving up content such as images and videos. You can either download the source code for FxOS Web Server on GitHub to include in your projects manually or you can utilize Bower to fetch the latest version:

bower install justindarc/fxos-web-server --save

Once you have the source code downloaded, you’ll need to include dist/fxos-web-server.js in your app using a <script> tag or a module loader like RequireJS.

Simple File Storage App

Next, I’m going to show you how to use FxOS Web Server to build a simple Firefox OS app that lets you use your mobile device like a portable flash drive for storing and retrieving files. You can see the source code for the finished product on GitHub.

As you can see, I’ve included fxos-web-server.js in addition to app.js. I’ve also included a DeviceStorage helper module called storage.js since enumerating files can get somewhat complex. This will help keep the focus on our code specific to the task at hand.

The first thing we’ll need to do is create new instances of the HTTPServer and Storage objects:

This will initialize a new HTTPServer on port 8080 and a new instance of our Storage helper pointing to the device’s SD card. In order for our HTTPServer instance to be useful, we must listen for and handle the “request” event. When an incoming HTTP request is received, the HTTPServer will emit a “request” event that passes the parsed HTTP request as an HTTPRequest object to the event handler.

The HTTPRequest object contains various properties of an HTTP request including the HTTP method, path, headers, query parameters and form data. In addition to the request data, an HTTPResponse object is also passed to the “request” event handler. The HTTPResponse object allows us to send our response as a file or string and set the response headers:

When a user requests the root URL of our web server, we’ll want to present them with a listing of files stored in the “WebDrive” folder on the device along with a file input for uploading new files. For convenience, we’ll create two helper functions to generate the HTML string to send in our HTTP response. One will just generate the listing of files which we’ll reuse to display the files on the device locally and the other will generate the entire HTML document to send in the HTTP response:

You’ll notice that we’re using ES6 Template Strings for generating our HTML. If you’re not familiar with Template Strings, they allow us to have multi-line strings that automatically include whitespace and new lines and we can do basic string interpolation that automatically inserts values inside the ${} syntax. This is especially useful for generating HTML because it allows us to span multiple lines so our template markup remains highly readable when embedded within JavaScript code.

As of right now, our “request” event handler will always respond with an HTML page listing all the files in the device’s “WebDrive” folder. However, we must first start the HTTPServer before we can receive any requests. We’ll do this once the DOM is ready and while we’re at it, let’s also render the file listing locally:

At this point, our web server should be up and running! Go ahead and install the app on your device or simulator using WebIDE. Once installed, launch the app and point your desktop browser to your device’s IP address at port 8080 (e.g.: http://10.0.1.12:8080).

You should see our index page load in your desktop browser, but the upload form still isn’t wired up and if you have any files in your “WebDrive” folder on your device, they cannot be downloaded yet. Let’s first wire up the file upload by first creating another helper function to save files received in an HTTPRequest:

This function will first convert the file’s contents to an ArrayBuffer using the BinaryUtils utility that comes with fxos-web-server.js. We then create a Blob that we pass to our Storage helper to save it to the SD card in the “WebDrive” folder. Note that the filename can be extracted from the file’s metadata object since it gets passed to the server using ‘multipart/form-data’ encoding.

Now that we have a helper for saving an uploaded file, let’s wire it up in our “request” event handler:

Now, anytime an HTTP POST request is received that contains a “file” parameter in the request body, we will save the file to the “WebDrive” folder on the SD card and respond with an updated file listing index page. At the same time, we’ll also update the file listing on the local device to display the newly-added file.

The only remaining part of our app to wire up is the ability to download files. Once again, let’s update the “request” event handler to do this:

This time, our “request” event handler will check the requested path to see if a URL other than the root is being requested. If so, we assume that the user is requesting to download a file which we then proceed to get using our Storage helper. If the file cannot be found, we return an HTTP 404 error. Otherwise, we set the “Content-Type” in the response header to the file’s MIME type and send the file with the HTTPResponse object.

You can now re-install the app to your device or simulator using WebIDE and once again point your desktop browser to your device’s IP address at port 8080. Now, you should be able to upload and download files from your device using your desktop browser!

The possible use cases enabled by embedding a web server into Firefox OS apps are nearly limitless. Not only can you serve up web content from your device to a desktop browser, as we just did here, but you can also serve up content from one device to another. That also means that you can use HTTP to send and receive data between apps on the same device! Since its inception, FxOS Web Server has been used as a foundation for several exciting experiments at Mozilla: