Tuesday, December 11, 2012

Sustainable JavaScript

JavaScript is certainly one of the most popular programming language [1][2][3] and the never increasing popularity of complex web application will not cut that trend soon.
Despite some intrinsic flaws of the language and environment, trends have emerged to turn JavaScript into clearer, more structured, deployable, testable environment. In this post, I will go through some of them. As a subjective point of view, they will reflect some techniques we daily use in the scope of large scale projects, computing and displaying rich information in bio-informatics.

I will mainly make here a short introduction to underscore.js, require.js and jasmine as the tools that recently made my day change and shine again. I will skip discussions about jQuery, chrome developer tool, html5 and backbone which also are invaluable, among many others.

-"JavaScript sucks"

Despite its ubiquitousness, it is often quoted as a highly hated language among the developer community. Cryptic, unreadable, poorly organized, not portable, conter-intuitive are among the most recurrent criticisms.
And when the question comes to how to adopt good practice (code organization, testing, continuous deployment), the answer is: "yep, but we found no one stop shop, we tried a couple of them and... we gave up."
To get more about some flaws of the language itself, visit JavaScript: the real bad parts, or if you want to go for a really hot argument about a single semi column go here, thanks to Andrew Dupont.

-"but JavaScript is magic"

Looking around at application as Gmail, Google docs, cloud 9 IDE etc., the sky is the limit. Just think of modern css features, html5, webGL. In scientific application, we can use its power to compute graphical rendering and even some mid intensive computations tasks that would otherwise ask to round trip to the server.
And the language escapes from the browser to the server side, with the rise of NoSql database (mongoDB, couchDB) or Node.js.The easiness of deployment (a web browser is the main target), the intrinsic freedom of a dynamic language, the galaxy of available libraries, techniques and frameworks and the heterogeneity of the community makes it no easy to find a simple path for our daily programmer life.

What is sustainability?

"Sustainability in a general sense is the capacity to support, maintain or endure [...] In ecology, sustainability describes how biological systems remain diverse, robust, and productive over time." (wikipedia)
If we go back to software development, sustainability could encompass the abilities:

to adapt to the environment: use productive libraries to improve the language itself;

to evolve: refactoring, survive moving targets;

robustness: enforced by testing;

to meet the environment, the customer: continuous deployment.

I propose here some personal solutions towards which I converged recently. There are for sure plenty of other paths to reach the same goal.

Illustration: the problem of the day

We'll need to illustrate this post, I will not go for so ToDo list example but head for some bio-informatics problem, as it's my daily meat. Don't panic, it should be rather straightforwards:

this protein can be digest by a cleavage enzyme, which is for us equivalent to a regular expression and produce a list of substrings, named peptides; trypsin is a typical cleavage enzyme,

peptide are also string of upper case letter, with an integer offset (their relative position to the protein).

So, we'll have a page with a textarea containing a protein sequence, a button to launch the digestion process, keeping only the peptide longer than a given length and reporting them in a list.

Available libraries: the 2012 winner is... underscore.js

Among jQuery, Backbone, Boostrap, html5 wrappers, I had to pick up one which changed my day: underscore.js

Core JavaScript is rather limited, and we often tend to re-invent the wheel again and again: Sorting list, extracting one the same attribute to a list of maps, checking validity, looping, filtering, getting the minimun value, contains. Zipping, shuffling or intersecting arrays. Mapping data to html templates and so on.

Underscore is a neat little librarie that provides plenty of those commons constructs.

Let's have a list of peptides, sequence and offset:

var peptideList = [{

sequence : "MALWMR",

offset : 0

}, {

sequence : "LLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGER",

offset : 6

}, {

sequence : "GFFYTPK",

offset : 46

}, {

sequence : "TR",

offset : 53

}, {

sequence : "R",

offset : 55

}, {

sequence : "EAEDLQVGQVELGGGPGAGSLQPLALEGSLQK",

offset : 56

}, {

sequence : "R",

offset : 88

}, {

sequence : "GIVEQCCTSICSLYQLENYCN",

offset : 89

}];

Extracting an attribute

We want to make an array with all the sequence:

var otherList = _.pluck(peptideList, "sequence");

Transforming each element

Each element into a composed string:

var otherList = _.collect(peptideList, function(pept) {

return pept.sequence + "("+ pept. offset+ ")"

});

Sorting on a function

Let's sort the list on the sequence length

var sorted = _.sortBy(petideList, function(pept) {

return pept.sequence.length

});

Chaining underscore functions

We can transform a structure by successive calls

_.chain(peptideList).

groupBy(function(pept) {

return pept.sequence

}).

each(function(g) {

$("#target-4").append(g.length + ' x '+ g[0].sequence + '<br/>')

});

Applying templates

Mapping an object to a piece of html is also a common pattern

var tmpl = "<b><%= sequence %></b> (<%= offset %>)"

var content =

_.collect(peptideList, function(pept) {

return _.template(tmpl, pept);

}).join('<br/>');

There is much more to the library and a very good documentation on their web page.

Organizing the source code: breaking into pieces

For our Protein/Enzyme/Peptide problem, we need three types of object, residing in independent files, for clarity sake (they could then reside in folders)

Protein.js

var Protein = function(sequence) {

this.sequence = sequence.replace(/\s+/g, '');

return this

};

Peptide.js

var Peptide = function(sequence, start){

this.sequence=sequence;

this.start= start;

return this;

};

Peptide.prototype.length = function(){

return this.sequence.length;

};

Trypsin.js

var Trypsin = function() {

this.rule = '(.+?[KR])(?!P)';

};

/**

* digest a protein sequence with the given regular expression and return the list of peptide with more than 4 letters

*/

Trypsin.prototype.cleave = function(prot) {

var re = new RegExp(this.rule, 'g');

var offset = 0;

var allPeptides = [];

while ( m = re.exec(prot.sequence)) {

allPeptides.push(new Peptide(m[0], offset));

offset += m[0].length;

}

return _.filter(allPeptides, function(p) {

return p.length() >= 5;

});

}

Dependencies

These file have some dependencies, when Tryspin instanciate new Peptide(), or to the external world, with link to underscore.js for example.

The classic setup would be put the *.js files int a js/ forlder and list them all, in the correct order, in the html <header>.

"classes", singleton text content are defined into a resource file. A define instance can depend on other resources, which will mean that they'll be loaded only when the upstream dependencies are ready and will export one entity (class, object text etc.).

a script can require some resources. Requirement can be chained of course.

Asynchronous download

Require.js offers a clean way to organize a JavaScript project. Even though it provides cache ready http header, the application can soon have hundreds of un-minified files to load.
Hopefully r.js comes to the rescue.
Let's create a new repository (also available within the example) deploy/ with the setup and build/ for the target/deploy/ lib/ r.js app.build.js
Install node, cd deploy/ and runnode lib/r.js -o app.build.js
Et voilà!, all JavaScript files are packed into one big uglified build/js/app.jsbuild/index.html refers to it and makes the minimal amount of queries:

The file app.build.js is quite similar to the app.js. Well, I personally break the DRY principle here and certainly deserve a few dozen push-ups. DRY extremists can find their way to more factorization here;)
There are much more options to r.js (uglify:'none' is a good place to start if the packaged application does not works). You can find them here.

As often with JavaScript, There Is Also More Than One Way To Do It. I present here one unit testing framework, because it works well with require and allow for headless continuous integration with phantomjs. Source code also presents a example_jasmine project.

Setup

in a test directory

test/

index.html

js/

Peptide-tests.js

Trypsin-tests.js

lib/jasmine-1.3.1

MIT.LICENSE

jasmine-html.js

jasmine.css

jasmine.js

index.html

Where jasmine is setup and the list of test suites is given. we use require js, with its definition embedded into the html page (once again DRY could reduce some copy/paste)

Test files

In our example, two test files are located in the test/suite directory.

Peptide-tests.js

OK, for the purpose of the demo, we check that the constructor works and that prettyString() return a meaningful string

define(['mypackage/Peptide'], function(Peptide) {

describe('Peptide', function() {

it("Constructor", function(){

expect(new Peptide('HAHAHA') instanceof Peptide).toBe(true);

})

});

describe('prettyString', function() {

expect(new Peptide('HAHAHA', 42).prettyString()).

toEqual('<b>HAHAHA</b> (42)')

});

});

Trypsin-tests.js

This case is more interesting. A cleavage enzyme cut a protein string after K or R, except if it is following by a P. Javascript regular expression does not allow look behind (only look ahead), so putting up the regular expression is a bit more cumbersome.

Therefore, we can setup a check function and call it with a list of testing sequences (and add more when we'll find bug...)

2 comments:

Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging. If anyone wants to become a Front end developer learn from Javascript Training in Chennai . or Javascript Training in Chennai. Nowadays JavaScript has tons of job opportunities on various vertical industry. ES6 Training in Chennai