Ian Hague Perl 6 Grant Application: JavaScript backend for Rakudo

We have received the following Perl 6 Ian Hague Grant Application. Before we vote on this proposal we would like to have a period of community consultation for 10 days. Please leave feedback in the comments or if you prefer send email with your comments to karen at perlfoundation.org.
**Name:** Paweł Murias
**Project Title:** JavaScript backend for Rakudo
**Synopsis:**
Improve the JavaScript backend from handling NQP (Not Quite Perl) to full Perl 6.
**Benefits to to Perl 6 Development:**
A JavaScript backend for Rakudo will allow the use of Perl 6 in many new niches. The main focus of the grant is to allow Perl 6 to be used for writing the frontend part of single page applications (for the backend part we can use MoarVM).
A side benefit of the grant is that I intend to create a web-based REPL that should allow users to play around with Perl 6 without installing it. The goal of the grant is provide a JavaScript backend with enough features that the community can start experimenting with what running inside the browser will allow us to accomplish.
While working on the JavaScript backend I write test for the things I'm implementing. Expanding the test suite will directly help future backend authors. It also helps anyone doing non trivial changes to the existing backends. I have found MoarVM and JVM bugs in the past doing that.
**Deliverables:**
- Upload rakudo-js to npm and CPAN.
- Have this rakudo-js be able compile our chosen subset of the 6.c roast (official Perl 6 test suit) to JavaScript and pass them in a modern browser.
- Write a simple REPL in Perl 6 that will run in a modern browser.
- Write a tutorial showing how to use the JavaScript backend.
**Project Details:**
Rakudo compiles Perl 6 and NQP (a subset of Perl 6 that Rakudo itself is written in) to an abstract syntax tree form called QAST. QAST is then passed to either the MoarVM, JVM or JavaScript backends. Currently the JavaScript backend can only handle AST that is produced from NQP. The goal of this project is to improve the JavaScript backend to handle the QAST produced from full Perl 6.
I started the original work on the JavaScript backend while Rakudo was transitioning from being a Parrot targeting compiler to a multi-platform one. Parts of the work on the backend was done as parts of a GSoC project. After the GSoC projects I undertook a rewrite of the backend. The rewrite allowed me to add source maps support and use more type information to generate better code. The JavaScript backend is now merged in the the master branch of the NQP repo.
After reviewing the initial draft of this grant proposal Jonathan Worthington pointed out that implementing gather/take proved to be tricky on other backends. To reduce this risk I added basic continuations support to the backend. This was enough to run a basic form of gather/take: [https://github.com/perl6/nqp/blob/master/t/js/continuations.t#L76](https://github.com/perl6/nqp/blob/master/t/js/continuations.t#L76) . I implemented this using a CPS transform with a trampoline (to work around the lack of tail call optimization).
Most of Perl 6 is built from smaller building blocks. This will mean that a large part of the effort will be needed before I get to the point where the test module compiles and the first test passes. On the other hand when the needed building blocks are implemented correctly I will be able to reuse the quality work that went into Rakudo and the setting.
**Inch-stones:**
- Cleanup the array handling in nqp-js.
- Finish up serialization of closures in the nqp-js-running-on-js.
- Go through the MoarVM opcode list and where it's possible write tests for untested opcode and implement them in nqp-js
- Do the obvious speedups for the code generated by nqp-js.
- Compile the meta-model and bootstrap support with nqp-js.
- Implement a bunch of p6 specific ops.
- Get rakudo to compile on nqp-js.
- Get rakudo compiled to js to correctly compile nqp::say("Hello World").
- Get the rakudo setting to compile.
- Get rakudo compiled to js to correctly compile say("Hello World").
- Get Test.pm to correctly compile.
- Pass a first test.
- Go through roast test fixing bugs and implementing missing things to make them pass.
- Get continuations support fully functional. Get nqp-js to pass test all test in full CPS mode.
- Pass the part of roast we focus on node.js.
- Be able to webpack the generated javascript code.
- Be able to run tests in the browser.
- Pass the part of roast we focus on in a browser.
- Polish up source maps support.
- Implement (and test) interoperability with javascript code
- Upload rakudo-js on npm.
- Write a simple Perl6 REPL that should run in modern browser.
- Put the REPL on try.perl6.org when the community views it as good enough.
- Write a tutorial that describes how to use the backend.
- Make sure our source map support works correctly and integrates well with browsers.
- Fix issues that early adopters will encounter.
- Fix the most obvious performance issues.
**Completeness Criteria:**
- Rakudo-js released on npm
- Rakudo-js passes our chosen subset of roast
- A simple Perl 6 REPL running in a modern browser (for evaluation purposes a modern version of Google Chrome)
**List of tests we want to pass:**
We want to pass tests from the official Perl 6 Test Suit "Roast". It can be found on github.com/perl6/roast.
For the purpose of the grant we focus on a subset of those (mainly excluding IO and OS interaction). Some tests might be broken for reasons outside of the JavaScript backend so that for grant completeness purposes we are only concerned with the tests that pass on the Rakudo MoarVM backend.
We want to pass the tests in the following subdirectories of roast:
- S02-lexical-conventions
- S02-lists
- S02-literals
- S02-magicals
- S02-names
- S02-names-vars
- S02-one-pass-parsing
- S02-packages
- S02-types
- S03-binding
- S03-feeds
- S03-junctions
- S03-metaops
- S03-operators
- S03-sequence
- S03-smartmatch
- S04-blocks-and-statements
- S04-declarations
- S04-exception-handlers
- S04-exceptions
- S04-phasers
- S04-statement-modifiers
- S04-statement-parsing
- S04-statements
- S05-capture
- S05-grammar
- S05-interpolation
- S05-mass
- S05-match
- S05-metachars
- S05-metasyntax
- S05-modifier
- S05-nonstrings
- S05-substitution
- S05-syntactic-categories
- S05-transliteration
- S06-advanced
- S06-currying
- S06-macros
- S06-multi
- S06-operator-overloading
- S06-other
- S06-routine-modifiers
- S06-signature
- S06-traits
- S07-iterators
- S09-autovivification
- S09-hashes
- S09-subscript
- S09-typed-arrays
- S10-packages
- S11-modules
- S12-attributes
- S12-class
- S12-construction
- S12-enums
- S12-introspection
- S12-meta
- S12-methods
- S12-subset
- S12-traits
- S13-overloading
- S13-syntax
- S13-type-casting
- S14-roles
- S14-traits
- S32-array
- S32-basics
- S32-container
- S32-exceptions
- S32-hash
- S32-list
- S32-num
- S32-scalar
- S32-str
- S32-temporal
- S32-trig
**Project Schedule:**
The project is expected to take 4 months of full time effort. I will begin work as soon as the grant gets accepted. Based on the progress of previous backends most of the effort will be needed to get to the point where all the basic building blocks are working and we start passing tests.
The plan for the months of the grant is:
- Getting to the point where can begin compiling the CORE setting.
- Correctly compile and load the setting.
- Fixing inevitable bugs that cause failing tests and implementing missing bits of functionality. After this steps we should be passing the roast test suit.
- Working on making the backend easy to install and use. This will include writing a tutorial, fixing issues that early users find, tweaking source maps, improving obvious performance problems.
**Report Schedule:**
I will report on the progress of the grant on a blog at least every two weeks, preferably more often. I will also keep the #perl6 channel updated on my progress.
**Public Repository:**
The backend code will be hosted at github.com/perl6/nqp. Any required modifications to rakudo will be hosted at github.com/perl6/rakudo. (Work on rakudo itself will intially be done in either a branch or a github fork).
**Grant Deliverables ownership/copyright and License Information:**
All the work produced as a result of this grant will be licensed under the Artistic License Version 2.0. I will send in the CLA and if required transfer the copyright to The Perl Foundation.
**Things not addressed by the scope of the grant:**
Performance and size of the generated JavaScript code will likely be an important concern before using the backend for serious production use. While I'll to keep it in mind and attempt to solve the most obvious problems as it's a very open ended issue it's not the main focus of the grant. As the feedback from the #perl6 channel seemed to imply webapps are the thing the community is the most interested in, I'm moving IO support while running on node.js outside of the scope of grant.
**Amount Requested:** $10000.
**Bio:**
Worked on the mildew/smop/kp6 Perl 6 implementations. Once the Perl 6 implementations converged on Rakudo I started working on the JavaScript backend for it, starting first with one for NQP. I worked on the JavaScript backend for NQP outside of and as part of a Google Summer of Code project. I have tweaked both the MoarVM and JVM backends so if changes to the whole of NQP are required I will be capable of that.

+1 I think that would be a great boost to the language and I am eager to reimplement my frontends with perl6.

[ 1682 ] |
Tue, 09-Feb-2016 by geoffreybroadwell

+1: pmurias has already been working on the JS backend for NQP for quite some time, and has *greatly* expanded the NQP test suite as he did so. He's got a proven track record and is by far the best choice for this grant -- and having full Rakudo working on both node.js and Chrome would be a huge win.

[ 1683 ] |
Tue, 09-Feb-2016 by wendyvandijk

Perl 6 needs this (and more like this) to show the world how awesome it is (it already is, and this will make it even more awesome). So, please:
+1

[ 1684 ] |
Tue, 09-Feb-2016 by brockwilcox

+1: I am so psyched about this that I will make an additional (small :) ) donation to TPF if it goes forward! This will open us up to some very interesting experiments.

[ 1685 ] |
Thu, 11-Feb-2016 by zoffixznet

I think node's strength is the ability for frontend devs to code backends, without a need to learn a new language.

This project will create a niche for the Perl 6 language to become one that can be used in the backend as well as the frontend, possibly increasing its adoption.

One point of concern, however, is performance. There are plenty of human-written JavaScript web apps that slow the browser to a crawl. Working off the assumption that compiler-generated JavaScript won't as be performant as human-written one, I have a worry this project will never see wide adoption due to dismal performance.

The monetary amount requested, however, is very small for the amount of work to be done, so my vote on this is +0.4

[ 1686 ] |
Sun, 14-Feb-2016 by pawełmurias

I'm unable to predict with a lot of certainty how fast the performance of Perl 6 compiled to JavaScript will be after the completion of this grant.

The current JavaScript implementations are fairly complex JITs so it's really hard to predict how they will handle the way some of the more tricky Perl 6 features will be compiled.

What I hope to achieve with the grant is to allow the community to explore what having Perl 6 in the browser (and other places where JavaScript is barging in) will allow us too do. For a lot of the more demanding uses no doubt a ton of work will still be required.

In the long term the performance of the code produced by dart2js (which according to some benchmarks seems to approach native JavaScript) gives us hope that given enough effort we might be able to achieve really good performance.
To be fair Dart is not Perl 6 but it still has significantly different semantics from JavaScript.
A lot of the trick dart2js seem potentially very helpful to us.