Topics

Featured in Development

Alex Bradbury gives an overview of the status and development of RISC-V as it relates to modern operating systems, highlighting major research strands, controversies, and opportunities to get involved.

Featured in Architecture & Design

Will Jones talks about how Habito, the leading digital mortgage broker, benefited from using Haskell, some of the wins and trade-offs that have brought it to where it is today and where it's going next. He also talks about why functional programming is beneficial for large projects, and how it helps especially with migrating the data store.

Featured in AI, ML & Data Engineering

Katharine Jarmul discusses research related to fair-and-private ML algorithms and privacy-preserving models, showing that caring about privacy can help ensure a better model overall and support ethics.

Featured in Culture & Methods

This personal experience report shows that political in-house games and bad corporate culture are not only annoying and a waste of time, but also harm a lot of initiatives for improvement. Whenever we become aware of the blame game, we should address it! DevOps wants to deliver high quality. The willingness to make things better - products, processes, collaboration, and more - is vital.

Featured in DevOps

Service mesh architectures enable a control and observability loop. At the moment, service mesh implementations vary in regard to API and technology, and this shows no signs of slowing down. Building on top of volatile APIs can be hazardous. Here we suggest to use a simplified, workflow-friendly API to shield organization platform code from specific service-mesh implementation details.

Nada Amin on Scala DSLs with Lightweight Modular Staging and Compiling to JavaScript

Bio

Nada Amin is a PhD student at EPFL, in the programming methods laboratory (LAMP) led by Martin Odersky. Previously, she was a software engineer at Google, in the Gmail and Closure compiler teams. She holds bachelor and master degrees in Computer Science from the Massachusetts Institute of Technology.

About the conference

Scala Days is the premier event for Scala enthusiasts, researchers, and practitioners. Scala is a general-purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way. It smoothly integrates features of object-oriented and functional programming.

I’m first year PhD student in Martin Odersky's lab at EPFL, previously I worked for two years as a software engineer at Google, and before that I got my bachelor and my master [degrees] in the US and I’m from Switzerland and Bahrain.

Lightweight Modular Staging is a technique for embedding DSLs in Scala purely as libraries, so that’s why it’s Lightweight and it’s modular because you specify the construct of your DSL in Scala traits in a modular fashion, you just mix in a trait to have a feature, and it’s also modular because the implementation is completely separate from the specification of the DSL. The idea of staging is that the program is split into two parts, and the first part is a code generator which when executed uses the second part, the program, so in our case the DSL would be the program generator and the second stage program would be the generated JavaScript code. So this staging feature actually allows flexibility and abstractions only exist in the DSLs snippets and are not part of the generated code. In our DSL for JavaScript we actually use this feature to implement a nice abstraction over asynchronous programming that avoids callback hell, and we do this by using features that are available in Scala like the CPS plugin, continuation passing style plugin, to implement this and this all gets evaluated away at staging time, which means that the generated code is not complicated by these abstractions, so that is the basic idea of staging and lightweight modular staging.

Exactly, at a first approximation the DSL is just a thin layer on top of JavaScript so the idea is that you can really take snippets of JavaScript and with very simple changes just run it in the DSL, and then we added type safety where you have the usual Scala type system in place that just works for the DSL, and we also allow a gradual shift for more dynamic features to more type safe features using the new Scala feature, Dynamic. So the basic idea is that you write these snippets of DSL and then you can generate JavaScript code from them or you can run it as Scala code.

So one big use case is sharing code between the clients and the server and that is very useful for validation logic for example, so in that case often you need to have the code on the client for interactivity and on the server for security. And this duplicated logic needs to maintained and it’s always bad to have to duplicate things, so what we do here is solve the impedance mismatch between having server and client code differ. So we’ve actually integrated our DSL with the Play framework and what we do there is that Play has already a language to specify validation logic so we’ve extended this with the ability to specify constraints in our DSL and the whole validation logic is checked on both the client and the server.

The idea there is to actually use staging to be able to use the continuation plugin to reify the continuation for us so that we can then basically pause and resume execution of things. So the way this works is that in JavaScript you have to write a lot of code using callbacks, and we twist this around by just saying: “Ok, these callbacks will be generated by the CPS plugin”, and you can write your code in a more linear fashion. Even a very simple example where you just want to print, just want to do something in a loop, and in the end if you have callbacks involved it becomes very convoluted, but here you can just write your for loop as if it was a linear operation and it all works magically, and the nice thing is that because of staging all of this gets abstracted away in the generated code, and of course you have a lot of functions in the code, but you don’t have any of the abstractions from the continuation plugin.

Yes, it’s actually from this, for example, we can also use the AJAX API to do, you can just write a for loop and do AJAX calls in parallel for example, so that is an interesting use case which we’ve done a little application which goes and fetches Twitter feeds in parallel and it’s very simple to write because you avoid this callback hell style programming.

At this moment we don’t , the framework, the Lightweight Modular Staging Framework allows us to do further optimizations. We have not explored that too much, we do some simple optimizations to be able to hook it up with the Clojure compiler, we would need to also propagate the type information, but since we start with Scala, this is feasible, we have that information, so this could be a further work to improve the generated code.

The idea there was that we do the data flow analysis to allow more precise inference of types within a method, and then we have type checking which actually checks that when you call a method you have the right parameters and the return type is correct and things like this.

I think some of the issues are with the semantics of JavaScript. One issue we had with compiling from Scala was the semantics of this, we actually also have some lightweight structure for reifying classes and simple object literals, and for classes and inheritance, in JavaScript this is very peculiar, you can dynamically bind it, if you have multiple function calls, you can “lose” it if you want by just having a bare function call where you don’t use this.method() , so this is a bit complicated when you are compiling to JavaScript as a target, specifically when you are trying to also have class based inheritance. I guess another thing is that there's lots of quirks with the browser incompatibilities and we actually have kind of glossed over this, we have not thoroughly checked if our code is running in all browsers, so that why it’s still a research project in that sense.

We also gloss over this in the sense that we just translate to JavaScript numbers without regarding the issues, like the particularities of, eg. the addition operation mixed with strings and all that, this is probably not preserved exactly as it is in the translation, so there are a lot of quirks, I guess. JavaScript was not meant to be a target of compilation, but now it’s something that is increasingly happening.

I actually like the way Scala is right now, one thing is that the type system seems pretty complicated from the outside, there are a lot of features, a lot of options and one thing that we are trying to do is actually to simplify it, to make it more uniform in the next version of Scala, so this is something that I’m exploring right now with Martin Odersky, the type system foundation of Scala.

As a Monad? Well ok, so I guess I’m not really sure what means but I would be a free Monad, I saw that in one of the talk titles and I guess is just appeals to me to be able to just, yes, freely follow my interests.

Every scalar is a Monad

Suggestion

Your message is awaiting moderation. Thank you for participating in the discussion.

I have a suggestion for these interviews. Let the interviewee drive the conversation and describe their work freely (preferably with a demonstration). I think that will result in a higher quality compared to an interviewer driving the conversation who is obviously not an expert on the topic that the interviewee is an expert on.

async.js

Your message is awaiting moderation. Thank you for participating in the discussion.

writing callbacks doesn't have to be hard if you use functional patterns (monads) to deal with it like in async.jsdon't need to do staging shit or TAMEJS stuff so long as functions are first class.also, people using for loops in javascript, with a scala background, should really be asking themselves 'wtf am i doing?'JS is a functional language, scala is too, the code shouldn't look much different, minus the pattern matching/actors. callback hell is a solved problem long ago.

b) The interviewer (that's me BTW) is not an expert on the topic, which is good because most readers won't be experts on the topic either. I ask for explanations so I can understand the topic and, in the process, the audience can follow along.

Re: Suggestion

Your message is awaiting moderation. Thank you for participating in the discussion.

It's not a problem that you're not an expert, but a non expert cannot drive the conversation in a logical fashion like an expert can. In this interview I only understood what the project is actually about near the end. For example your first question is: "So you are talking here about an interesting topic of compiling JavaScript and the Scala feature called Lightweight Modular Staging." This put me on the wrong path: it's actually NOT about compiling Javascript, it's about compiling a scala dsl TO javascript (but I still had no idea what lightweight modular staging actually is). The interview is not bad if you already know what everything is about (for example if you've watched the talk you linked to), but otherwise it doesn't make a lot of sense. Either you should do an introductory interview (perhaps first have an off-line conversation with the interviewee about the questions you'll ask to introduce the topic) or you should put the link to the talk in the description so that people can watch it before they watch the interview. /2c