Deno 1.0

2020-05-13

Ryan Dahl, Bert Belder, and Bartek Iwańczuk

Dynamic languages are useful tools. Scripting allows users to rapidly and
succinctly tie together complex systems and express ideas without worrying about
details like memory management or build systems. In recent years programming
languages like Rust and Go have made it much easier to produce sophisticated
native machine code; these projects are incredibly important developments in
computer infrastructure. However, we claim it is still important to have a
powerful scripting environment that can address a wide range of problem domains.

JavaScript is the most widely used dynamic language, operating on every device
with a web browser. Vast numbers of programmers are fluent in JavaScript and
much effort has been put into optimizing its execution. Through standards
organizations like ECMA International, the language has been carefully and
continuously improved. We believe JavaScript is the natural choice for dynamic
language tooling; whether in a browser environment or as standalone processes.

Our original undertaking in this area, Node.js, proved to be a very successful
software platform. People have found it useful for building web development
tooling, building standalone web servers, and for a myriad of other use-cases.
Node, however, was designed in 2009 when JavaScript was a much different
language. Out of necessity, Node had to invent concepts which were later taken
up by the standards organizations and added to the language differently. In the
presentation
Design Mistakes in Node, this is
discussed in more detail. Due to the large number of users that Node has, it is
difficult and slow to evolve the system.

With the changing JavaScript language, and new additions like TypeScript,
building Node projects can become an arduous endeavor, involving managing build
systems and other heavy handed tooling that takes away from the fun of dynamic
language scripting. Furthermore the mechanism for linking to external libraries
is fundamentally centralized through the NPM repository, which is not inline
with the ideals of the web.

We feel that the landscape of JavaScript and the surrounding software
infrastructure has changed enough that it was worthwhile to simplify. We seek a
fun and productive scripting environment that can be used for a wide range of
tasks.

Deno is a new runtime for executing JavaScript and TypeScript outside of the web
browser.

Deno attempts to provide a standalone tool for quickly scripting complex
functionality. Deno is (and always will be) a single executable file. Like a web
browser, it knows how to fetch external code. In Deno, a single file can define
arbitrarily complex behavior without any other tooling.

Here a complete HTTP server module is added as a dependency in a single line.
There are no additional configuration files, there is no install to do
beforehand, just deno run example.js.

Also like browsers, code is executed in a secure sandbox by default. Scripts
cannot access the hard drive, open network connections, or make any other
potentially malicious actions without permission. The browser provides APIs for
accessing cameras and microphones, but users must first give permission. Deno
provides analogous behaviour in the terminal. The above example will fail unless
the --allow-net command-line flag is provided.

Deno is careful to not deviate from standardized browser JavaScript APIs. Of
course, not every browser API is relevant for Deno, but where they are, Deno
does not deviate from the standard.

We want Deno to be applicable to a wide range of problem domains: from small
one-line scripts, to complex server-side business logic. As programs become more
complex, having some form of type checking becomes increasingly important.
TypeScript is an extension of the JavaScript language that allows users to
optionally provide type information.

Deno supports TypeScript without additional tooling. The runtime is designed
with TypeScript in mind. The deno types command provides type declarations for
everything provided by Deno. Deno's standard modules are all written in
TypeScript.

Node was designed before JavaScript had the concept of Promises or async/await.
Node's counterpart to promises was the EventEmitter, which important APIs are
based around, namely sockets and HTTP. Setting aside the ergonomic benefits of
async/await, the EventEmitter pattern has an issue with back-pressure. Take a
TCP socket, for example. The socket would emit "data" events when it received
incoming packets. These "data" callbacks would be emitted in an unconstrained
manner, flooding the process with events. Because Node continues to receive new
data events, the underlying TCP socket does not have proper back-pressure, the
remote sender has no idea the server is overloaded and continues to send data.
To mitigate this problem, a pause() method was added. This could solve the
problem, but it required extra code; and since the flooding issue only presents
itself when the process is very busy, many Node programs can be flooded with
data. The result is a system with bad tail latency.

In Deno, sockets are still asynchronous, but receiving new data requires users
to explicitly read(). No extra pause semantics are necessary to properly
structure a receiving socket. This is not unique to TCP sockets. The lowest
level binding layer to the system is fundamentally tied to promises - we call
these bindings "ops". All callbacks in Deno in some form or another arise from
promises.

Rust has its own promise-like abstraction, called Futures. Through the "op"
abstraction, Deno makes it easy to bind Rust future-based APIs into JavaScript
promises.

The primary component that we ship is the Deno command-line interface (CLI). The
CLI is the thing that is version 1.0 today. But Deno is not a monolithic
program, but designed as a collection of Rust crates to allow integration at
different layers.

The deno_core crate is a very bare bones
version of Deno. It does not have dependencies on TypeScript nor on
Tokio. It simply provides our Op and Resource
infrastructure. That is, it provides an organized way of binding Rust futures to
JavaScript promises. The CLI is of course built entirely on top of deno_core.

The rusty_v8 crate provides high quality
Rust bindings to V8's C++ API. The API tries to match the original C++ API as
closely as possible. It's a zero-cost binding - the objects that are exposed in
Rust are exactly the object you manipulate in C++. (Previous attempts at Rust V8
bindings forced the use of Persistent handles, for example.) The crate provides
binaries that are built in Github Actions CI, but it also allows users to
compile V8 from scratch and adjust its many build configurations. All of the V8
source code is distributed in the crate itself. Finally rusty_v8 attempts to be
a safe interface. It's not yet 100% safe, but we're getting close. Being able to
interact with a VM as complex as V8 in a safe way is quite amazing and has
allowed us to discover many difficult bugs in Deno itself.

We promise to maintain a stable API in Deno. Deno has a lot of interfaces and
components, so it's important to be transparent about what we mean by "stable".
The JavaScript APIs that we have invented to interact with the operating system
are all found inside the "Deno" namespace (e.g. Deno.open()). These have been
carefully examined and we will not be making backwards incompatible changes to
them.

All functionality which is not yet ready for stabilization has been hidden
behind the --unstable command-line flag. You can see the documentation for the
unstable interfaces
here.
In subsequent releases, some of these APIs will be stabilized as well.

In the global namespace you'll find all sorts of other objects (e.g.
setTimeout() and fetch()). We've tried very hard to keep these interfaces
identical to those in the browser; but we will issue corrections if we discover
inadvertent incompatibilities. The browser standards define these interfaces,
not us. Any corrections issued by us are bug fixes, not interface changes. If
there is an incompatibility with a browser standard API, that incompatibility
may be corrected before a major release.

Deno also has many Rust APIs, namely the deno_core and rusty_v8 crates. None of
these APIs are at 1.0 yet. We will continue to iterate on them.

It's important to understand that Deno is not a fork of Node - it's a completely
new implementation. Deno has been under development for just two years, while
Node has been under development for over a decade. Given the amount of interest
in Deno, we expect it to continue to evolve and mature.

For some applications Deno may be a good choice today, for others not yet. It
will depend on the requirements. We want to be transparent about these
limitations to help people make informed decisions when considering to use Deno.

Unfortunately, many users will find a frustrating lack of compatibility with
existing JavaScript tooling. Deno is not compatible, in general, with Node (NPM)
packages. There is a nascent compatibility layer being built at
https://deno.land/std/node/ but it is far from complete.

Although Deno has taken a hardline approach to simplifying the module system,
ultimately Deno and Node are pretty similar systems with similar goals. Over
time, we expect Deno to be able to run more and more Node programs
out-of-the-box.

Deno's HTTP server is implemented in TypeScript on top of native TCP sockets.
Node's HTTP server is written in C and exposed as high-level bindings to
JavaScript. We have resisted the urge to add native HTTP server bindings to
Deno, because we want to optimize the TCP socket layer, and more generally the
op interface.

Deno is a proper asynchronous server and 25k requests per second is quite enough
for most purposes. (If it's not, probably JavaScript is not the best choice.)
Furthermore, we expect Deno to generally exhibit better tail latency due to the
ubiquitous use of promises (discussed above). All that said, we do believe there
are more performance wins to be had in the system, and we hope to achieve that
in future releases.

Internally Deno uses Microsoft's TypeScript compiler to check types and produce
JavaScript. Compared to the time it takes V8 to parse JavaScript, it is very
slow. Early on in the project we had hoped that "V8 Snapshots" would provide
significant improvements here. Snapshots have certainly helped but it's still
unsatisfyingly slow. We certainly think there are improvements that can be done
here on top of the existing TypeScript compiler, but it's clear to us that
ultimately the type checking needs to be implemented in Rust. This will be a
massive undertaking and will not happen any time soon; but it would provide
order of magnitude performance improvements in a critical path experienced by
developers. TSC must be ported to Rust. If you're interested in collaborating on
this problem, please get in touch.

We have a nascent plugin system for extending the Deno runtime with custom ops.
However this interface is still under development and has been marked as
unstable. Therefore, accessing native systems beyond that which is provided by
Deno is difficult.

Many thanks to the
many contributors who
helped make this release possible. Especially:
@kitsonk who has had a massive hand in many parts
of the system, including (but not limited to) the TypeScript compiler host,
deno_typescript, deno bundle, deno install, deno types, streams implementation.
@kevinkassimo has contributed countless bug
fixes over the whole history of the project. Among his contributions are the
timer system, TTY integration, wasm support. Deno.makeTempFile, Deno.kill,
Deno.hostname, Deno.realPath, std/node's require, window.queueMircotask, and
REPL history. He also created the logo. @kt3k
implemented the continuous benchmark system (which has been instrumental in
almost every major refactor), signal handlers, the permissions API, and many
critical bug fixes. @nayeemrmn contributes bug
fixes in many parts of Deno, most notably he greatly improved the stack trace
and error reporting, and has been a forceful help towards the stabilizing the
APIs for 1.0. @justjavac has contributed many
small but critical fixes to align deno APIs with web standards and most famously
he wrote the VS Code deno plugin. @zekth has
contributed a lot of modules to std, among them the std/encoding/csv,
std/encoding/toml, std/http/cookies, as well as many other bug fixes.
@axetroy has helped with all things related to
prettier, contributed many bug fixes, and has maintained the VS Code plugin.
@afinch7 implemented the plugin system.
@keroxp implemented the websocket server and
provided many bug fixes. @cknight has provided a
lot of documentation and std/node polyfills.
@lucacasonato built almost the entire
deno.land website. @hashrock has done a lot of
amazing artwork, like the loading page on doc.deno.land and the lovely image at
the top of this page!