Harfbuzz Adventures

Lately, I've been thinking about text layout in Rust. We'll get back
to that at some point. This led to me to looking at the Rust bindings
for Harfbuzz. Since then, I've started some submitting some fixes to
the Rust bindings, including working towards updating to the current
version of Harfbuzz. (This was a couple of weeks ago. Then I went on an
extended holiday.)

Then, I decided to look at making some improvements upstream in Harfbuzz.
I've now had a few minor patch submissions (#669, #670) accepted
there as well and am working on more, including improving the support
for the cmake build system.

One annoying thing is that since I use ninja as a build tool (instead
of make), I have to modify many projects to include this bit of
configuration (#674):

Query responses are decided by the client rather than the
server. A GraphQL query returns exactly what a client asks
for and no more.

COMPOSITIONAL

A GraphQL query itself is a hierarchical set of fields. The
query is shaped just like the data it returns. It is a natural
way for product engineers to describe data requirements.

STRONG-TYPED

A GraphQL query can be ensured to be valid within a GraphQL
type system at development time allowing the server to make
guarantees about the response. This makes it easier to build
high-quality client tools.

The GraphQL website has a lot of additional information, documentation,
and some initial implementations.

I'm going to assume that you're at least roughly familiar with it, but
as a quick example, this query:

I'm building something for building developer tools. That's pretty
vague, but I'd like to keep it that way for now. To do this, I
wanted to use an existing user interface library. Unfortunately,
it seems like it is always impossible to find what I want. I apparently
want too much.

My Requirements

So, what am I looking for? These are just my own personal desires and
I certainly don't expect that everyone will agree with them.

React

I'm planning to use React in my rendering layer, so it would be nice if
the user interface framework that I use has solid support for React.
This seems like a no-brainer. That said, there is another layer in my
architecture which describes user interfaces and that is what gets
lowered or converted to the actual widget definitions. In theory, this
means that multiple view layers could exist (perhaps even using something
like React Native). But for the start, I need solid React support.

TypeScript

Much like my desire for React support, I am writing my code in TypeScript.
That doesn't mean that the UI has to be in TypeScript, but it would
make it more convenient. Otherwise, typing definition files (.d.ts …

For a project that I'm working on, I would like to be able to retrieve
information from the OS about a variety of things:

File system

Current processes running

Process memory maps

Open file handles in a process

File attributes

File type, mime type

... and a lot more!

Some of these are very easy and are just a set of calls to POSIX functions.
Others are highly dependent on the underlying platform.

I want to be able to access this data from an application that is either
running natively or from within node.js when using Electron. I want it on
a minimum of Mac OS X, Linux, FreeBSD and Windows. Support for other
platforms such as NetBSD would be nice.

I don't want this to be very large. The overall code required is likely to
be fairly small. Hopefully, a large number of dependencies would not be
brought in by using a library that provides this information.

I want the data in JSON-LD format. This is JSON, but with some additional
fields, like @type, that help my application present the data correctly.
It would be nice if there were a way to subset the data such that some of …

JavaScript is commonly seen as a flawed language. It has plenty
of flaws, some of which are more painful than others. One common
way that people deal with these flaws is to use another language
that compiles to JavaScript and to then write their code in that
other language.

At first, this sounds pretty appealing. You get to choose which
of a few languages to use to address your complaints with JavaScript.
Unhappy with the type system? Try out Dart, TypeScript, AtScript, Elm,
or even PureScript. Want a more concise syntax? Perhaps CoffeeScript
is your thing. Do you wish you were using a Lisp? Well, there are lots of
options there as well, starting with ClojureScript or Parenscript.
There are many choices.

My Project

My project involves creating a platform for building tools that is
based on Atom Shell and uses a plug-in model to allow users to
extend the core platform.

I also have some requirements:

I need access to things that will be coming in future versions
of JavaScript (and fairly soon). An example of this is support
for working with 64 bit integers (a firm requirement).

For the project that I am working on (not yet disclosed), I was
thinking that it would be interesting to be able to integrate a
view of what is happening on the network, much like Wireshark
and other tools can provide. The view would be more targeted
towards what the user was doing, but the overall idea would be
the same: capture network traffic and perform some basic analysis
on it to display it visually.

This led to some interesting research, which I've decided to discuss
here! (For those who feel this is a long post, there's a summary
of sorts at the end.)

Capturing Traffic

node_pcap

Given that I'm using Atom Shell to build this application and that
uses Node.JS, one of my first thoughts was to take a look at
node_pcap. After all, node_pcap was used successfully in
tools like htracr, so perhaps it would work well here.

Unfortunately, I ran into a series of issues with node_pcap. For
one thing, it wants to run the capture in the same process, and since
performing a capture requires putting the network interface into
promiscuous mode, it requires elevated privileges. I am not comfortable
with the idea that my …

When building an application using the Atom Shell, you may want to
take advantage of newer Javascript features from ES6 (Harmony) like
arrow functions, generators, for ... of loops and many other things
which are not enabled in the V8 Javascript engine by default.

When starting up your Atom Shell application, you can append switches
to the command line that will be given to the embedded Chromium browser
engine.

Note that if you have an incorrect flag being passed, the other flags
that you set may be ignored, so keep an eye on this as you update to
newer versions of Atom Shell as the valid flags may have changed.

Unfortunately, it can be confusing figuring out which version of V8
is being used and what ES6 features are available. As of the time
of this writing, Atom Shell 0.18.1 is using a Chromium 38 and so
it falls in between the Chromium 37 and 39 columns on this
compatibility chart. (But don't look at the Node Harmony column
for figuring out what Atom Shell supports!)

Many applications benefit from a mix of a graphical and textual interface.
This has ranged from entire environments, like the Lisp Machine and the
follow-on Common Lisp Interface Manager (CLIM), to specific applications
like Mathematica and others which employ a "notebook interface". There
have also been various attempts at producing richer terminal applications
which can work with either standard or specialized applications to produce
richer, more interactive output. Graphically enhanced textual interfaces
have also found their place in various genres of games over the years.

In our case, we're interested in enhancing the interaction between a
user and a specific tool. We'll take a look here at some of the ideas
that have come before to get an idea of what people have already done
to inform what we do for ourselves.

Some of the things discussed here are old, some are new. Some are long
lost and forgotten (one link below is to archive.org) while others are
actively being worked on today.

Common Lisp Interface Manager

Like many things, early research into graphically enhanced and interactive
textual interfaces was carried out within the Lisp world. An early paper
Presentation Based User Interfaces by Eugene C. Ciccarelli IV at …

I've been doing some web development off and on for many years (since 1996).
I started out in an interpreted language on top of an object database that
was serving up a web-based discussion forum that I was writing and shipping.
I moved on from there to various other tools over the years including various
home-grown templating languages, Django templates, Jinja2 and others. I
helped produce web applications using ExtJS, others using jQuery / jQuery UI,
and experimented with various other frameworks over the years. I've followed
along with what people are doing in ReactJS and related tools like Om.
It is all interesting.

A Recent Project

I recently built a web-based UI for a memory / heap profiler for emscripten.
This is just some Jinja2 templates and Flask on the server side. The UI is
using Bootstrap and went through several iterations of various JS tools and
frameworks for handling graphs, grids, and charts. This was something of
a nightmare (and honestly, the code still is terrible). Many tools didn't
play well together. Some had issues due to CSS or JS clashing. Others just
wanted to control more than I wanted to let them control. Getting jQuery UI,
Bootstrap, Slick Grid …