Software Engineering and Data Visualization

I’ve recently started working back through Jon Bentley’s Programming Pearls [1]. I first bought this book back in 2013, when I was first starting to move back to professional software development after over a decade’s hiatus. It came highly recommended by a number of experienced programmers, and even then I could appreciate that this was a very valuable book. That said, I understand now that I lacked the programming maturity to get much out of it. Back when I first read the book, I had forgotten many of those data structures and algorithms I had learned back 15-20 years ago when I first started programming, which to be frank was not many. And I struggled to get much out of Programming Pearls. Despite my many years of hobby-level programming, I simply lacked the programming maturity to appreciate a lot of what he was discussing.

Since that time, I’ve done a methodical review of the most common data structures and algorithms using Sedgewick and Wayne’s Algorithms, among other textbooks and online resources, such as Khan Academy and Hackerrank [2,3,4]. And it’s amazing how much things are now “clicking” as I read Programming Pearls. It’s a great resource for someone with a modest amount of professional programming experience and a basic knowledge of data structures and algorithms. Bentley covers a wide range of topics relevant to the working programmer, including multiple chapters with case studies of algorithmic development from Bentley’s life as a professional developer, a chapter on the importance of choosing the right data structure(s) for your program, program performance/tuning, and finally a number of chapters that discuss specific algorithms and data structures.

So, I’ve recently started work on React-d3, a chart library for Facebook’s React that leverages the power of Mike Bostock’s awesome d3 data visualization library. The project is in its very early stages (as in, I just started it two weeks ago), but it’s presently up on Github as well as NPM.

Background

As I describe in the project’s README, the approach I’m using here was inspired by this blog post by Ben Smith of Binary Consulting. What’s interesting about Ben’s approach is that it uses React – not d3 – to actually generate a given’s chart’s svg. Most other examples of React and d3 I’ve found online have only used React components to abstract out the d3 chart code; in these examples, d3 remains solely responsible for generating the needed svg markup.

Furthermore, the charts in Ben’s examples are mostly stateless, which means they’re in a position to take advantage of the optimizations React offers to stateless components.

So if we’re aiming to use React to actually generate the charts, why use d3 at all? Well, the answer to that can be found in d3’s API reference, specifically in that initial list of d3’s main “libraries”. Of those 8 libraries (Core, Scales, SVG, Time, Layouts, Geography, Geometry, Behaviors), only 1 – the SVG library – is actually used to generate chart markup. The rest are support libraries, with utilities for converting and parsing data, calculating positions, projections, and geometries, and responding to user events.

So basically, I’m using d3 as a utility library, albeit an extremely expansive one that would take man-years to re-create.

Naturally, as with any real-life project, I soon had to deviate from my initial plans of using only React to generate the chart svg. It turns out that generating chart axes is a pretty complex task, so for now I’m using d3 to generate chart axes until I’ve have the time to develop a React chart axis component that is general enough to accomodate all the use cases that I have in mind.

I’ve been playing around recently with cellular automata, particularly a type of automata known as elementary cellular automata. These are the extremely simple computational models that so captures the imagination of Steven Wolfram in his book A New Kind of Science [1].

At its most basic, each cellular automaton can be described as a simple binary digit that lives together with two neighbors, also binary digits. Each automaton can give rise to a new automaton, the value of which depends on the current state of its “parent” automaton in combination with the current state of its parent’s two neighbors to the right and left.

Updated for d3.v4!

I’ve noticed that tutorials to GUI toolkits often build an analog clock as a way to teach the fundamentals of the toolkit. And because a basic analogue clock is nothing but three data fields bound to an object, it occurred to me that an analog clock might be a good way to teach d3. So I put together the following tutorial. This tutorial assumes that the reader has a good understanding of basic JavaScript, and has had at least some exposure to d3.

As I wrote previously, Kivy is an amazing framework for cross-platform mobile development with Python. Over the course of a few years, a small group of dedicated developers have build an open-source GUI framework that can be used across almost every common platform: Windows, OSX, Linux, Android, and iOS. Not only is it possible to release Kivy apps in the form of ‘native’ packages on all of these operating systems, but with modern hardware, it’s also possible – simple, even – to create Kivy apps that run as smoothly as apps build using native app frameworks. When you consider all these features in combinations, Kivy is clearly a winning platform.

I’ve recently been attempting to create an educational visualization of the k-means algorithm. It seemed like the kind of algorithm that would lend itself well to a visualization. Unfortunately, it’s not been so easy. Have a look below the fold to see what I’ve come up with so far.