As much fun as coding can be, I prefer to write code that generates code.

My most recent effort is gostd2joker, which collects information on Go’s standard library (the packages provided in its source tree) and generates the Go and Clojure code to provide access to some of those APIs within Joker, a Clojure interpreter that is written in Go. While there remain substantial limitations in terms of which APIs are eligible for transformation, the resulting list of converted packages is noticeably larger than that for “vanilla” Joker.

Having successfully deployed a few Joker scripts on my email server, mainly to test the waters (but also to replace fairly ugly Bash scripts with more-elegant Clojure code), I’m looking to further my use of Clojure. I’d like to achieve “Clojure Everywhere”, in which all my full-stack code (UI, backend, and quick-running scripts) is written in Clojure on top of various hosts (JVM, JavaScript, and the Go runtime).

Much work remains to be done, mainly to instill, in Joker, a proper concept of the Go runtime as a “host” to parallel Clojure’s JVM and ClojureScript’s JavaScript engines, thus introducing Go types (native and those provided via packages), field and method access to them, instantiation of instances of them, and so on.

The result should be quite useful, not only in my own research and development, but potentially to others who might find a quick-starting, low-overhead scripting language, with full access to the Go runtime, to be a formidable tool in their scripting arsenal.

I’ve started deleting users who haven’t commented on, or posted, anything, to get rid of the many likely-spam/bot users.

The bulk-delete tool I’m using doesn’t let me filter on whether a user has a profile image, and it appears to have already deleted at least two legit-looking subscribers. So I’m holding off deleting more for now.

Please login and comment/post on occasion, at least for awhile, as I clear out inactive users. (I’ve added a plugin to reduce new-user registrations from known spammers as well.)

How can organizations that depend on free-software components ensure that they are maintained and available, which requires (first and foremost) funding component developers? This article explains some different approaches to that problem.

Over the past several weeks, I turned my attention to assessing the performance of various Lisp implementations, focusing on suitability for short scripts, launched as processes, very frequently as is consistent with the traditional Unix model and exploited by messaging systems such as qmail.

Here are some interesting results on the use of Go, versus C, as an implementation language for a Lisp interpreter (Clojure in particular, though that’s not involved in these results):

tl;dr: Go is. Really, really fast, at least on this simple code and its few tests. It’s certainly not enough slower than C to justify implementing a Lisp interpreter in C, especially when there’s already a decent, albeit young, Clojure interpreter written in Go.

There’s plenty to be said (and has already been said) about Go versus C. I found debugging Go code to be challenging, as the use of GDB was discouraged, and Delve does not meet my requirements for easily debugging such low-level code. But the packaging system (the go tool) is fairly well-thought-out, easy to come up to speed on, and surprisingly free of warts.

Why is “Low Latency” Important at an Architectural Level?https://burleyarch.com/wp-content/themes/engic.new/images/empty/thumbnail.jpg150150craigcraighttps://secure.gravatar.com/avatar/898daee4950ac9b8f8fb211de1d9e645?s=96&d=mm&r=pg2018-02-262018-02-27

Of the four major performance characteristics, or dimensions — processor (CPU) speed, memory (storage), bandwidth, and latency — latency has seen only modest improvements over the past several decades of modern computing, while the other three have experienced growth substantially described by Moore’s Law.

That is, the speed, storage, and bandwidth offered by computing (information) systems have improved substantially year over year for many years. Meanwhile, the limiting factor for latency — the speed of light — has experienced no improvement whatsoever, nor is there any such improvement (increase in the speed of light or an alternative means of communication not limited by it) anticipated in the near future, short of a stunning breakthrough in physics. Most improvements in latency have been in the form of modest percentages “on the edges” (of electronic or light-based transmission) as pertinent circuitry has leveraged the benefits in those other three dimensions of performance.

As a “green-field” starting point for information-systems architecture, today’s state of the art would suggest that systems be optimized to minimize latency, first and foremost, to the extent those systems encompass large-scale networks, even if at the expense of increased demands for processing speed, storage, and bandwidth.

A system that requires N round trips to perform an action can therefore expect performance no better than N*300ms in the worst case documented on that page, N*8ms in the best case.

Such “round trips” include not only overt actions performed by the application, but actions performed at lower levels in the stack in order to carry out those high-level actions. E.g. setting up an outbound TCP connection takes several round trips in and of itself.

Optimizing various endpoints, or across individual stack levels, cannot improve beyond this minimum threshold, unless the system itself is optimized (essentially, re-architected) in some fashion.

While keeping latencies low is well-understood at various levels, or slices, of the computing stack (see Understanding Bandwidth and Latency for just one example), these efforts can fall well short of the full potential of optimizing for low latency as it pertains throughout the stack, including when done via Architecture.

Here, viewing Architecture‘s goal as “making fundamental structural choices which are costly to change once implemented”, the protocols used to implement the communications must necessarily be optimized, via their selection and/or design, for latency — as must any other aspects of the system where performance is a consideration. (Note that optimizing the design of a protocol is distinct from optimizing its implementation, similarly to how choosing an algorithm, based on its performance characteristics, is distinct from optimizing the implementation of that algorithm.)

Thinking back in time prior to today’s theoretical “green-field” starting point, it becomes easier to understand why and how network protocols (including TCP, SMTP, HTTP, and so on) were designed to meet performance requirements that did not prioritize latency above all. In particular, architects and designers had to cope with systems and networks of comparatively limited processing power, storage, and bandwidth — but seemingly “instantaneous” communication (at near the speed of light). Naturally, they chose to ensure their protocols would be realizable by computing machinery with power that was severely limited compared to today’s.

Scaling up systems built on top of those legacy protocols, to support an increasingly widespread online population — of both man and machine — is proving to be increasingly challenging, for reasons of latency at least, nevermind the ongoing challenges presented by sheer complexity, security issues, and so on.

That’s why it’s important for new designs to start out architected to avoid high-latency pitfalls, perhaps even to the extent of avoiding those legacy protocols in favor of lower-latency alternatives (such as UDP and datagrams rather than TCP).

It’s also worth considering whether a “21st-Century” Internet, architected for low latency, is worth pursuing as a “Beyond IPv6” offering.

What might a protocol that optimizes for latency look like? I’m working on a Proof of Concept, and hope to have something to show for it in the near future.

Please let me know what experiences and thoughts you have on optimizing for latency!