Saturday, October 24

userlands, 1 of ?

I want to start sketching out this idea I keep having before I lose it again to
distraction.

I wrote userland while experiencing, I now think, the end stages of a
particular relationship to an existing toolset and intellectual apparatus.
Namely, that of the Unix ecosystem, broadly considered.

That is to say: I had reached a certain level of competence, if not exactly
mastery, in the most generally useful command-line environment. I had passed
through the amateur enthusiast phase into the working professional1 phase, while retaining a sense that the system we had
was grounded in sound principles, to the extent that in its broad outlines it
was historically inevitable. Ragged around the edges, sure, and complicated by
all sorts of counterproductive bullshit, but still under it all a sort of
workingperson’s rugged theoretical coherence.

Last fall, Paul Ford wrote this, in some thoughts on
tilde.club (emphasis mine):

Looking at the cabal it turns out the nerds best known to me are realllly a
lot of white dudes. And looking at the signup sheet, this community def. is
going to be okay on white dudes. We’re nailing it, white-dudewise. But Unix
is for everyone. It’s like the Sistine Chapel or Angkor Wat. It’s bigger than
the culture that made it. It’s for our servers and our phones and it’s for
all the genders and all the races and most of all Unix is for the children.

That last part really resonated with me, like a lot of tilde.club-related ideas
did. I figured ok, we have this thing (these many interlocking things) that
are a deep part of our cultural heritage. If they were accessible to more
people, people would have greater recourse in coping with the evils of
computing and the power imbalances of the modern network. The tech is here, I
might have said, it is just not very equitably distributed.2

I retain a lot of that impulse, but implicit in it was a judgment of the Unix
things as also being more or less the right ground for the efforts of the nerd
class. Or at least one such ground: A probability cloud of useful
abstractions and scaffolding within which sensible people built tooling, and in
ignorance of which much absurd wheel-reinventing was constantly enacted.

I have for a long time now been a conservative, of sorts, about actually
existing software and innovation in user-facing systems. Harshly critical of
the last decade’s inventions and revolutions in UX and system architecture.

✤

Two aphorisms, as they sound in my head, without looking them up:

“Those who do not understand Unix are condemned to reinvent it, poorly.”

“Unix is the worst OS except for all of the others.”

✶

I switched jobs this year. Where I used to focus on a handful of big,
application-layer software projects and a pile of business logic, instead I did
customer-facing stuff for Adafruit: I wrote little
introduction-to-Linux tutorials and documented simple hardware projects and
helped support the Raspberry Pi. I read too much of
debian-devel. I wrote a lot of shell scripts. I fought with
version control. I did everything wrong.

I also listened to my friends talk about intrinsically hard problems of their
work: Deploying and configuring and coordinating big systems. Making little
systems behave consistently for many users. Asking questions about system
state and getting consistent answers.

❀

Here is a very partial list of things I think I know right now:

— The Bourne Shell and its descendants are genuinely terrible programming
languages.

— Systems tasks are of sufficient complexity to benefit from a first-class
high level language where fundamental abstractions at the level of “string”,
“variable”, “loop”, “conditional”, etc. are not debilitating to use.

— Regexps are a total fucking horrorshow. Because of the Turing
Tarpit, or something very like it. Because semantic density
only contributes to legibility up to a certain point. Because you can always
hurt yourself real bad with a chainsaw, even if you know what you are doing.

— The packaging of software for Debian is an unremitting nightmare.
Packaging software is just an unremitting nightmare. Also configuration
management.

— Configuration management and deployment tools (Puppet / Chef / Ansible /
Salt / your bespoke artisanal Python script) have in recent years adopted
the language of declarative state and idempotence. This is, in practice,
somewhere between a polite fiction and a dirty lie.

— Determinism in configuration of an entire system is a noble goal, it’s
just totally fucking impossible for mere mortals to achieve in the current
scheme of things.

— Most humans aren’t smart enough to write safe or secure programs in C.
Most useful systems-level programs and services are written in C.

— One way to think of a filesystem is as a powerful baseline abstraction for
most things in a computer system. Another is as a massive collection of
inconsistently represented and needlessly hierarchical state.

— Statefulness is the Fruit of the Tree of Knowledge of Good and Evil.

— Arbitrarily structured text files as baselines for configuration and source
code have nearly intractable advantages. You only need to deal with a system
where Every Last Thing is in XML, think hard on the Windows Registry, or
maintain somebody’s Microsoft Access business logic to understand this much.
On the other hand, it renders every system configuration task a nightmare of ad
hoc text manipulation.

— The officially-sanctioned desktop environment situation on nearly every
Unix derivative is deeply unpleasant, and has been in a state of decreasing
quality since the first decade of this century.3

— The terminal as a primary working environment is a jungle of sharp edges,
and has become ever more difficult to treat as a refuge in a world where a)
people want to communicate in pictures, and b) device manufacturers are doing
everything in their power to kill the physical keyboard no matter what you have
to say about it.

I’ll stop.

✨

I guess my point is this: Unix sucks. I say this with love, but still,
sucks. And not just “the Unix we actually got”, but also the theoretical,
idealized, ur-Unix. Its basic ideas, for all of the brilliance and pragmatic
down-in-the-dirt utility threaded through them, are not really sufficient to
building humane systems.

More broadly, operating systems as they are currently built are bad
environments for normal human beings, and nightmare environments for the
specialists who work on them directly.

I’m inclined to think this matters, because I’m still a strong partisan of the
view that regular users should have ownership and control over their machines.
That a large share of computational utility can (and should) be kept local and
distributed, if we are to stand any chance of avoiding the worst outcomes of
the architecture of centralized control that the internet has made so
practical, so necessary-seeming to all the people with the power to make tech
decisions.

I also have a selfish desire, as a specialist, for a world in which the working
conditions of programmers and sysadmins are less full of broken crap. Because
for all that technical gigs are some of the better work you can get these days,
as far as money and respect and not breaking your back go, they are also
kind of shit in ways that both grind people down and filter out to the rest of
society through countless little channels.

❂

I’m eliding so many things here. The history of this kind of criticism. The
other could-have-been worlds of OS structure. Ted Nelson and all that. So on
and so forth. Maybe I’ll get to those things. I’m not exactly sure.

To be less negative, though, it’d be interesting to spend time thinking about
not just what it is that’s wrong with the current dispensation, but what it is
that could be fixed and outright replaced in the light of how our existing
systems fail—but also how they succeed, or nearly do. Because it seems
like right now we’re in the middle of replacing everything (necessary) with
stuff that is built in resolute ignorance of a huge tradition of qualified
success and tragicomic failure (doomed).

thursday, october 22

filing cabinets

people ask me sometimes about becoming a programmer
it's a fair enough question
i'm at least nominally a professional in the field
and it can be good work if you can get it

i think my answers have varied in quality,
but it seems like almost no one ever finds them very satisfying

by definition
being in the condition of wanting to be a programmer
in line with many ambitions that merely require the investment
of many thousands of hours
is being unable to discern
bad answers from good — not knowing how to know

next time maybe i will take a different tack and ask
well, are you interested in being a career bureaucrat?

If by “Conservative SF” one means “military SF with space battles and laser
rifles, where the good guys are an expansionist, conquering force, where the
protagonist is a strong-jawed and clear-headed hero who outfights and outwits
those who would confound the plot, and where tactics, strategy, and technology
are both visionary and critical story elements,” then may I recommend the Hugo
Award-winning Ancillary Justice?

…which somehow kind of struck a nerve, because I’m not sure I did want to
read about the good guys as expansionist, conquering force. But then again I
have read a decent amount of MilSF at one time or another, and I am still a
sucker for space battles, and anyway I was curious what it was about Leckie’s
stuff that would so upset the GamerGate-slash-John-Birch-Society-slash-Christofascist
wing of marginally-published SF.

With respect to KathrynT (and thanks for nudging me toward the read), I don’t
think that comment’s a very accurate description of the work. So far, it feels
like a complicated meditation on the mechanics of empire, oppression, political
power, surveillance, and distributed identity. It’s ambivalent about empire,
about the situation of its protagonist—a piece of literal machinery in a
ruthless, exploitative colonialist project—but it often feels like an essay
on empire’s dysfunctions and cruel hypocrisy.

Then there’s the gender thing. Which I think works, mostly, and is
interesting, and reads for the most part far more as an element of the
worldbuilding than it does as polemic. Which, oddly, despite the deeply sexist
and gender-essentialist past of so much of the genre, is what I remember about
all the SF that used to shake my younger brain’s half-formed notion of what
gender was, exactly.

Sunday, October 18

New York always feels something like a foreign country. Not like one so
foreign that its modes and routines are completely illegible, but one far
enough from usual life that I’m often unsure what the rhythm is, the protocol,
the whatness-of-what. I speak the language (one of them, part of it) but I
don’t exactly speak the place.

Is it as far from Colorado, on some hypothetical graph of civilization, as
Colorado is from London, Budapest, Tirana, Christchurch, Prague? Maybe not.
There’s no mistaking the American superstructure of both places for anything
else. Still, it’s a trip.

Thursday, October 15

Circling back to thoughts I had a while ago, it sure is depressing
how much the actual technology of the database is invisible to the general
population.

Everybody talks about scale. (Me included.) You know what? Fuck scale.
Scale the way it gets meant in software and networks might be as bad as the
Empire State Plaza as an architectural ideal. Nobody wants to live in a
landscape of fascist concrete spaceships. Normal humans mostly don’t want to
spend their days smack in the middle of Times Square either. These places
might be a living for a few, but they’re not exactly a solid material and
spiritual foundation for a thriving and humane civilization. You need fields
and neighborhoods and places where things are quietly growing.

Here’s what I’d like in a database technology: The kind of scale it should
primarily worry about isn’t the scale of running a single web app or social
network silo with a massive userbase. Not corporate scale. Instead, it
should “scale” to the level of widespread use by individuals and small
organizations, at about the level of a tool like spreadsheets, except with the
available guarantees and structural rigor that are so specifically lacking from
passing around files full of arbitrarily input row-based data.

And by this I don’t mean that the thing in question should be a single-provider
closed-source service running on other people’s computers. That’s 90% of the
problem in the first place. Not that I’m claiming I know how to make it
available otherwise. I think what I’m asking for is a modality for exchanging
and accessing data that affords users a bunch of strong abstractions without
enmeshing them in all the hard details of service configuration and messing
around with files. And most of the ways we know how to do that kind of thing
involve corporate scale and other people’s computers.

As an example, the technology of files is still available to most users, even
though the BigCorp tech world is trying to hide it from non-developers, and
seems likely to succeed once everyone uses iOS/Android-style clients where the
app silo is the whole paradigm. Still, it’s taking a long time to get there.

So why are files resistant to total erasure from the user experience, despite
so much effort to supplant them and so much disdain from the Ted Nelson wing of
computer intellectualism? It’s not just that they’re a fundamental
abstraction for building systems at this point, is it?

Maybe a big part of it is that filesystems with roughly equivalent semantics
got built into all of the now-major operating system families so early in the
history. No really equivalent thing happened for databases. (People with
experience in the mainframe world might quibble, but that world was always
about corporate scale.) And it’s kind of like you might need that level of
support (at the baseline of the system in every common end-user OS) for the
tech to be both user-accessible and not instantly susceptible to centralization
of control.

Which is sort of a problem, since it would run directly counter to the
perceived interests of most corporations with the resources to make decisions
about end-user operating systems these days.

Wednesday, October 7

I don’t really care about Jonathan Franzen, in a way that probably
functions as a microcosm of all the ways I usually fail to care about literary
fiction as a genre. Life is short, the available writing is well-nigh
infinite, and there is so much else to be interested in one way or another.
It’s even possible that, once or twice, I have defended some notion Franzen has
expressed in prose as perhaps not entirely worthy of the overheated loathing
his stylings tend to generate in some quarters of the internet, but that’s the
sum total of my involvement to date, and in general I’m happy to keep it that
way.

Still, sometimes an author or work you don’t have the energy to care about at
all directly affords the pleasure of reading a really good bad review.

There’s an old joke, which holds that in heaven, the cooks are French, the cops
are English, and the engineers are German; whereas in hell, the cooks are
English, the cops are German, and the engineers are French. We live in a sort
of silly cultural hell where the columns are composed by Thomas Friedman, the
novels are written by Jonathan Franzen, the debate is framed by CNN, and the
fact-checking is done by no one.

Monday, October 5

Somewhere around nine o'clock at night, I get out of the shower and
develop a craving for bad junk food. I wind up driving to the Diamond Shamrock
with the radio on. It’s classical I don’t exactly recognize, but it seems
familiar. Jeremy’s getting out of his truck as I come around the corner, but I
don’t stop to say hey. There’s a version of tomorrow where I have elected to
stop by for a beer tonight, and it doesn’t start with me waking up sober, let
alone at a decent hour.

The gas station has Gardettos Original Recipe Snack Mix, so I get a bag of
that, the big package of Twizzlers, and a bottle of iced tea.

☙

It’s shameful how much of the tea I consume these days comes out of a plastic
bottle from a gas station. I was raised on two kinds: Made in bulk with
powder, and sun tea made with Lipton teabags in a big glass jar. The important
thing wasn’t so much the origin, though: It was that you made the stuff at
home, or at least mixed it there, and always had a pitcher in the fridge. At
least through the hot months.

❂

Of course the hot months are just about done. There’ll be more hot days —
maybe even weeks — but it’s been in the 60s, hazy gray, the light weirdly
muted, the ground wet with occasional rain. I’ve been wearing long pants and
shoes, digging out sweatshirts and hats. This morning I turned the heat on in
my car as I rattled over to the office.

Somebody died on Long’s Peak a couple of days ago. Reading the forums now
confirms a gut feeling that it’s awfully late in the year for that shit, if you
aren’t a lot more experienced than me. Maybe even if you are.