Last Saturday, when the bus from Canberra pulled into Sydney's central
station, I found myself feeling curiously nostalgic for this city, and
particularly this bustling and somewhat seedy[1] neighborhood of Haymarket
and Redfern.

I only spent 5 days in Sydney, but living in a shared house there, walking
up to Central every day, and returning to the outskirts of the Haymarket
every evening to slurp noodles or other asian food, I got into a routine.
And got a sense of the place that goes perhaps a bit beyond the tourist
sights.

Perhaps if I'd had more time I would have found a decent coffee shop
that had both free wifi and abundant seating. They seem scarce in Sydney.
I instead often got on the ferry to Manly when I wanted some sit down and
code time.

One time when I was exploring the headlands above Manly beach, I noticed
this sign.
Then I ran into this guy. Click him for an amusing video.

Anyway, Sydney is on my very short list of cities I'd actually enjoy spending
some more time in some day, along with San Francisco, Vancouver, Oslo, and
London.

[1] Depending on what's inside all the "VIP lounges" and "Thai massage
parlours" on every corner that I did not explore, perhaps thoroughly seedy?

After 27 hours of travel, I'm finally back from my
first Linux.conf.au.

This was a great conference! Some of my favorite highlights:

Out for dinner the first night, my whole table started spontaneously
talking about Haskell, including details of IO management frameworks
like conduit and pipes, and a new one that's waiting in the wings.
That just doesn't happen in the real world. A lot of us continued to
do Haskell stuff in the hallway track, although for some reason there
were no Haskell talks on the official schedule. Maybe it's time for a
Haskell miniconf at LCA?

Meeting Josh, Jamie, Sarah, Brendan, Craig, and others I've worked with
online but never encountered IRL. Also reconnecting with old friends
(some I'd not seen in 13 years) and finding new ones.

The speaker's dinner in a revolving restaurant overlooking Canberra.
Leave it to a restaurant full of geeks to invent an asychronous communications
medium in such a setting. (Notes attached to windows to be read and
answered by tables as they rotated by.)

Meeting quite a lot of git-annex users, Kickstarter backers, and
people interested in using git-annex. Thanks to everyone who came up
to me for a chat.

The evening pickup board game sessions. Especially my wiping out three
other Tsuro players at once by forcing them all to end on the same square. ;)
These were where I felt the most at home in Australia.

Robert Llewellyn dropping
in and mingling with fans of Red Dwarf and Scrapheap Challenge. One of the
very few actors I could possibly fanboy on, and LCA not only somehow got
him, but constructed an atmosphere that allowed for this photo of us.

My git-annex talk
went ok, despite technical problems with the video output, which the video
team did a great job of working around. The first demo went well, and the Q&A
was excellent.

The demo of the git-annex assistant webapp was marred by, apparently, a
bad build of the webapp -- which I ironically used rather than my usual
development build because I assumed it'd be less likely to fail. At least I
may have a way to reproduce that hang somewhat reliably now, so I can get
on with debugging it. I will be redoing a demo of the webapp as a screencast.

Here are some of the best talks I attended. (I have quite a lot more queued
up to watch still, and had to cut several due to space.)
Click titles for videos, or browse all the videos.

Git for Ages 4 and Up
Schwern has found a new, excellent way to explain git. I felt rather
bad for using up a seat, especially once people were kicked out
when the room was filled over capacity. But I enjoyed every minute of it.
(Also has the best speaker intro ever. "Schwern once typed git pull --hard
and pulled GitHub's sever room across the street.")

BTW, I must confess: I left the red apple on teacher's desk.

Radia Perlman's keynote
Network protocol design and poetry from one of the quiet heros of
our field. I knew Spanning Tree Protocol was used in ethernet,
but it just works, so like many I never paid much attention to it.
This talk felt like the best lectures, where you're learning from a master
on multiple levels at once.
Well done work often becomes an unremarked part of the landscape,
which I sometimes find unrewarding, so it was great to have Radia give
some perspective on what that's like over the course of decades.

Lightning Talks
A 90 second time limit really helps. Too many conferences have 5 minute
talks, which is less exciting. If you still find them boring,
skip forward to 13:50, where pjf
does two talks in 90 seconds! (If a 20 second talk on depression
is too .. manic, there's an encore at the end.)

The IPocalypse 20 months later
A reality check, with real data. Very important stuff here.
We need to work to avoid this worst case scenario, and we also need to design
around it.

REPENT!!! FOR THE END OF THE UNIX EPOCH IS NIGH!!!
Wildest talk beginning I've seen since RMS put the hard drive halo on his
head. And to an important point: Any programs that deal with dates 25 years
in the future already need to be fixed today to deal with the epoch rollover.
This got me digging around in Haskell date libraries, to make sure they're
ok.

After arriving yesterday, and doing the obligatory jetlagged wander around
Circular Quay to the
Opera House, I crashed for 15 hours sleep and seem reasonably well over the
jetlag now. I'm staying near Central Station, which is great because it's
very convenient to transport and there's lots of cheap ethnic food.

view down my street

Today I took the ferry over to Manly Beach.
I had to get in some in swimming of course, although today was cloudy and only
in the 80's (F). That was ok, but the really nice bit was walking over to
Shelly Beach
and up along the sandstone cliffs near
North Head.

Highlight was finding a small gotto high on the cliff,
with some beautiful sandstone filigree overhead,
that seemed carved by water (but probably really by wind), and
contained a shady bench overlooking the Pacific.
Great place to enjoy a lunch of crusty bread and salami if you're
ever in the area.

Topped off a great day with a random trip to
Neutral Bay
(the D&Der in me liked the name, and why not, ferries are free) and
a wander through the
Royal Botanic Gardens.

I'm hoping to spend either Thursday or Friday at a co-working or hackerspace
in Sydney, but I don't know where yet. Any locals have ideas?

My least favorite forecast, and we've been having a lot of it lately. Since
winter began (Dec 21st), my PV array has produced only 102 amp-hours of
power. There have been only 3 days that could be considered sunny at all
(even for a few hours). To put this in some perspective for you, Wolfram
Alpha tells me that 102 amp-hours is half of a typical car battery
capacity. So I've really had to scrimp & save. I disconnected my router to
save power, and I plumbed the depths of the battery bank (further than it's
really safe to go). And I was completely out of power for several days.

So, I've bought a generator. I had to order it online, because it seems
every store in the entire eastern US is completly out of stock, due to
hurricane Sandy. This house has a generator shed (complete with a massive
old generator that doesn't work), but I put the new, small generator in the
furthest outbuilding, from where it can only barely be heard in the house.
Later I'll be able to run a cable from its 12v terminal to charge the
battery bank (if the sun doesn't get there first); for now it only
powers my laptop. Nasty noisy smelly dirty thing, it pains me to run it.

Back to the "overcast" forecast. I've heard meterologists say that one
unexpected consequence of climate change seems to be that weather patterns
are persisting longer, giving us this summer's weeks of neverending heat,
and large, slow-moving storms. I suspect this could also explain the
blankets of clouds that have been settling in for weeks at a time, on and off
since mid-November. But I only have 2.5 years of not very good data. Before,
I paid about as much attention to whether it was sunny as anyone
else, which is to say, not really very much. Anyone know of a source of
historical cloud cover data?

This year, I've gradually realized that I no longer identify as a perl
programmer. For a decade and a half, perl was the language I reached
for to solve any problem that didn't have a good reason to be solved in
some other language. Now I only reach for it in the occasional one-liner --
and even then I'm more likely to find myself in ghci and end up with a
small haskell program.

I still maintain plenty of perl code, but even when I do, I'm not thinking
in perl, but traslating from some internal lambda calculus. There's quite a
few new-ish perl features that I have not bothered to learn, and I can feel
some of the trivia that perl encourages be kept in mind slipping gradually
away. Although the evil gotchas remain fresh in my mind!

More importantly, my brain's own evaluation of code has changed; it doesn't
evaluate it imperatively (unless forced to by an appropriate monad), but sees
the gesalt, sees the data flow, and operates lazily and sometimes, I think
in parallel. The closest I can come to explaining the feeling is how
you might feel when thinking about a shell pipeline, rather than a for loop.

Revisiting some of my older haskell code, I could see the perl thinking
that led to it. And rewriting it into pure, type-driven, code that took
advantage of laziness for automatic memoization, I saw, conclusively
that the way I think about code has changed. (See the difference for yourself:
beforeafter
)

I hear of many people who enjoy learning lots of programming languages, one
after the other. A new one every month, or year.
I suspect this is a fairly shallow learning. I like to dive deep. It took
me probably 6 years to fully explore every depth of perl. And I never saw a
reason to do the same with python or ruby or their ilk; they're too similar
to perl for it to seem worth the bother. Though they have less arcania in
their learning curves and are probably better, there's not enough value to
redo that process. I'm glad haskell came along as a language that is
significantly different enough that it was worth learning. The deep dive
for haskell goes deep indeed. I'm already 5 years in, and have more to
learn now than I ever did before.

I'm glad I didn't get stuck on perl. But I may be stuck on haskell now
instead, for the foreseeable future. I'd sort of like to get really fluent
in javascript, but only as a means to an end -- and haskell to javascript
compilers are getting sufficiently good that I may avoid it.
Other than that, I sense adga and coq beckoning with their promises of proof.
Perhaps one of these years.

Of course if Bradley Khun is right and
perl is the new cobol,
I know what I'll be doing come the unix rollover in 2038. ;)

I haven't used my yurt much at all lately. It was an excellent getaway
when I needed to get away; these days I don't. Or, perhaps, some would
say, I've gotten away...

So yesterday, since I happened to be over there, and after we'd played
board games to our hearts content, we decided to take it down. This had
been on the todo list since last fall, so it was not entirely spur of the
moment. Still, the days are short this near to solstice and this one only
had a few hours of light left.

It'd been up for well over 4 years, in a wet environment. The roof is
pretty well shot and will need to be replaced, but the canvas walls are
still in good shape. There was a surprise; it seems a mouse or something
started nibbling on all the wood of the walls recently, and even the center
ring is pitted and worn by rodent teeth. It all seems still structurally
sound. And looking back at my setup pictures, I'm not bothered by the yurt
having aged, it just makes it seem more real.

We left the bones of the yurt standing on its hill, seeming very
appropriate under the bare winter trees. Will return soon and finish taking
it down. And perhaps in some years I'll find a need for it again.

(Pictures by my sister, who will be annoyed at me linking to this
not-yet-formally published post about the yurt from her POV:
Decline and fall of the yurt)

Apologies in advance for writing a long blog post about the dull and
specialised subject of
double-entry accounting
from the Unix tools perspective, that ends up talking about Monads to boot.
I can't believe I'm about to write such a thing, but I'm finding it an oddly
interesting subject.

double-entry accounting

I've known of, though probably not really understood double entry
accounting for a while, thanks to GnuCash. I think
GnuCash did something rather special in making such a subject approachable
to the layman, and I've been happy to recommend GnuCash to friends. I was
stoked to find a chapter in my sister Anna's
new book
that happily and plausibly suggests readers use GnuCash.

But for my personal use, GnuCash is clunky. I mean, I'm a programmer,
but I can't bring any of my abilities to bear on it, without a deep dive
into the code. The file format is opaque (and a real pain to keep checked
into git with all the log files); the user interface is often confusing,
but there's no benefit to its learning curve, it never seems to get better
than manually entering data into its ledger, or perhaps importing data from
a bank. I've never found the reports very useful.

I've got perhaps a year of data in GnuCash, but it's fragmented and
incomplete and not something I've been able to motivate myself to keep up
with. So I have less financial data than I'd like. I'm hoping ledger will
change that.

ledger

I've known about ledger for a while, at least since
This Linux Weekly News article.
It's a quintessential Unix tool, that simply processes text files.
The genius of it is the simplicity of the file format, that gets the essence
and full power of double entry bookeeping down to something that approaches a
meme. Once you get the file format stuck in your head, you're done for.

2004/05/27 Book Store
Expenses:Books $20.00
Liabilities:Visa

starting to use hledger

Now as a Haskell guy, I was immediately drawn to
the Haskell clone, hledger. It's nice that there
are two (mostly) compatable implementations of ledger too. So from here on,
I'll be talking about hledger.

I sat down and built a hledger setup for myself the other day.
I started by converting the GnuCash books I have been keeping up-to-date,
for a small side business (a rental property). It quickly turned into
something like programming, in the best way, as I used features like:

Include directives, so I can keep my business data in its own file,
while pulling it into my main one.

Simple refactorings, like putting "Y 2012" at the top, so I don't have
to write the year in each transaction.

Account aliases, so I can just type "rent", rather than
"income:rental" and "repairs:contractor" rather than
"expenses:home repair:contractor"

All the power of my favorite editor.

a modern unix program

While I've been diving into hledger, I've been drawing all kinds of parallels
between it and other modern Unix-friendly programs I use lately. I think
we've gone over a bit of a watershed recently. Unix tools used to be either
very simple and crude (though often quite useful), or really baroque and
complex with far too many options (often due to 10-30 years of evolution).
Or they were a graphical user interface, like GnuCash, and completely
divorced from Unix traditions.

The new unix programs have some commonalities...

They're a single command, with subcommands. This keeps
the complexity of doing any one thing down, and provides many
opportunities for traditional unix tools philosophy, without
locking the entire program into being a single-purpose unix tool.

hledger's subcommands range from querying and reports, to pipable
print, to a interactive interface.

They have a simple but powerful idea at their core, that can be
expressed with a phrase like "double entry accounting is a simple
file format" (ledger), or "files, trees, commits" (git).

Building a tool on a central idea is something I strive
to do myself. So the way ledger manages it is particularly interesting to
me.

They are not afraid to target users who have a rather special kind
of literacy (probably the literacy you need to have gotten to here
in this post). And reward them with a lot of power.

Ledger avoids a lot of the often confusing terminology around accounting,
and assumes a mind primed with the Unix tools philosophy.

If there's a GUI, it's probably web based. There's little trust in
traditional toolkits having staying power, and the web is where the
momentum is. The GUI is not the main focus, but does offer special
features of its own.

hledger's web UI completely parallels what I've doing with the git-annex
webapp, right down to being implemented using Yesod -- which really needs
to be improved to use some methods I've developed to make it easier to make
webapps that integrate with the desktop and are more secure, if there are
going to be a lot more programs like this using it.

importing data

After manually converting my GnuCash data, I imported all my PayPal history
into hledger. And happily it calculates the same balance Paypal does. It
also tells me I've paid PayPal $180 in transaction fees over the years,
which is something PayPal certianly doesn't tell you on their website.
(However, my current import from PayPal's CSV files is a hackish, and only
handles USD currency, so I miss some currency conversion fees.)

I also imported my Amazon Payments history, which includes all the
Kickstarter transactions. I almost got this to balance, but hledger and
Amazon disagree about how many hundreths of a cent remain in my account.
Still, pretty good, and I know how much I paid Amazon in fees for my
kickstarter, and how much was kicked back to Kickstarter as well.
(Look for a cost breakdown in some future blog post.)

At this point, hledger stats says I have 3700 transactions on file,
which is not bad for what was just a few hours work yesterday.

One problem is hledger's gotten a little slow with this many transactions. It
takes 5 seconds to get a balance. The ledger program, written in C, is
reportedly much faster. hledger recently had a O(n^2) slowdown fixed,
which makes me think it's probably only starting to be optimised. With
Haskell code, you can get lucky and get near C (language, not speed of
light) performace without doing much, or less lucky and get not much better
than python performance until you dive into optimising. So there's hope.

harnessing haskell

If there's one place hledger misses out on being a state of the art modern
Unix program, it's in the rules files that are used to drive CSV
imports. I found these really hard to use; the manual notes that "rules
file parse errors are not the greatest"; and it's just really inflexible.
I think the state of the art would be to use a Domain Specific Language here.

For both my Amazon and PayPal imports I had CVS data something like:

date, description, amount, fees, gross

I want to take the feeds into account, and make a split transaction, like this:

This does not seem possible with the rules file. I also wanted to combine
multiple CVS lines, to do with currency conversions, into a single
transaction, and couldn't.

The problem is that the rules file is an ad-hoc format, not a fully
programmable one. If instead, hledger's rules files were compiled into
standalone haskell programs that performed the import, arbitrarily
complicated conversions like the above could be done.

So, I'm thinking about something like this for a DSL.. I doubt I'll get
much further than this, since I have a hacked together multi-pass importer
that meets my basic needs. Still, this would be nice, and being able to
think about adding thing kind of thing to a program cleanly is one of the
reasons I reach for the Haskell version when possible these days.

First, here's part of one of my two paypal import rules files
(the other one extracts the transaction fees):

That seems like something non-haskell people could get their heads around,
especially if they didn't need to write the boilerplate function
definitions and types at the top. But I may be biased. :)

Internally, this seems to be using a combination Reader and Writer monad
that can get at fields from a CSV line and build up a Transaction.
But I really just made up a simple DSL as I went along and thew in
enough syntax to make it seem practical to implement. :)

Of course, a Haskell programmer can skip the monads entirely, or use others
they prefer. And could do arbitrarily complicated stuff during imports,
including building split transactions, and combining together multiple
related CVS lines into a single transaction.

Imagine if you could send git pushes to any of your friends on Google Talk
or other Jabber (XMPP) servers. Even though you're in different places
and your computers probably cannot talk to one-another directly, you can
share a git repository, without relying on a git hosting provider such
as GitHub.

To xmpp::joey@kitenet.net
* [new branch] master -> xmpp/master

That's what I've built over the past week. So far, it's only supported
by the git-annex assistant, but it would not be very hard to make this
into a separate git-remote-xmpp program, and a push listening daemon,
that could be used with non-git-annex repisitories too.

This is not the kind of place I live, though I don't mind a visit.
I'm not bothered by a more rural life, I thrive on peace and quiet, and
am not bothered by needing to haul my water, split firewood, or spend an hour
just mowing my driveway.

I bought a GPS the other day, and it doesn't know where my driveway is..
or where the road leading to my driveway is..
or even about the highway leading to that road!

The only thing that really bothers me about living here off the map is that
I'm in the middle of a food desert too. The nearest
semi-decent grocery store and produce market are 40 minutes away. There's
one restaurant worth eating at, and while a very congenial place to eat
rustic food while watching the river flow by, it serves almost no
vegetables. Adding to the problem, my fridge is a rather expensive propane
model, which is about half a normal size, and has no freezer. Managing a
healthy meal plan is a constant struggle, though I do have time to cook, at
least, thanks to not having the kind of commute that is typical around
here.

Well, the food desert was my only real problem. Now I can add to that the
UPS emergency. A week ago, a package left Shenzen China for here. It quickly
passed through Anchorage Alaska, and with a few days had arrived an hour
and a half from here. Getting it here has taken the whole rest of the week.

I only knew UPS attempted delivery because a neighbor told me. Seems they
took one look at my driveway and turned around. They never tried again, and
they didn't bother to leave a note on the mailbox, or anything. Now, the US
postal service is happy to drive up this driveway; they drive a sensible
4x4 around here. I think DHL and Fedex have been here too. (Contrary
to John Goerzen's experiences)
My compact car is not bothered by my driveway either...

But UPS's brand involves large brown trucks, even where inappropriate, and
damage to them from such things as low-hanging branches apparently comes out
of the driver's paycheck. And while normally UPS leaves small things in my
mailbox, or with a neighbor, this was an important package from China, and
it needed a signature. But really, a post card? "emergency"?!

So, I bought the GPS to find UPS, because my package was in an industrial
park in some town I've never been to. And on Tuesday I was within an hour
of there and swung by. And it was closed. UPS's website had not mentioned
this; other businesses are open the day after Memorial day. That turned out
to be a really annoying day all around. When I got home, after a night
away, I found my fridge's fire had blown out, and nearly all my food was
only fit for the dog.

Today I went out to try again (and buy groceries). When I reached the
UPS depot.. it was closed! Until 3 pm. At this point I felt this story
was reaching full-on farce.

After cooling my heels in a library for three hours, I was finally able to
open the most over-packed package I've probably ever seen.

My kickstarter reward sample looks good.. I'll be ordering a lot more, but
not to this address.

As a coda to the UPS farce, arriving home today I checked my mailbox, and
found the UPS postcard had finally arrived. It included mention of the 3 pm
opening time, and noted that I had until tomorrow before they'd return my
package to China.

The coda to the coda is that on my way home from this second annoying day
delivered courtesy of UPS, I stumbled on a restaurant overlooking a
waterfall, just outside the town where I get my groceries. It serves food
that satisfies both my Cajun and healthy cravings. Now I just need
a larger fridge..