Posts about python

I have, intermitently, for the past few months, been writing a book. You can see it here and I am fairly happy with how it's going.

I am not so happy with the tooling.

When I started writing it, I didn't want to write a tool, I just wanted to write a book.
So, I looked for "things to make a book out of markdown" and found a few, including mdBook and gitbook and tried them both, then decided to go with gitbook because it seemed more developed.

But, you know how this works. Things started drifting.

One initial obstacle was that I wanted the code in the book to come from the actual working code, not be a copy. I also wanted to "execute" the chapters and make the output of the chapter's code part of the chapter itelf.

I found a nice tool for this called pyLiterate which... I ended up extending to allow for things such as "include this piece of code from that file but do not execute it" and whatever, and integrating it into gitbook involved ... a lot of Makefiles.

And then wen the output appeared in gitBook ... I really didn't like how syntax highlighting worked. I wanted prettier and specifically to be able to highlight some lines, and to have line numbers matching the file contents and not starting from 1 ...
and I ended up writing a Gitbook Extension

And I added support for graphviz in pyLiterate because I wanted SVG diagrams as part of the code and not as separate things.

And I wrote and I wrote.

And then life happened and I stopped writing. And whenever I wanted to start writing again in a new setup something broke.

Maybe the gitbook version installed or its dependencies had a security issue and needed updating.

Maybe the updated gitbook broke my plugin

Maybe things just were "off" and I had to track it down*

And a couple of days ago, I just got angry. And what the hell this is just a lame static file generator and I know those suckers.

So, I decided to see how much work would it be to reimplement, somewhat dumbly, gitbook.

A long time ago I "wrote a web browser". Those there are some very heavy quotes.
You may imagine me doing air quotes while I write it, maybe?

That's because I didn't really, what I actually did was write UI around
Qt's webkit-based widget. It was a fun project, specially because I did it with
the absurd constraint of staying below 128 lines of code.

I want to learn new languages. But new as in "new to me", not new as in
"created last week". So I decided to play with the grandaddy of all cool
languages, LISP. Created in 1958, it's even older than I, which is good because
it's experienced.

One "problem" with LISP is that there are a million LISPs. You can use Scheme
or Common Lisp, or Emacs' Lisp, or a bazillion others. I wanted something
simple so it was supposed to be Scheme... but a few days ago I ran into
something called Picolisp and it sounded so cool.

And... why not. I am a bit stale in my programming language variety. I used to be fluent in a dozen, now I do 80% python 10% go, some JS and almost nothing else.
Because I learn by doing, I decided to do something. Because I did not want a problem I did not know how to solve to get in the way of the language, I decided to reimplement the example for the python book I am writing: a text layout engine that outputs SVG, based on harfbuzz, freetype2 and other things.

This is a good learning project for me, because a lot of my coding is glueing things together, I hardly ever do things from scratch.

So, I decided to start in somewhat random order.

Preparation

I read the Nim Tutorial quickly. I ended referring to it and to Nim by example a lot. While trying out a new language one is bound to forget syntax. It happens.

Wrote a few "hello world" 5 line programs to see that the ecosystem was installed correctly. Impression: builds are fast-ish. THey can get actually fast if you start using tcc instead of gcc.

SVG Output

I looked for libraries that were the equivalent of svgwrite, which I am using on the python side. Sadly, such a thing doesn't seem to exist for nim.
So, I wrote my own. It's very rudimentary, and surely the nim code is garbage for experienced nim coders, but I ended using the xmltree module of nim's standard library and everything!

To build a svg tag, you can use <>svg(attr=value) which is delightful syntax. But what happens if the attr is "xmlns:ev"? That is not a valid identifier, so it doesn't work.
So I worked around it by creating a StringTable filling it and setting all attributes at once.

A good thing is the when keyword. usingit as whenisMainModule means that code is built and executed when svgwrite.nim is built standalone, and not when used as a module.

Another good thing is the syntax sugar for what in python we would call "object's methods".

Because Add takes a Drawing as first argument, you can just call d.Add() if d is a Drawing. Is simple, it's clear and it's useful and I like it.

One bad thing is that sometimes importing a module will cause weird errors that are hard to guess. For example, this simplified version fails to build:

WAT? I am not using newStringTable anywhere! The solution is to add importstrtabs which
defines it, but there is really no way to guess which imports will trigger this sort of issue.
If it's possible that importing a random module will trigger some weird failure with
something that is not part of the stdlib and I need to figure it out... it can hurt.

Not much to say, other that the code for parsing --page-size is slightly less graceful
than I would like because I can't figure out how to split the string and convert to
float at once.

So, at that point I sort of have the skeleton of the program done. The missing pieces
are calling harfbuzz and freetype2 to figure out text sizes and so on.

Interfacing with C libs

One of the main selling points of Nim is that it interfaces with C and C++ in a striaghtforward manner.
So, since nobody had wrapped harfbuzz until now, I could try to do it myself!

First I tried to get c2nim working, since it's the recommended way to do it. Sadly, the version
of nim that ships in Arch is not able to build c2nim via nimble, and I ended having to manually
build nim-git and c2nim-git ... which took quite a while to get right.

And then c2nim just failed.

So then I tried to do it manually. It started well!

To link libraries you just use pragmas: {.link:"/usr/lib/libharfbuzz.so".}

To declare types which are equivalent to void* just use distinctpointer

As part of a book project aimed at almost-beginning programmers I have written what may as well pass as the first part of a Git tutorial. It's totally standalone, so it may be interesting outside the context of the book.

It's aimed at people who, of course, don't know Git and could use it as a local version control system. In the next chapter (being written) I cover things like remotes and push/pull.

LIke mentioned before I am trying to write a book and ... well,
I may be actually making progress? At least the generated PDF is about 170 pages long, which means I have written a bunch in this past month.

I have finished the second of four planned parts, which means I have done about half of it. Since I expect the next two parts to be shorter, it's actually more than that.

The target audience are people who have finished the python tutorial but are not exactly programmers yet. They have the syntax more or less in their heads, but how do you turn that into an actual piece of code?

Part 1 is about "prototyping", the process of dumping an idea into rough code.

Part 2 is about polishing that rough code into ... not so rough code. Includes a gentle introduction to testing, for example.

Part 3 (to be written) is about things that are not code:

Git / Gitlab

Issues

Packaging

Setting up a website

CI

Lots more

Part 4 is still to be thought but basically it will cover implementing a large feature from the ground up.

Once upon a time, I tried to write a book. It did not end well. I was trying to dump a whole lot of knowledge at once. Knowledge I did not really have, to be honest. When I look at that book I see a failed thing.

So, of course, many years later, I am trying again, but with the lessons learned in my mind.

It will be a smaller book.

I am not also writing a whole tool chain for it.

It will be about things I know.

So, what is it?

The temporary title, right now, is something like "Boxes: your second Python book". It says your second Python book because you do need a working knowledge of Python syntax as provided by the official Python Tutorial, but not much else. When there is a particularly hairy piece of code it may link to the tutorial or the reference or something.

The "idea" of the book is to bridge a gap that exists between knowing the basics of reading and writing a language (specially if it's your first!) and being able to effectively using it to create a useful project.

It follows the growth of "Boxes", a simplistic text layout engine, from a vague idea to a fully working, useful, tested, and published piece of software.