Erlang with Fred Hebert

Intro

Erlang releases are a bit like magnets. Everyone who thinks about them shares the same thought: f**king releases, how do they work?

Fortunately, since the years of Emakefiles, reltool and systools, the Erlang community has stood up and improved its tooling continuously.

Rebar has been improving non-stop and keeps getting better for many functions. The newest generation, Rebar3, tries to provide an end-to-end experience to building Erlang projects.

Along with installing Erlang (version R16B03-1 at least), getting a hold of Rebar3 is all you’re gonna need.

For rebar, just download it and follow the instructions. Rebar3 will basically generate a self-executable that you can store in your repository, or install globally on your computer. This tutorial expects that you have installed it in your system and made it available in your $PATH.

Once they’re all installed somewhere in your system, arm yourself with the text editor or IDE of your choice (mine is Vim, because I’m a terrible person) and get ready to write a few things.

My Environment

Despite you being free to develop on whatever you want, I’m gonna go through whatever my setup is.

I don’t use a lot of material outside of that, and the OS will tend to be my IDE – for projects that I tend to work on a lot, I will use tmux scripts (see this blog post for an example) – to get everything going early.

The Project

To avoid the usual Hello World stuff, this tutorial will use a somewhat more fun application to get up and running from a basic Erlang app that can be run within a module, to a proper OTP library that can be included by other projects, to a release than can be self-executing and distributed to client’s computer, or on a server.

Our project will be the replication of one of the most well-known software programs in popular history, used in life-critical situations: Homer Simpson’s console in the episode where he’s so fat he can work at home.

From this episode we can infer the following about the software:

When the program boots, it asks you to press any key.

The program will ask you questions that can be answered by yes/no, but also Y/N or y/n

Most questions can be turned into commands. Each assertion is equivalent to answering a given question positively. For example, Vent radioactive gas? Yes/No can be turned into the Vent gas command.

Nothing should go wrong if you keep pressing Y all the time

After a given delay, a new question is asked

Too many times without venting radioactive gas risks exploding everything

Some behaviours aren’t defined by the TV show, so we go somewhat anyway we feel like

Out of this, a finite-state machine can be created. The one that follows explains what I understood as possible, but you’ll notice I’m not really good at having a consistent notation for events, states, and so on:

Based on this, we’ll be able to draw up a first prototype with all the required state transitions. I’ve also looked for transcripts of the show and extracted the following questions and consequences:

Check core temperature. yes/no:

yes: Core temperature normal.

no: –

Vent radioactive gas?

yes: *gas blows away corn crop*

no: venting prevents explosion (allow yes, show only the first time?)

Sound alertness horn?

yes: *horn sounds in the distance*

no: –

Decalcify calcium ducts?

yes: –

no: –

Special case: after denying venting too many times, the valve must be disabled manually.

The simplest way to write a basic FSM for this one is to use a bunch of function calls. Given Erlang has last call optimization (a call that happens as a return value does not leave a stack trace, and therefore can happen infinitely many times), this is more than adequate.

The Prototype

Our glorious application will be called ‘muumuu’. Whenever I don’t exactly know where I’m going, I decide to prototype stuff. And here I stress the importance of prototype. Despite this fact, it will often end up being in production, but yeah – that’s to be avoided.

I decide to start with the basic stuff to prototype, state transitions. I go for them in a fairly simple manner, top-down:

Cool. Not fantastic looking yet. Basically, an option will only fetch a line of text entered by the user, look at the first response, and return what it is. Showing the options just wraps things up so they look like a prompt.

Interestingly enough, the command has to be waited for in a different process. The problem with this it that Erlang’s standard library doesn’t support a timeout mode for io operations, which would tell us “wait 10 seconds for input or quit”. Therefore, there is a need to move this to a process.

The rest relies on an elusive opts() function that apparently returns all questions and options offered to the user:

This basically is a tuple (I use a tuple because it makes random selection with a fixed position more efficient) of all questions, positive and negative response and consequences, paired up with a regular expression that represents fuzzy matching – for example, someone typing it check temperature should match Check core temperature? as a question, and return both options. The code back in wait_for_command/0 will only execute the core_temperature/0 function.

Here the two last functions implement the special last requirement: after denying venting too many times, the valve must be disabled manually.

Here we use a dirty ugly counter for prototyping’s sake. In fact I had forgotten about that requirement at the time and just bolted it on that way. The prototype helped figure that requirement out, and the final version can now be designed with this in mind.

The registered entry specifies what processes are going to be globally registered on the node. In this case, none. The applications tuple is a list of all applications we depend on. All applications depend on both kernel and stdlib. These entries have to always be in there. On the other hand, crypto is optional to most apps, but we need it because we use it to seed our pseudo-random number generator in start/0.

The other option considered here is mod. If your library requires no process to be started and you’re just shipping code around, you’re done. In our case however, we’re starting a process (or we want to), and therefore we specify an application module named muumuu_app. This module is also in src/:

That module is basically giving callbacks to the Erlang VM. See it a bit as the main function in C, except you also have to provide a stop function that will clean up once the process exits. In this case we need nothing.

Unfortunately, our process isn’t OTP-compliant. The guys at Ericsson have long ago hit that problem and developed a supervisor bridge, which basically acts as a wrapper. This is what we could use if I were not the kind of person to want my OTP processes done correctly everywhere.

For the time being, I’ll stick with a regular supervisor and will rewrite the FSM right after:

This will start muumuu_fsm as a permanent worker that can die once every 5 seconds before the entire system crashes. I don’t have a good way to pick frequencies, but 1 in 5 seconds sounds like something reasonable for someone to mash keys in ways bad enough it causes errors.

So then comes the rewrite from prototype to gen_fsm. This is stuff that has been covered in multiple tutorials before, so I’m going to skip most of it. You can instead look at books and docs for gen_fsm, follow along the final module, muumuu_fsm.erl, and see for yourself.

The biggest changes there, outside of providing the gen_fsm callbacks required by the OTP behavior, are related to the general information flow. Rather than being really direct sequences of functions doing whatever they want, the OTP version of the module becomes a lot more declarative.

We no longer enter a state function, ask a question, and wait for the response within the same context. The logic has moved so that an event in a state (say first_gas_vent) causes a question to be asked before transitioning to the state that will handle that response.

This doesn’t make the code particulalry harder to read, just different:

This form, along with the experience gained in the prototype, allows for simpler state management via the State variable, which allows us to be more transparent about our usage of venting limits, for example. We also instantly benefit from everything OTP gives us in terms of transparency: tracing, logging, statistics, and so on (see the sys module)

With that code in place, we can compile and run the entire application:

This basically just tells rebar3 what the release-building tool it includes (relx) should do to give us our release. The release will only include our custom Erlang code, and use the currently installed Erlang VM to run things rather than installing a fully self-contianed program. Then the magic happens:

Arguments in there I merged into one. A good practice for any Erlang system is to give it a name, which will let you connect to it while it’s running. In this case I could go in and debug the console as the user is maintaining the powerplant.

The last arguments (-smp disable +A 1) are basically optimizations for this very app: they remove Erlang parallelism (I’m running a single active process for the thing, so why bother?) and removes the number of asynchronous threads for IO to a single one (for the same reason – one active process, why bother?).

In more serious apps, tweaking your VM options can be worthwhile, but outside of this text’s scope.

Cool, everything works. I now have a binary executable I can link to from anywhere in the system and will require no magical arguments to work!

Tests

As much as I like to try and get testing done ahead of time – it’s the only time it’s not super terrible and crappy – I often end up adding it after the fact when I know I’ll have to maintain it.

For this, each app should have its tests, so I’ll have to add a test/ directory in apps/muumuu/.

My tool of choice is Common Test, which while it is kind of full of annoying overheads for unit testing and is mostly useless for shell output (you gotta deal with HTML files), it scales fairly well for integration and system tests.

Note that rather than having a top-level deps entry as we usually would, we define this one to be into the test profile. This will allow the dependency to only be fetched and used when running tests, and to avoid bundling it when shipping the application.

Rebar3 pulls stuff from a package repository for this one (github dependencies are also an option). Rebar3 will add it to a lock file when it fetches and compiles it later.

Mocking the io system is a fun way to basically take it and make it return messages we can look at. That all takes place in mock_io(), and after that’s in place, we start a muumuu instance directly (no application needed):

Ugly. The first step is unstickying the directory for Erlang code. Most modules don’t require that, only those in Erlang’s standard library. Unstickying allows to load new versions of code at run time, which meck dynamically does.

Here what I’m doing is mocking the functions io:format/1, io:format/2 and io:get_line/1 to send messages of the form {in, Msg} and {out, Msg} from input and output, respectively. meck:unload(io) will undo that.

We also had the wait_for_death/1 call. I’m using these everywhere in tests. Timers are the enemy of good concurrent testing, and if you rely on a timer:sleep(1000) of some sort to make sure everything is clean, you’re doing it wrong.

Here the function polls to return ASAP, with a tiny sleep to not heat up your room too much via the CPU:

I basically just write the test the way I want it to look like. I will start expecting messages that will match the regex "press.*any.*key.*>" being output, after which I’ll insert <tab>. Rinse and repeat.

Here, my desire is pretty much to turn the interactions I’d write in the shell into a bunch of function calls and matches.

That’s why I planned having a message-passing interface. I can now write functions to wrap that functionality:

If we look back into the mocked function, the mocked function sends us {in, ProcessThatWaitsForInput}. We take the Input argument, and send it back to the mocked function (which runs in its own process).

If we never receive the in message, we crash, but printing the debugging information. Interestingly here the function I use is ct:pal. It works exactly like io:format, except:

It outputs to both the shell and HTML logs for Common Test

It’s not gonna be used in production systems and it’s surely never going to be mocked (unlike io).

After this, I check in the rebar lock files into version control, and I go do something else because I’m pretty much done. You can see all the code here.

Fred Hebert

Erlang

Author of Learn You Some Erlang For Great Good

Fred is the author of Learn You Some Erlang for Great Good!, a free online (also paid for, on paper) book designed to teach Erlang, as well as Erlang in Anger, a guide about how to be the Erlang medic in a time of war. He’s worked on writing and teaching training course material for Erlang Solutions Ltd, and on Real Time Bidding software for AdGear. He has since then moved to Heroku’s routing team, writing and maintaining large scale distributed systems in the cloud.