∴ikura

OTP releases by hand

Although rebar3, reltool, & relx are here to
save us pain when it comes to building OTP
releases,
after reading up on the task,
it became crystal clear that it’s important to
know the lay of the land when it comes to
shipping our code. After all, OTP
itself is a manifestation of the release
ecosystem; it’s probably best that Erlang
hackers have a decent grasp of the layout
& working parts of this corner of OTP.
And what a better way to understand
this than by creating some releases
by hand for a change?

Our application

We will begin this journey by creating an
application called ‘cache.’ Instead of
using a storage technique like ‘ets’ or
‘mnesia’ to hold key/values, we don’t care if
our data is lost so we will hold the values
in the loop-data of dynamically generated
‘gen_server’ processes. A mapping of each
process to a key will be stored using the
global registry, and the value will be stored
in each server’s loop state.

If all went well, our application should show up
in the list of applications from what’s returned in
line two.

The cache server

The purpose of our cache is to hold values in
a manner that isn’t mission critical, but just so
we can get and put values into something for the
sake of this tutorial. Again, we decided to use the
state of a ‘gen_server’ for value storage, and
use the global name registry to map our dynamic
‘gen_server’ ‘pids’ to the many keys we create.

To start, we want to have our ‘gen_server’ spit out
its state (the cached value for that particular key)
when asked for it. We change ‘cache_srv.erl’ in the
following way:

And of course, we need to freshen the actual application resource
file in ‘ebin’ too:

cat src/cache.app.src > ebin/cache.app

If we open a new Unix shell/tab and head to our cache project
root, we don’t have to stop the Erlang application. We can
load our new code and test it. Running the following will
compile the new code:

erlc -o ebin src/*.erl

And in our running Erlang shell, we can load what’s new
without bringing our service down by performing the following:

3> l(cache_srv), l(cache_lib).

Testing our complete OTP application via the Erlang shell
will suffice for now (making sure it behaves the way we want).
Using our new API, we are able to get
and put key/value pairs with ease. For example:

All went well with our back-of-the-envelope tests. If you
hadn’t noticed, in the ‘cache_lib.erl’ source, we designed
it so keys must be bitstrings. If you were to attempt to
have your key as anything else, you’d be met with an exception.

Building a release from hand

Creating a release is all about minding your
‘Ps’ & ‘Qs.’ It doesn’t take too long to do it
all manually, and there’s oodles to be gained by
doing so.

So, dismissing all the automation tools out there
for the time being, let’s turn our cache application
into a release!

Step one

First off, we need to create a release resource file.

Creating a release resource file is not unlike
how we create an application resource file
(cache.app). Its contents are just an Erlang
term that provides information to the build
system. However, we must gather a few details
in order to know what to place in our new
‘.rel’ file.

Do you still have your Erlang shell still going?
Let’s poke and prod in there & take notes all the while.
First, run the following in the Erlang shell:

10> application:which_applications().

From that, note the version numbers of ‘kernel,’ ‘stdlib,’
and our new ‘cache’ application (the version is the
last element in each tuple).

Also, we need to know the version of the OTP runtime
system we want to bundle with our release. The Erlang
runtime system version can be discovered in the following way:

11> erlang:system_info(version).

Finally, we want to include the ‘sasl’ application with
our release (it includes some useful tools). You can
discover the version to include in many ways, but here’s
one method:

We should have everything we need to create our release
resource file. Just to be clear, we will name our
release ‘cacheapp’ so not to confuse the ‘cache’
application for the sake of this tutorial. It will need to
sit in our project root; this is what the file looks like for
me (yours will certainly vary slightly) :

It looks like our application works with our generated boot file. It’s
time to move on!

Step three

Believe it or not, our release is almost complete, and the experience
hasn’t been that grueling thus far. Of course, along the way, we have
picked the most vanilla of approaches. Almost every step can be
embellished with further options & configurations. All one has to
do is read the apt manuals.

That said, it’s time to create a release package. In this step, we
mostly wear our ‘sys-admin’ hat and move things around via the Unix
shell. Follow along with some care:

For those who have built releases with, say, rebar3, or relx, when
we ‘guntar’ in the last line above, we should be in familiar waters.
At the minimum, acknowledge all your hard work for a moment by
perusing what files were just unpackaged. Do this as follows:

find . | less

Behold: our complete embedded OTP system. But wait! Because we
did this all by hand, sadly, we are missing a few helpful things
build tools usually supply. So, let’s take care of that in a final
step.

Step four

We want to be able to start our system in some canonical way.
Follow along with some care & we will achieve just that in
the last stretch of things.

Working all the time in our ‘_rel/cacheapp’ directory, we do the
following:

Lastly, we need to specify both the ‘erts’ version and the
‘cacheapp’ version in this one place. Perform the following,
being sure to use your version of ‘erts,’ not mine:

echo '6.4 1986.a' > releases/start_erl.data

Done. We have made it though the weeds in four somewhat brief
steps. Congratulations! We have an embedded target system
ready to be used & abused.

Trying things out

In our targeted directory — ‘_rel/cacheapp’ — let’s go
ahead and test our cache. We can start the system &
attach to it as follows:

./bin/start
./bin/to_erl /tmp/

N.B. The argument in the second line above requires
a trailing slash.

You should be attached to the system now & can detach with
CTRL-D. Go ahead and try a few caching ‘gets’ & ‘puts’ before
stopping the system with init:stop().

Wrapping up

So, there you have it. With an OTP application in hand,
in four strides, we can make an embedded target system with
not too much pain.

It’s never regretful to go deep once in a while & learn things
from first principles. Moreover, I hope this tutorial gave some
perspective on how the building blocks of OTP permeates into
our own systems.

Update: Nov. 2016

Even a toy system will probably want some emulator flags
specified. Perhaps not obvious, but ‘bin/start_erl’ is
the place we can sneak in whatever we want. For example,
the following change now accommodates a short-name for the
node: