erl_ddll

Dynamic Driver Loader and Linker

The erl_ddll module provides an interface for loading
and unloading erlang linked in drivers in runtime.

Note!

This is a large reference document. For casual use of the
module, as well as for most real world applications, the
descriptions of the functions load/2 and unload/1 are enough to get
going.

The driver should be provided as a dynamically linked library
in a object code format specific for the platform in use,
i. e. .so files on most Unix systems and .ddl
files on windows. An erlang linked in driver has to provide
specific interfaces to the emulator, so this module is not
designed for loading arbitrary dynamic libraries. For further
information about erlang drivers, refer to the ERTS reference
manual section erl_driver.

When describing a set of functions, (i.e. a module, a part of a
module or an application) executing in a process and wanting to
use a ddll-driver, we use the term user. There can be
several users in one process (different modules needing the same
driver) and several processes running the same code, making up
several users of a driver. In the basic scenario, each
user loads the driver before starting to use it and unloads the
driver when done. The reference counting keeps track of
processes as well as the number of loads by each process, so that
the driver will only be unloaded when no one wants it
(it has no user). The driver also keeps track of ports that are
opened towards it, so that one can delay unloading until all
ports are closed or kill all ports using the driver when it is
unloaded.

The interface supports two basic scenarios of loading and
unloading. Each scenario can also have the option of either
killing ports when the driver is unloading, or waiting for the
ports to close themselves. The scenarios are:

Load and unload on a "when needed basis"

This (most common) scenario simply supports that each
user of the driver loads
it when it is needed and unloads it when the user no longer have any use for
it. The driver is always reference counted and as long as a
process keeping the driver loaded is still alive, the driver
is present in the system.

Each user of the driver
use literally the same pathname for the driver when
demanding load, but the users are not really concerned
with if the driver is already loaded from the filesystem or
if the object code has to be loaded from filesystem.

Two pairs of functions support this scenario:

load/2 and unload/1

When using the load/unload interfaces, the
driver will not actually get unloaded until the
last port using the driver is closed. The function
unload/1 can return immediately, as the users are not really concerned
with when the actual unloading occurs. The
driver will actually get unloaded when no one needs it any longer.

If a process having the driver loaded dies, it will have
the same effect as if unloading was done.

When loading, the function load/2 returns
ok as soon as there is any instance of the driver
present, so that if a driver is waiting to get unloaded
(due to open ports), it will simply change state to no
longer need unloading.

load_driver/2 and unload_driver/1

These interfaces is intended to be used when it is considered an
error that ports are open towards a driver that no user
has loaded. The ports still open when the
last user calls
unload_driver/1 or when the last process having the
driver loaded dies, will get killed with reason
driver_unloaded.

The function names load_driver and
unload_driver are kept for backward
compatibility.

Loading and reloading for code replacement

This scenario occurs when the driver code might need
replacement during operation of the Erlang
emulator. Implementing driver code replacement is somewhat
more tedious than beam code replacement, as one driver
cannot be loaded as both "old" and "new" code. All users of a driver must have it
closed (no open ports) before the old code can be unloaded
and the new code can be loaded.

The actual unloading/loading is done as one atomic
operation, blocking all processes in the system from using
the driver concerned while in progress.

The preferred way to do driver code replacement is to let
one single process keep track of the driver. When
the process start, the driver is loaded. When replacement
is required, the driver is reloaded. Unload is probably never
done, or done when the process exits. If more than one user has a driver loaded when code
replacement is demanded, the replacement cannot occur until
the last "other" user has
unloaded the driver.

Demanding reload when a reload is already in progress is
always an error. Using the high level functions, it is also
an error to demand reloading when more than one user has the driver loaded. To
simplify driver replacement, avoid designing your system so
that more than than one user has the driver loaded.

The two functions for reloading drivers should be used
together with corresponding load functions, to support the two
different behaviors concerning open ports:

load/2 and reload/2

This pair of functions is used when reloading should be
done after the last open port towards the driver is
closed.

As reload/2 actually waits for the reloading to
occur, a misbehaving process keeping open ports towards
the driver (or keeping the driver loaded) might cause
infinite waiting for reload. Timeouts has to be provided
outside of the process demanding the reload or by using
the low-level interface try_load/3 in combination
with driver monitors (see below).

load_driver/2 and reload_driver/2

This pair of functions are used when open ports towards
the driver should be killed with reason
driver_unloaded to allow for new driver code to
get loaded.

If, however, another process has the driver loaded,
calling reload_driver returns the error code
pending_process. As stated earlier,
the recommended design is to not allow other users than the "driver
reloader" to actually demand loading of the concerned
driver.

info(Name) -> InfoList

Returns a list of tuples {Tag, Value}, where
Tag is the information item and Value is the result
of calling info/2 with this driver name and
this tag. The result being a tuple list containing all
information available about a driver.

load(Path, Name) -> ok | {error, ErrorDesc}

Loads and links the dynamic driver Name. Path
is a file path to the directory containing the driver.
Name must be a sharable object/dynamic library. Two
drivers with different Path parameters cannot be
loaded under the same name. The Name is a string or
atom containing at least one character.

The Name given should correspond to the filename
of the actual dynamically loadable object file residing in
the directory given as Path, but without the
extension (i.e. .so). The driver name provided in
the driver initialization routine must correspond with the
filename, in much the same way as erlang module names
correspond to the names of the .beam files.

If the driver has been previously unloaded, but is still
present due to open ports against it, a call to
load/2 will stop the unloading and keep the driver
(as long as the Path is the same) and ok is
returned. If one actually wants the object code to be
reloaded, one uses reload/2 or the low-level
interface try_load/3
instead. Please refer to the description of different scenarios for
loading/unloading in the introduction.

If more than one process tries to load an already loaded
driver withe the same Path, or if the same process
tries to load it several times, the function will return
ok. The emulator will keep track of the
load/2 calls, so that a corresponding number of
unload/2 calls will have to be done from the same
process before the driver will actually get unloaded. It is
therefore safe for an application to load a driver that is
shared between processes or applications when needed. It can
safely be unloaded without causing trouble for other
parts of the system.

It is not allowed to load
several drivers with the same name but with different
Path parameters.

Note!

Note especially that the Path is interpreted
literally, so that all loaders of the same driver needs to
give the same literalPath string, even
though different paths might point out the same directory
in the filesystem (due to use of relative paths and
links).

On success, the function returns ok. On
failure, the return value is {error,ErrorDesc},
where ErrorDesc is an opaque term to be
translated into human readable form by the format_error/1
function.

For more control over the error handling, again use the
try_load/3
interface instead.

The function throws a badarg exception if the
parameters are not given as described above.

load_driver(Path, Name) -> ok | {error, ErrorDesc}

Works essentially as load/2, but will load the driver
with other options. All ports that are using the
driver will get killed with the reason
driver_unloaded when the driver is to be unloaded.

The number of loads and unloads by different users influence the actual loading
and unloading of a driver file. The port killing will
therefore only happen when the lastuser unloads the driver, or the
last process having loaded the driver exits.

This interface (or at least the name of the functions) is
kept for backward compatibility. Using try_load/3 with
{driver_options,[kill_ports]} in the option list will
give the same effect regarding the port killing.

The function throws a badarg exception if the
parameters are not given as described above.

monitor(Tag, Item) -> MonitorRef

This function creates a driver monitor and works in many
ways as the function erlang:monitor/2,
does for processes. When a driver changes state, the monitor
results in a monitor-message being sent to the calling
process. The MonitorRef returned by this function is
included in the message sent.

As with process monitors, each driver monitor set will only
generate one single message. The monitor is
"destroyed" after the message is sent and there is then no
need to call demonitor/1.

The MonitorRef can also be used in subsequent calls
to demonitor/1 to
remove a monitor.

The function accepts the following parameters:

Tag

The monitor tag is always driver as this function
can only be used to create driver monitors. In the future,
driver monitors will be integrated with process monitors,
why this parameter has to be given for consistence.

Item

The Item parameter specifies which driver one
wants to monitor (the name of the driver) as well as
which state change one wants to monitor. The parameter
is a tuple of arity two whose first element is the
driver name and second element is either of:

loaded

Notify me when the driver is reloaded (or loaded if
loading is underway). It only makes sense to monitor
drivers that are in the process of being loaded or
reloaded. One cannot monitor a future-to-be driver
name for loading, that will only result in a
'DOWN' message being immediately sent.
Monitoring for loading is therefore most useful when
triggered by the try_load/3 function,
where the monitor is created because the
driver is in such a pending state.

Setting a driver monitor for loading will
eventually lead to one of the following messages
being sent:

{'UP', reference(), driver, Name, loaded}

This message is sent, either immediately if the
driver is already loaded and no reloading is
pending, or when reloading is executed if
reloading is pending.

The user is
expected to know if reloading is demanded prior
to creating a monitor for loading.

{'UP', reference(), driver, Name, permanent}

This message will be sent if reloading was
expected, but the (old) driver made itself
permanent prior to reloading. It will also be
sent if the driver was permanent or statically
linked in when trying to create the monitor.

{'DOWN', reference(), driver, Name, load_cancelled}

This message will arrive if reloading was
underway, but the user having requested
reload cancelled it by either dying or calling
try_unload/2
(or unload/1/unload_driver/1)
again before it was reloaded.

{'DOWN', reference(), driver, Name, {load_failure, Failure}}

This message will arrive if reloading was
underway but the loading for some reason
failed. The Failure term is one of the
errors that can be returned from try_load/3. The
error term can be passed to format_error/1
for translation into human readable form. Note
that the translation has to be done in the same
running erlang virtual machine as the error
was detected in.

unloaded

Monitor when a driver gets unloaded. If one
monitors a driver that is not present in the system,
one will immediately get notified that the driver got
unloaded. There is no guarantee that the driver was
actually ever loaded.

A driver monitor for unload will eventually result
in one of the following messages being sent:

{'DOWN', reference(), driver, Name, unloaded}

The driver instance monitored is now
unloaded. As the unload might have been due to a
reload/2 request, the driver might once
again have been loaded when this message
arrives.

{'UP', reference(), driver, Name, unload_cancelled}

This message will be sent if unloading was
expected, but while the driver was waiting for
all ports to get closed, a new user of the driver
appeared and the unloading was cancelled.

This message appears when an {ok, pending_driver}) was returned from try_unload/2)
for the last user of the driver and
then a {ok, already_loaded} is returned
from a call to try_load/3.

If one wants to really monitor when the
driver gets unloaded, this message will distort
the picture, no unloading was really done.
The unloaded_only option creates a monitor
similar to an unloaded monitor, but does
never result in this message.

{'UP', reference(), driver, Name, permanent}

This message will be sent if unloading was
expected, but the driver made itself
permanent prior to unloading. It will also be
sent if trying to monitor a permanent or
statically linked in driver.

unloaded_only

A monitor created as unloaded_only behaves
exactly as one created as unloaded with the
exception that the {'UP', reference(), driver, Name, unload_cancelled} message will never be
sent, but the monitor instead persists until the
driver really gets unloaded.

The function throws a badarg exception if the
parameters are not given as described above.

Note!

On success, the function returns ok. On
failure, the function returns an opaque error, with the
exception of the pending_process error described
above. The opaque errors are to be translated into human
readable form by the format_error/1 function.

For more control over the error handling, again use the
try_load/3
interface instead.

The function throws a badarg exception if the
parameters are not given as described above.

This function provides more control than the
load/2/reload/2 and
load_driver/2/reload_driver/2 interfaces. It
will never wait for completion of other operations related
to the driver, but immediately return the status of the
driver as either:

{ok, loaded}

The driver was actually loaded and is immediately
usable.

{ok, already_loaded}

The driver was already loaded by another process
and/or is in use by a living port. The load by you is
registered and a corresponding try_unload is
expected sometime in the future.

{ok, pending_driver}or {ok, pending_driver, reference()}

The load request is registered, but the loading is
delayed due to the fact that an earlier instance of the
driver is still waiting to get unloaded (there are open
ports using it). Still, unload is expected when you are
done with the driver. This return value will
mostly happen when the
{reload,pending_driver} or
{reload,pending} options are used, but
can happen when another user is unloading a driver in
parallel and the kill_ports driver option is
set. In other words, this return value will always need
to be handled!

{ok, pending_process}or {ok, pending_process, reference()}

The load request is registered, but the loading is
delayed due to the fact that an earlier instance of the
driver is still waiting to get unloaded by another
user (not only by a
port, in which case {ok,pending_driver} would
have been returned). Still, unload is expected when you
are done with the driver. This return value will
only happen when the {reload,pending}
option is used.

When the function returns {ok, pending_driver} or
{ok, pending_process}, one might want to get information
about when the driver is actually loaded. This can
be achieved by using the {monitor, MonitorOption} option.

When monitoring is requested, and a corresponding {ok, pending_driver} or {ok, pending_process} would be
returned, the function will instead return a tuple {ok, PendingStatus, reference()} and the process will, at a later
time when the driver actually gets loaded, get a monitor
message. The monitor message one can expect is described in
the monitor/2
function description.

Note!

Note that in case of loading, monitoring can
not only get triggered by using the {reload, ReloadOption} option, but also in special cases where
the load-error is transient, why {monitor, pending_driver} should be used under basically
all real world circumstances!

The function accepts the following parameters:

Path

The filesystem path to the directory where the driver
object file is situated. The filename of the object file
(minus extension) must correspond to the driver name
(used in the name parameter) and the driver must
identify itself with the very same name. The
Path might be provided as an iolist(),
meaning it can be a list of other iolist()s, characters
(eight bit integers) or binaries, all to be flattened
into a sequence of characters.

The (possibly flattened) Path parameter must be
consistent throughout the system, a driver should, by
all users, be loaded
using the same literalPath. The
exception is when reloading is requested, in
which case the Path may be specified
differently. Note that all users trying to load the
driver at a later time will need to use the newPath if the Path is changed using a
reload option. This is yet another reason
to have only one loader of a driver one wants to
upgrade in a running system!

Name

The name parameter is the name of the driver to be used
in subsequent calls to open_port. The
name can be specified either as an iolist() or
as an atom(). The name given when loading is used
to find the actual object file (with the
help of the Path and the system implied
extension suffix, i.e. .so). The name by which
the driver identifies itself must also be consistent
with this Name parameter, much as a beam-file's
module name much correspond to its filename.

OptionList

A number of options can be specified to control the
loading operation. The options are given as a list of
two-tuples, the tuples having the following values and
meanings:

{driver_options, DriverOptionList}

This option is to provide options that will change
its general behavior and will "stick" to the driver
throughout its lifespan.

The driver options for a given driver name need
always to be consistent, even when the driver is reloaded, meaning that they are as much a part
of the driver as the actual name.

Currently the only allowed driver option is
kill_ports, which means that all ports opened
towards the driver are killed with the exit-reason
driver_unloaded when no process any longer
has the driver loaded. This situation arises either
when the last user calls try_unload/2, or
the last process having loaded the driver exits.

{monitor, MonitorOption}

A MonitorOption tells try_load/3 to
trigger a driver monitor under certain
conditions. When the monitor is triggered, the
function will return a three-tuple {ok, PendingStatus, reference()}, where the reference() is
the monitor ref for the driver monitor.

Only one MonitorOption can be specified and
it is either the atom pending, which means
that a monitor should be created whenever a load
operation is delayed, and the atom
pending_driver, in which a monitor is
created whenever the operation is delayed due to
open ports towards an otherwise unused driver. The
pending_driver option is of little use, but
is present for completeness, it is very well defined
which reload-options might give rise to which
delays. It might, however, be a good idea to use the
same MonitorOption as the ReloadOption
if present.

If reloading is not requested, it might still be
useful to specify the monitor option, as
forced unloads (kill_ports driver option or
the kill_ports option to try_unload/2) will
trigger a transient state where driver loading
cannot be performed until all closing ports are
actually closed. So, as try_unload can, in
almost all situations, return {ok, pending_driver}, one should always specify at least
{monitor, pending_driver} in production
code (see the monitor discussion above).

{reload,ReloadOption}

This option is used when one wants to
reload a driver from disk, most often in a
code upgrade scenario. Having a reload option
also implies that the Path parameter need
not be consistent with earlier loads of
the driver.

To reload a driver, the process needs to have previously
loaded the driver, i.e there has to be an active user of the driver in the process.

The reload option can be either the atom
pending, in which reloading is requested for
any driver and will be effectuated when all
ports opened against the driver are closed. The
replacement of the driver will in this case take
place regardless of if there are still
pending users
having the driver loaded!
The option also triggers port-killing (if the
kill_ports driver option is used) even though
there are pending users, making it usable for forced
driver replacement, but laying a lot of
responsibility on the driver users. The pending option is
seldom used as one does not want other users to have loaded the
driver when code change is underway.

The more useful option is pending_driver,
which means that reloading will be queued if the
driver is not loaded by any other users, but the driver has
opened ports, in which case {ok, pending_driver} will be returned (a
monitor option is of course recommended).

If the driver is unloaded (not present in the
system), the error code
not_loaded will be returned. The
reload option is intended for when the user
has already loaded the driver in advance.

The function might return numerous errors, of which some
only can be returned given a certain combination of options.

A number of errors are opaque and can only be interpreted by
passing them to the format_error/1 function,
but some can be interpreted directly:

{error,linked_in_driver}

The driver with the specified name is an erlang
statically linked in driver, which cannot be manipulated
with this API.

{error,inconsistent}

The driver has already been loaded with either other
DriverOptionList or a different literalPath argument.

This can happen even if a reload option is given,
if the DriverOptionList differ from the current.

{error, permanent}

The driver has requested itself to be permanent, making
it behave like an erlang linked in driver and it can no
longer be manipulated with this API.

{error, pending_process}

The driver is loaded by other users when the {reload, pending_driver} option was given.

{error, pending_reload}

Driver reload is already requested by another user when the {reload, ReloadOption} option was given.

{error, not_loaded_by_this_process}

Appears when the reload option is given. The
driver Name is present in the system, but there is no
user of it in this
process.

{error, not_loaded}

Appears when the reload option is given. The
driver Name is not in the system. Only drivers
loaded by this process can be reloaded.

All other error codes are to be translated by the format_error/1
function. Note that calls to format_error should be
performed from the same running instance of the erlang
virtual machine as the error was detected in, due to system
dependent behavior concerning error values.

If the arguments or options are malformed, the function will
throw a badarg exception.

This is the low level function to unload (or decrement
reference counts of) a driver. It can be used to force port
killing, in much the same way as the driver option
kill_ports implicitly does, and it can trigger a
monitor either due to other users still having the driver
loaded or that there are open ports using the driver.

Unloading can be described as the process of telling the
emulator that this particular part of the code in this
particular process (i.e. this user) no longer needs the
driver. That can, if there are no other users, trigger
actual unloading of the driver, in which case the driver
name disappears from the system and (if possible) the memory
occupied by the driver executable code is reclaimed. If the
driver has the kill_ports option set, or if
kill_ports was specified as an option to this
function, all pending ports using this driver will get
killed when unloading is done by the last user. If no port-killing is
involved and there are open ports, the actual unloading
is delayed until there are no more open ports using the
driver. If, in this case, another user (or even this user) loads the
driver again before the driver is actually unloaded, the
unloading will never take place.

To allow the user that
requests unloading to wait for actual unloading to
take place, monitor triggers can be specified in much
the same way as when loading. As users of this function however
seldom are interested in more than decrementing the
reference counts, monitoring is more seldom needed. If the
kill_ports option is used however, monitor trigging is
crucial, as the ports are not guaranteed to have been killed
until the driver is unloaded, why a monitor should be
triggered for at least the pending_driver case.

The possible monitor messages that can be expected are the
same as when using the unloaded option to the
monitor/2 function.

The function will return one of the following statuses upon
success:

{ok, unloaded}

The driver was immediately unloaded, meaning that the
driver name is now free to use by other drivers and, if
the underlying OS permits it, the memory occupied by the
driver object code is now reclaimed.

The driver can only be unloaded when there are no open
ports using it and there are no more users requiring it to be
loaded.

{ok, pending_driver}or {ok, pending_driver, reference()}

This return value indicates that this call removed the
last user from the
driver, but there are still open ports using it.
When all ports are closed and no new users have arrived, the driver
will actually be reloaded and the name and memory
reclaimed.

This return value is valid even when the option
kill_ports was used, as killing ports may not be
a process that completes immediately. The condition is,
in that case, however transient. Monitors are as always
useful to detect when the driver is really unloaded.

{ok, pending_process}or {ok, pending_process, reference()}

The unload request is registered, but there are still
other users holding
the driver. Note that the term pending_process
might refer to the running process, there might be more
than one user in the
same process.

This is a normal, healthy return value if the call was
just placed to inform the emulator that you have no
further use of the driver. It is actually the most
common return value in the most common scenario
described in the introduction.

The function accepts the following parameters:

Name

The name parameter is the name of the driver to be
unloaded. The name can be specified either as an
iolist() or as an atom().

OptionList

The OptionList argument can be used to specify
certain behavior regarding ports as well as triggering
monitors under certain conditions:

kill_ports

Force killing of all ports opened using this driver,
with the exit reason driver_unloaded, if you are
the lastuser of the driver.

If there are other users having the driver
loaded, this option will have no effect.

If one wants the consistent behavior of killing ports
when the last user
unloads, one should use the driver option
kill_ports when loading the driver instead.

{monitor, MonitorOption}

This option creates a driver monitor if the condition
given in MonitorOption is true. The valid
options are:

pending_driver

Create a driver monitor if the return value is to
be {ok, pending_driver}.

pending

Create a monitor if the return value will be either
{ok, pending_driver} or {ok, pending_process}.

The pending_driverMonitorOption is by far
the most useful and it has to be used to ensure that the
driver has really been unloaded and the ports closed
whenever the kill_ports option is used or the
driver may have been loaded with the kill_ports
driver option.

By using the monitor-triggers in the call to
try_unload one can be sure that the monitor is
actually added before the unloading is executed, meaning
that the monitor will always get properly triggered,
which would not be the case if one called
erl_ddll:monitor/2 separately.

The function may return several error conditions, of which
all are well specified (no opaque values):

{error, linked_in_driver}

You were trying to unload an erlang statically linked in
driver, which cannot be manipulated with this interface
(and cannot be unloaded at all).

{error, not_loaded}

The driver Name is not present in the system.

{error, not_loaded_by_this_process}

The driver Name is present in the system, but
there is no user of
it in this process.

As a special case, drivers can be unloaded from
processes that has done no corresponding call to
try_load/3 if, and only if, there are no users of the driver at all, which may happen if the
process containing the last user dies.

{error, permanent}

The driver has made itself permanent, in which case it
can no longer be manipulated by this interface (much
like a statically linked in driver).

The function throws a badarg exception if the
parameters are not given as described above.

unload(Name) -> ok | {error, ErrorDesc}

Unloads, or at least dereferences the driver named
Name. If the caller is the last user of the driver, and there
are no more open ports using the driver, the driver will
actually get unloaded. In all other cases, actual unloading
will be delayed until all ports are closed and there are no
remaining users.

If there are other users of the driver, the reference
counts of the driver is merely decreased, so that the caller
is no longer considered a user of the driver. For usage
scenarios, see the description in the beginning
of this document.

The ErrorDesc returned is an opaque value to be
passed further on to the format_error/1
function. For more control over the operation, use the
try_unload/2
interface.

The function throws a badarg exception if the
parameters are not given as described above.

unload_driver(Name) -> ok | {error, ErrorDesc}

Unloads, or at least dereferences the driver named
Name. If the caller is the last user of the driver, all
remaining open ports using the driver will get killed with
the reason driver_unloaded and the driver will
eventually get unloaded.

If there are other users
of the driver, the reference counts of the driver is merely
decreased, so that the caller is no longer considered a
user. For
usage scenarios, see the description in the beginning
of this document.

The ErrorDesc returned is an opaque value to be
passed further on to the format_error/1
function. For more control over the operation, use the
try_unload/2
interface.

The function throws a badarg exception if the
parameters are not given as described above.

loaded_drivers() -> {ok, Drivers}

Drivers = [Driver]

Driver = string()

Returns a list of all the available drivers, both
(statically) linked-in and dynamically loaded ones.

The driver names are returned as a list of strings rather
than a list of atoms for historical reasons.

More information about drivers can be obtained using one of
the info functions.

format_error(ErrorDesc) -> string()

ErrorDesc = term()

Takes an ErrorDesc returned by load, unload or
reload functions and returns a string which
describes the error or warning.

Note!

Due to peculiarities in the dynamic loading interfaces on
different platform, the returned string is only guaranteed
to describe the correct error if format_error/1 is called in the same instance of the erlang virtual machine as the error appeared in (meaning the same operating
system process)!