DESCRIPTION

These functions create, manipulate, and use cryptographic modules in the
form of ENGINE objects. These objects act as containers for
implementations of cryptographic algorithms, and support a
reference-counted mechanism to allow them to be dynamically loaded in and
out of the running application.

The cryptographic functionality that can be provided by an ENGINE
implementation includes the following abstractions;

Reference counting and handles

Due to the modular nature of the ENGINEAPI, pointers to ENGINEs need to be
treated as handles - ie. not only as pointers, but also as references to
the underlying ENGINE object. Ie. you should obtain a new reference when
making copies of an ENGINE pointer if the copies will be used (and
released) independantly.

ENGINE objects have two levels of reference-counting to match the way in
which the objects are used. At the most basic level, each ENGINE pointer is
inherently a structural reference - you need a structural reference
simply to refer to the pointer value at all, as this kind of reference is
your guarantee that the structure can not be deallocated until you release
your reference.

However, a structural reference provides no guarantee that the ENGINE has
been initiliased to be usable to perform any of its cryptographic
implementations - and indeed it's quite possible that most ENGINEs will not
initialised at all on standard setups, as ENGINEs are typically used to
support specialised hardware. To use an ENGINE's functionality, you need a
functional reference. This kind of reference can be considered a
specialised form of structural reference, because each functional reference
implicitly contains a structural reference as well - however to avoid
difficult-to-find programming bugs, it is recommended to treat the two
kinds of reference independantly. If you have a functional reference to an
ENGINE, you have a guarantee that the ENGINE has been initialised ready to
perform cryptographic operations and will not be uninitialised or cleaned
up until after you have released your reference.

We will discuss the two kinds of reference separately, including how to
tell which one you are dealing with at any given point in time (after all
they are both simply (ENGINE *) pointers, the difference is in the way they
are used).

Structural references

This basic type of reference is typically used for creating new ENGINEs
dynamically, iterating across OpenSSL's internal linked-list of loaded
ENGINEs, reading information about an ENGINE, etc. Essentially a structural
reference is sufficient if you only need to query or manipulate the data of
an ENGINE implementation rather than use its functionality.

The ENGINE_new() function returns a structural reference to a new (empty)
ENGINE object. Other than that, structural references come from return
values to various ENGINEAPI functions such as; ENGINE_by_id(),
ENGINE_get_first(), ENGINE_get_last(), ENGINE_get_next(),
ENGINE_get_prev(). All structural references should be released by a
corresponding to call to the ENGINE_free() function - the ENGINE object
itself will only actually be cleaned up and deallocated when the last
structural reference is released.

It should also be noted that many ENGINEAPI function calls that accept a
structural reference will internally obtain another reference - typically
this happens whenever the supplied ENGINE will be needed by OpenSSL after
the function has returned. Eg. the function to add a new ENGINE to
OpenSSL's internal list is ENGINE_add() - if this function returns success,
then OpenSSL will have stored a new structural reference internally so the
caller is still responsible for freeing their own reference with
ENGINE_free() when they are finished with it. In a similar way, some
functions will automatically release the structural reference passed to it
if part of the function's job is to do so. Eg. the ENGINE_get_next() and
ENGINE_get_prev() functions are used for iterating across the internal
ENGINE list - they will return a new structural reference to the next (or
previous) ENGINE in the list or NULL if at the end (or beginning) of the
list, but in either case the structural reference passed to the function is
released on behalf of the caller.

To clarify a particular function's handling of references, one should
always consult that function's documentation ``man'' page, or failing that
the openssl/engine.h header file includes some hints.

Functional references

As mentioned, functional references exist when the cryptographic
functionality of an ENGINE is required to be available. A functional
reference can be obtained in one of two ways; from an existing structural
reference to the required ENGINE, or by asking OpenSSL for the default
operational ENGINE for a given cryptographic purpose.

To obtain a functional reference from an existing structural reference,
call the ENGINE_init() function. This returns zero if the ENGINE was not
already operational and couldn't be successfully initialised (eg. lack of
system drivers, no special hardware attached, etc), otherwise it will
return non-zero to indicate that the ENGINE is now operational and will
have allocated a new functional reference to the ENGINE. In this case,
the supplied ENGINE pointer is, from the point of the view of the caller,
both a structural reference and a functional reference - so if the caller
intends to use it as a functional reference it should free the structural
reference with ENGINE_free() first. If the caller wishes to use it only as
a structural reference (eg. if the ENGINE_init() call was simply to test if
the ENGINE seems available/online), then it should free the functional
reference; all functional references are released by the ENGINE_finish()
function.

The second way to get a functional reference is by asking OpenSSL for a
default implementation for a given task, eg. by ENGINE_get_default_RSA(),
ENGINE_get_default_cipher_engine(), etc. These are discussed in the next
section, though they are not usually required by application programmers as
they are used automatically when creating and using the relevant
algorithm-specific types in OpenSSL, such as RSA, DSA, EVP_CIPHER_CTX, etc.

Default implementations

For each supported abstraction, the ENGINE code maintains an internal table
of state to control which implementations are available for a given
abstraction and which should be used by default. These implementations are
registered in the tables separated-out by an 'nid' index, because
abstractions like EVP_CIPHER and EVP_DIGEST support many distinct
algorithms and modes - ENGINEs will support different numbers and
combinations of these. In the case of other abstractions like RSA, DSA,
etc, there is only one ``algorithm'' so all implementations implicitly
register using the same 'nid' index. ENGINEs can be registered into
these tables to make themselves available for use automatically by the
various abstractions, eg. RSA. For illustrative purposes, we continue with
the RSA example, though all comments apply similarly to the other
abstractions (they each get their own table and linkage to the
corresponding section of openssl code).

When a new RSA key is being created, ie. in RSA_new_method(), a
``get_default'' call will be made to the ENGINE subsystem to process the RSA
state table and return a functional reference to an initialised ENGINE
whose RSA_METHOD should be used. If no ENGINE should (or can) be used, it
will return NULL and the RSA key will operate with a NULLENGINE handle by
using the conventional RSA implementation in OpenSSL (and will from then on
behave the way it used to before the ENGINEAPI existed - for details see
RSA_new_method(3)).

Each state table has a flag to note whether it has processed this
``get_default'' query since the table was last modified, because to process
this question it must iterate across all the registered ENGINEs in the
table trying to initialise each of them in turn, in case one of them is
operational. If it returns a functional reference to an ENGINE, it will
also cache another reference to speed up processing future queries (without
needing to iterate across the table). Likewise, it will cache a NULL
response if no ENGINE was available so that future queries won't repeat the
same iteration unless the state table changes. This behaviour can also be
changed; if the ENGINE_TABLE_FLAG_NOINIT flag is set (using
ENGINE_set_table_flags()), no attempted initialisations will take place,
instead the only way for the state table to return a non-NULL ENGINE to the
``get_default'' query will be if one is expressly set in the table. Eg.
ENGINE_set_default_RSA() does the same job as ENGINE_register_RSA() except
that it also sets the state table's cached response for the ``get_default''
query.

In the case of abstractions like EVP_CIPHER, where implementations are
indexed by 'nid', these flags and cached-responses are distinct for each
'nid' value.

It is worth illustrating the difference between ``registration'' of ENGINEs
into these per-algorithm state tables and using the alternative
``set_default'' functions. The latter handles both ``registration'' and also
setting the cached ``default'' ENGINE in each relevant state table - so
registered ENGINEs will only have a chance to be initialised for use as a
default if a default ENGINE wasn't already set for the same state table.
Eg. if ENGINE X supports cipher nids {A,B} and RSA, ENGINE Y supports
ciphers {A} and DSA, and the following code is executed;

Application requirements

This section will explain the basic things an application programmer should
support to make the most useful elements of the ENGINE functionality
available to the user. The first thing to consider is whether the
programmer wishes to make alternative ENGINE modules available to the
application and user. OpenSSL maintains an internal linked list of
``visible'' ENGINEs from which it has to operate - at start-up, this list is
empty and in fact if an application does not call any ENGINEAPI calls and
it uses static linking against openssl, then the resulting application
binary will not contain any alternative ENGINE code at all. So the first
consideration is whether any/all available ENGINE implementations should be
made visible to OpenSSL - this is controlled by calling the various ``load''
functions, eg.

/* Make the "dynamic" ENGINE available */
void ENGINE_load_dynamic(void);
/* Make the CryptoSwift hardware acceleration support available */
void ENGINE_load_cswift(void);
/* Make support for nCipher's "CHIL" hardware available */
void ENGINE_load_chil(void);
...
/* Make ALL ENGINE implementations bundled with OpenSSL available */
void ENGINE_load_builtin_engines(void);

Having called any of these functions, ENGINE objects would have been
dynamically allocated and populated with these implementations and linked
into OpenSSL's internal linked list. At this point it is important to
mention an important API function;

void ENGINE_cleanup(void);

If no ENGINEAPI functions are called at all in an application, then there
are no inherent memory leaks to worry about from the ENGINE functionality,
however if any ENGINEs are ``load''ed, even if they are never registered or
used, it is necessary to use the ENGINE_cleanup() function to
correspondingly cleanup before program exit, if the caller wishes to avoid
memory leaks. This mechanism uses an internal callback registration table
so that any ENGINEAPI functionality that knows it requires cleanup can
register its cleanup details to be called during ENGINE_cleanup(). This
approach allows ENGINE_cleanup() to clean up after any ENGINE functionality
at all that your program uses, yet doesn't automatically create linker
dependencies to all possible ENGINE functionality - only the cleanup
callbacks required by the functionality you do use will be required by the
linker.

The fact that ENGINEs are made visible to OpenSSL (and thus are linked into
the program and loaded into memory at run-time) does not mean they are
``registered'' or called into use by OpenSSL automatically - that behaviour
is something for the application to have control over. Some applications
will want to allow the user to specify exactly which ENGINE they want used
if any is to be used at all. Others may prefer to load all support and have
OpenSSL automatically use at run-time any ENGINE that is able to
successfully initialise - ie. to assume that this corresponds to
acceleration hardware attached to the machine or some such thing. There are
probably numerous other ways in which applications may prefer to handle
things, so we will simply illustrate the consequences as they apply to a
couple of simple cases and leave developers to consider these and the
source code to openssl's builtin utilities as guides.

Using a specific ENGINE implementation

Here we'll assume an application has been configured by its user or admin
to want to use the ``ACME'' ENGINE if it is available in the version of
OpenSSL the application was compiled with. If it is available, it should be
used by default for all RSA, DSA, and symmetric cipher operation, otherwise
OpenSSL should use its builtin software as per usual. The following code
illustrates how to approach this;

Here we'll assume we want to load and register all ENGINE implementations
bundled with OpenSSL, such that for any cryptographic algorithm required by
OpenSSL - if there is an ENGINE that implements it and can be initialise,
it should be used. The following code illustrates how this can work;

/* Load all bundled ENGINEs into memory and make them visible */
ENGINE_load_builtin_engines();
/* Register all of them for every algorithm they collectively implement */
ENGINE_register_all_complete();

That's all that's required. Eg. the next time OpenSSL tries to set up an
RSA key, any bundled ENGINEs that implement RSA_METHOD will be passed to
ENGINE_init() and if any of those succeed, that ENGINE will be set as the
default for use with RSA from then on.

Advanced configuration support

There is a mechanism supported by the ENGINE framework that allows each
ENGINE implementation to define an arbitrary set of configuration
``commands'' and expose them to OpenSSL and any applications based on
OpenSSL. This mechanism is entirely based on the use of name-value pairs
and and assumes ASCII input (no unicode or UTF for now!), so it is ideal if
applications want to provide a transparent way for users to provide
arbitrary configuration ``directives'' directly to such ENGINEs. It is also
possible for the application to dynamically interrogate the loaded ENGINE
implementations for the names, descriptions, and input flags of their
available ``control commands'', providing a more flexible configuration
scheme. However, if the user is expected to know which ENGINE device he/she
is using (in the case of specialised hardware, this goes without saying)
then applications may not need to concern themselves with discovering the
supported control commands and simply prefer to allow settings to passed
into ENGINEs exactly as they are provided by the user.

Before illustrating how control commands work, it is worth mentioning what
they are typically used for. Broadly speaking there are two uses for
control commands; the first is to provide the necessary details to the
implementation (which may know nothing at all specific to the host system)
so that it can be initialised for use. This could include the path to any
driver or config files it needs to load, required network addresses,
smart-card identifiers, passwords to initialise password-protected devices,
logging information, etc etc. This class of commands typically needs to be
passed to an ENGINEbefore attempting to initialise it, ie. before
calling ENGINE_init(). The other class of commands consist of settings or
operations that tweak certain behaviour or cause certain operations to take
place, and these commands may work either before or after ENGINE_init(), or
in same cases both. ENGINE implementations should provide indications of
this in the descriptions attached to builtin control commands and/or in
external product documentation.

Issuing control commands to an ENGINE

Let's illustrate by example; a function for which the caller supplies the
name of the ENGINE it wishes to use, a table of string-pairs for use before
initialisation, and another table for use after initialisation. Note that
the string-pairs used for control commands consist of a command ``name''
followed by the command ``parameter'' - the parameter could be NULL in some
cases but the name can not. This function should initialise the ENGINE
(issuing the ``pre'' commands beforehand and the ``post'' commands afterwards)
and set it as the default for everything except RAND and then return a
boolean success or failure.

Note that ENGINE_ctrl_cmd_string() accepts a boolean argument that can
relax the semantics of the function - if set non-zero it will only return
failure if the ENGINE supported the given command name but failed while
executing it, if the ENGINE doesn't support the command name it will simply
return success without doing anything. In this case we assume the user is
only supplying commands specific to the given ENGINE so we set this to
FALSE.

Discovering supported control commands

It is possible to discover at run-time the names, numerical-ids, descriptions
and input parameters of the control commands supported from a structural
reference to any ENGINE. It is first important to note that some control
commands are defined by OpenSSL itself and it will intercept and handle these
control commands on behalf of the ENGINE, ie. the ENGINE's ctrl() handler is not
used for the control command. openssl/engine.h defines a symbol,
ENGINE_CMD_BASE, that all control commands implemented by ENGINEs from. Any
command value lower than this symbol is considered a ``generic'' command is
handled directly by the OpenSSL core routines.

It is using these ``core'' control commands that one can discover the the control
commands implemented by a given ENGINE, specifically the commands;

Whilst these commands are automatically processed by the OpenSSL framework code,
they use various properties exposed by each ENGINE by which to process these
queries. An ENGINE has 3 properties it exposes that can affect this behaviour;
it can supply a ctrl() handler, it can specify ENGINE_FLAGS_MANUAL_CMD_CTRL in
the ENGINE's flags, and it can expose an array of control command descriptions.
If an ENGINE specifies the ENGINE_FLAGS_MANUAL_CMD_CTRL flag, then it will
simply pass all these ``core'' control commands directly to the ENGINE's ctrl()
handler (and thus, it must have supplied one), so it is up to the ENGINE to
reply to these ``discovery'' commands itself. If that flag is not set, then the
OpenSSL framework code will work with the following rules;

if no ctrl() handler supplied;
ENGINE_HAS_CTRL_FUNCTION returns FALSE (zero),
all other commands fail.
if a ctrl() handler was supplied but no array of control commands;
ENGINE_HAS_CTRL_FUNCTION returns TRUE,
all other commands fail.
if a ctrl() handler and array of control commands was supplied;
ENGINE_HAS_CTRL_FUNCTION returns TRUE,
all other commands proceed processing ...

If the ENGINE's array of control commands is empty then all other commands will
fail, otherwise; ENGINE_CTRL_GET_FIRST_CMD_TYPE returns the identifier of
the first command supported by the ENGINE, ENGINE_GET_NEXT_CMD_TYPE takes the
identifier of a command supported by the ENGINE and returns the next command
identifier or fails if there are no more, ENGINE_CMD_FROM_NAME takes a string
name for a command and returns the corresponding identifier or fails if no such
command name exists, and the remaining commands take a command identifier and
return properties of the corresponding commands. All except
ENGINE_CTRL_GET_FLAGS return the string length of a command name or description,
or populate a supplied character buffer with a copy of the command name or
description. ENGINE_CTRL_GET_FLAGS returns a bitwise-OR'd mask of the following
possible values;

If the ENGINE_CMD_FLAG_INTERNAL flag is set, then any other flags are purely
informational to the caller - this flag will prevent the command being usable
for any higher-level ENGINE functions such as ENGINE_ctrl_cmd_string().
``INTERNAL'' commands are not intended to be exposed to text-based configuration
by applications, administrations, users, etc. These can support arbitrary
operations via ENGINE_ctrl(), including passing to and/or from the control
commands data of any arbitrary type. These commands are supported in the
discovery mechanisms simply to allow applications determinie if an ENGINE
supports certain specific commands it might want to use (eg. application ``foo''
might query various ENGINEs to see if they implement ``FOO_GET_VENDOR_LOGO_GIF'' -
and ENGINE could therefore decide whether or not to support this ``foo''-specific
extension).

Future developments

The ENGINEAPI and internal architecture is currently being reviewed. Slated for
possible release in 0.9.8 is support for transparent loading of ``dynamic''
ENGINEs (built as self-contained shared-libraries). This would allow ENGINE
implementations to be provided independantly of OpenSSL libraries and/or
OpenSSL-based applications, and would also remove any requirement for
applications to explicitly use the ``dynamic'' ENGINE to bind to shared-library
implementations.