The American Fuzzy Lop fuzzer has become a critical
tool for finding security vulnerabilities in all sorts of software. It has the ability to send fuzzing
data through programs on the order of hundreds of millions of executions per day on a capable system,
and can certainly put strain on your hardware and OS. If you are fuzzing a target program with the AFL
mode where a file is written and the target binary reads from this file, then AFL is going to conduct
a huge number of writes to the local disk. For a solid-state drive this can reduce its life expectancy
because of write amplification.

My main workstation is a Mac running OS X Yosemite, and I run a lot of Linux, FreeBSD, and OpenBSD
virtual machines under Parallels for development purposes. The drive on this system is an SSD which
keeps everything quite fast, but I don't want to prematurely shorten its life through huge AFL fuzzing
runs. Normally, I'm using AFL to fuzz fwknop from an Ubuntu-14.10 VM, and what is
needed is a way to keep disk writes down. The solution is to use a RAM disk from within the VM.

First, from the Ubuntu VM, let's get set up for AFL fuzzing and show what the disk writes look like
without using a RAM disk from the perspective of the host OS X system. This assumes AFL 0.89 has been
installed already and is in the current path:

We're not running AFL yet. Now, from the Mac, launch the Activity Monitor (under Applications
> Utilities) and look at current disk utilization:
So, not terrible - currently 31 writes per second at the time the snapshot was taken, and that includes
OS X itself and the VM at the same time. But, now let's fire up AFL using the digest cache wrapper on
the Ubuntu VM (the main AFL text UI is not shown for brevity):

And now let's take a look at disk writes again from OS X:
Whoa, that's a massive difference - nearly two orders of magnitude. AFL has caused disk writes to spike
to over 2,700 per second with total data written averaging at 19.5MB/sec. Long term fuzzing at this
level of disk writes would clearly present a problem for the SSD - AFL frequently needs to be left running
for days on end in order to be thorough. So, let's switch everything over to use a RAM disk on the Ubuntu
VM instead and see if that reduces disk writes:

Here is disk utilization once again from the Mac:
We're back to less than 10 writes per second to the SSD even though AFL is going strong on the Ubuntu
VM (not shown). The writes for the previous fuzzing run are still shown to the left of the graph (since
they haven't quite aged out yet when the screenshot was taken), and new writes are so low they don't
even make it above the X-axis at this scale. Although the total execs per second - about 2,000 - achieved
by AFL is not appreciably faster under the RAM disk, the main benefit is that my SSD will last a lot
longer. For those that don't run AFL underneath a VM, a similar strategy should still apply on the main
OS. Assuming enough RAM is available for whatever software you want to fuzz, just create a RAM disk and
run everything from it and extend the life of your hard drive in the process.

Over the past few months, the American Fuzzy Lop
(AFL) fuzzer written by Michal Zalewski has become a tour de force in the security field. Many interesting bugs
have been found, including a late breaking bug in the venerable cpio utility
that Michal announced to the full-disclosure list.
It is clear that AFL is succeeding perhaps where other fuzzers have either failed or not been applied, and this
is an important development for anyone trying to maintain a secure code base. That is, the ease of use coupled with
powerful fuzzing strategies offered by AFL mean that open source projects should integrate it directly
into their testing systems. This has been done for the fwknop project with some basic
scripting and one patch
to the fwknop code base. The patch was necessary because according to AFL documentation,
projects that leverage things like encryption and cryptographic signatures are not well
suited to brute force fuzzing coverage, and fwknop definitely fits into this category.
Crypto being a fuzzing barrier is not unique to AFL - other fuzzers have the same problem.
So, similarly to the libpng-nocrc.patch included in the AFL sources, encryption,
digests, and base64 encoding are bypassed when fwknop is compiled with
--enable-afl-fuzzing on the configure command line. One doesn't need to apply
the patch manually - it is built directly into fwknop as of the 2.6.4 release. This is
in support of a major goal for the fwknop project which is comprehensive
testing.

If you maintain an open source project that involves crypto and does any sort of
encoding/decoding/parsing gated by crypto operations, then you should patch your project so that
it natively supports fuzzing with AFL through an optional compile time step. As an example, here is a
portion of the patch to fwknop that disables base64 encoding by just copying data manually:

Fortunately, so far all fuzzing runs with AFL have turned up zero issues (crashes or
hangs) with fwknop, but the testing continues.

Within fwknop, a series of wrapper scripts are used to fuzz the following four areas
with AFL. These areas represent the major places within fwknop where data is consumed
either via a configuration file or from over the network in the form of an SPA packet:

Each wrapper script makes sure fwknop is compiled with AFL support, executes afl-fuzz
with the corresponding test cases necessary for good coverage, archives previous
fuzzing output data, and supports resuming of previously stopped fuzzing jobs. Here is
an example where the SPA packet encoding/decoding routines are fuzzed with AFL after
fwknop is compiled with AFL support:

And now AFL is up and running (note the git --abbrev-commit tag integrated
into the text banner to make clear which code is being fuzzed):
If the fuzzing run is stopped by hitting Ctrl-C, it can always be resumed as follows:

$ ./fuzzing-wrappers/spa-pkts.sh resume

Although major areas in fwknop where data is consumed are effectively fuzzed with
AFL, there is always room for improvement. With the wrapper scripts in place, it is
easy to add new support for fuzzing other functionality in fwknop. For example, the
digest cache file (used by fwknopd to track SHA-256 digests of previously seen SPA
packets) is a good candidate.

UPDATE (11/30/2014): The suggestion above about fuzzing the digest cache file proved
to be fruitful, and AFL discovered a bounds checking bug that has been fixed in
this commit.
The next release of fwknop (2.6.5) will contain this fix and will be made soon.

Taking things to the next level, another powerful technique that would make an interesting
side project would be to build a syntax-aware version of AFL that handles SPA packets
and/or configuration files natively. The AFL documentation hints at this type of
modification, and states that this could be done by altering the fuzz_one() routine in
afl-fuzz.c. There is already a fuzzer written in python for the fwknop project that is
syntax-aware of the SPA data format
(see: spa_fuzzing.py),
but the mutators within AFL are undoubtedly much better than in spa_fuzzing.py. Hence, modifying
AFL directly would be an effective strategy.

Please feel free to open an issue against fwknop in
github if you have a suggestion
for enhanced integration with AFL. Most importantly, if you find a bug please let me
know. Happy fuzzing!

The 2.6.4 release of fwknop is available for
download. New functionality has been developed
for 2.6.4, including a new UDP listener mode to remove libpcap as a dependency for
fwknopd, support for firewalld on recent versions of Fedora, RHEL, and Centos
(contributed by Gerry Reno), and support for Michal Zalewski's 'American Fuzzy Lop'
fuzzer. Further, on systems where execvpe() is available, all system() and popen()
calls have been replaced so that the shell is not invoked and no environment is used.
As usual, fwknop has a Coverity Scan score of zero,
and the code coverage report achieved by the 2.6.4 test suite is available
here.

Here is the complete ChangeLog for fwknop-2.6.4:

[server] Added a UDP server mode so that SPA packets can be acquired via
UDP directly without having to use libpcap. This is an optional feature
since it opens a UDP port (and therefore requires the local firewall be
opened for communications to this port), but fwknopd is careful to never
send anything back to a client that sends data to this port. So, from the
perspective of an attacker or scanner, fwknopd remains invisible. This
feature is enabled in fwknopd either with a new command line argument
--udp-server or in the fwknopd.conf file with the ENABLE_UDP_SERVER
variable. When deployed in this mode, it is advisable to recompile fwknop
beforehand with './configure --enable-udp-server' so that fwknopd does
not link against libpcap.

[server] Replaced all popen() and system() calls with execvpe() with no
usage of the environment. This is a defensive measure to not make use of
the shell for firewall command execution, and is supported on systems
where execvpe() is available.

(Gerry Reno) Added support for firewalld to the fwknopd daemon on RHEL 7
and CentOS 7. This is implemented using the current firewalld '--direct
--passthrough' capability which accepts raw iptables commands. More
information on firewalld can be found here:
https://fedoraproject.org/wiki/FirewallD

[server] Added support for the 'American Fuzzy Lop' (AFL) fuzzer from
Michal Zalewski. This requires that fwknop is compiled with the
'--enable-afl-fuzzing' argument to the configure script as this allows
encryption/digest short circuiting in a manner necessary for AFL to
function properly. The benefit of this strategy is that AFL can fuzz the
SPA packet decoding routines implemented by libfko. See the test/afl/
directory for some automation around AFL fuzzing.

(Bill Stubbs) submitted a patch to fix a bug where fwknopd could not
handle Ethernet frames that include the Frame Check Sequence (FCS)
header. This header is four bytes long, and is placed at the end of each
Ethernet frame. Normally the FCS header is not visible to libpcap, but
some card/driver combinations result in it being included. Bill noticed
this on the following platform:
BeagleBone Black rev C running 3.8.13-bone50 #1 SMP Tue May 13
13:24:52 UTC 2014 armv7l GNU/Linux

[client] Bug fix to ensure that a User-Agent string can be specified
when the fwknop client uses wget via SSL to resolve the external IP
address. This closes issue #134 on github reported by Barry Allard. The
fwknop client now uses the wget '-U' option to specify the User-Agent
string with a default of "Fwknop/<version>". In addition, a new command
line argument "--use-wget-user-agent" to allow the default wget
User-Agent string to apply instead.

[python module] When an HMAC key is passed to spa_data_final() then
default to HMAC SHA256 if no HMAC mode was specified.

In this blog post I'll pose a few challenges to open source projects. These challenges,
among other goals, are designed to push automated test suites to maximize code coverage.
For motivation, I'll present code coverage stats from the OpenSSL and OpenSSH test suites,
and contrast this with what I'm trying to achieve in the fwknop project.

With major bugs like
Heartbleed and
"goto fail",
there is clearly a renewed need for better automated testing. As Mike Bland
demonstrated, the
problem isn't so much about technology since unit tests can be built for both Heartbleed
and "goto fail". Rather, the problem is that the whole software engineering life cycle
needs to embrace better testing.

1) Publish your code coverage stats

Just as open source projects publish source code, test suite code coverage results should
be published too. The mere act of publishing code coverage - just as open source itself -
is a way to engage developers and users alike to improve such results. Clearly displayed
should be all branches, lines, and functions that are exercised by the test suite, and
these results should be published for every software release. Without such stats, how can
users have confidence that the project test suite is comprehensive? Bugs still crop up
even with good code coverage, but many many bugs will be squashed during the
development of a test suite that strives to exercise every line of source code.

For a project written in C and compiled with gcc, the
lcov tool provides a nice
front-end for gcov coverage
data. lcov was used to generate the following
codecoveragereports on OpenSSL-1.0.1h, OpenSSH-6.6p1,
and fwknop-2.6.3 using the respective test suite in each project:

The numbers speak for themselves. To be fair, fwknop has a much smaller code base than OpenSSL
or OpenSSH - less than 1/10th the size of OpenSSL and about 1/5th the size of OpenSSH in terms
of lines reported by gcov. So, presumaby it's a lot easier to reach higher levels of code
coverage in fwknop than the other two projects. Nevertheless, it is shocking that the line
coverage
in the OpenSSL test suite is below 50%, and not much better in OpenSSH. What are the odds that
the other half of the code is bug free? What are the odds that changes in newer versions won't
break assumptions made in the untested code? What are the odds that one of the next security
vulnerabilities announced in either project stems from this code? Of course, test suites are
not a panacea, but there are almost certainly bugs lying in wait within the
untested code. It is easier to have confidence in code that has at least some test
suite coverage than code that has zero coverage - at least as far as the test suite
is concerned (I'm not saying that other tools are not being used).
Both OpenSSL and OpenSSH use tools beyond their respective test suites to try and maintain
code quality - OpenSSL uses Coverity for example -
but these tools are not integrated with test suite results and do not contribute to the code
coverage stats above. What I'm suggesting is that the test suites themselves should get much
closer to 100% coverage, and this may require the integration of infrastructure like a custom
fuzzer or fault injection library (more on this below).

On another note, given that an explicit design goal of the fwknop test suite is to maximize
code coverage, and given the results above, it is clear there is significant work left to do.

2) Make it easy to automatically produce code coverage results

Neither OpenSSL nor OpenSSH make it easy to automatically generate code coverage stats like
those shown above. One can always Google for the appropriate
CFLAGS and LDFLAGS settings, recompile, and run lcov, but you shouldn't have to. This should
be automatic and built into the test suite as an option. If your project is using autoconf,
then there should be a top level --enable-code-coverage switch (or similar) to the
configure script, and the test suite should take the next steps to produce the
code coverage reports. Without this, there is unnecessary complexity and manual work,
and this affects users and developers alike. My guess is this lack of automation is a factor
for why code coverage for OpenSSL and OpenSSH is not better. Of course, it takes a lot of
effort to develop a test suite with comprehensive code coverage support, but automation is
low hanging fruit.

If you want to generate the code coverage reports above, here are two trivial scripts -
one for OpenSSH and
another for OpenSSL.
This one works for OpenSSH:

3) Integrate a fuzzer into your test suite

If your test suite does not achieve 99%/95% function/line coverage, architectural changes
should be made to reach these goals and beyond. This will likely require that test suite
drive a fuzzer against your project and measure how well it exercises the code base.

Looking at code coverage results for older versions of the fwknop project was an eye opener.
Although the test suite had hundreds of tests, there were large sections of code that were
not exercised. It was for this reason the 2.6.3 release concentrated on more comprehensive
automated test coverage. However, achieving better coverage was not a simple matter of
executing fwknop components with different configuration files or command line arguments -
it required the development of a dedicated SPA packet
fuzzer along
with a special macro
-DENABLE_FUZZING
built into the source code to allow the fuzzer to reach portions
of code that would have otherwise been more difficult to trigger due to encryption and
authentication requirements. This is a similar to the strategy proposed in Michal
Zalewski's fuzzer American Fuzzy Lophere (see the "Known
Limitations" section).

The main point is that fwknop was changed to support fuzzing driven by the test suite as a
way to extend code coverage. It is the strong integration of fuzzing into the test
suite that provides a powerful testing technique, and looking at code coverage results allows
you to measure it.

Incidentally, an example double free() bug that the fwknop packet fuzzer triggered
in conjunction with the test suite can be found
here (fixed in 2.6.3).

4) Further extend your code coverage with fault injection

Any C project leveraging libc functions should implement error checking against function
return values. The canonical example is checking to see whether malloc() returned
NULL, and if so this is usually treated as an unrecoverable error like so:

Some projects elect to
write a "safe_malloc()" wrapper for malloc() or other libc functions so that error handling
can be done in one place, but it is not feasible to do this for every libc function.
So, how to verify whether error conditions are properly handled at run
time? For malloc(), NULL is typically returned under extremely high memory
pressure, so it is hard to trigger this condition and still have a functioning system let
alone a functioning test suite. In other words, in the example above, how can the test suite
achieve code coverage for the clean_up() function? Other examples include filesystem
or network function errors that are returned when disks fill up, or a network communication
is blocked, etc.

What's needed is a mechanism for triggering libc faults artificially, without requiring the
underlying conditions to actually exist that would normally cause such faults. This is where
a fault injection library like libfiu comes in.
Not only does it support fault injection at run time against libc functions without the need to
link against libfiu (a dedicated binary "fiu-run" takes care of this), but it can
also be used to trigger faults in arbitrary non-libc functions within a project to see how
function callers handle errors. In fwknop, both strategies are used by the test suite, and
this turned up a number of bugs
like this one.

Full disclosure: libfiu does not yet support code coverage when executing a
binary under fiu-run because there are problems interacting with libc functions necessary
to write out the various source_file.c.gcno and source_file.c.gcda coverage files. This
issue is being worked on for an upcoming release of libfiu. So, in the context of fwknop,
libfiu is used to trigger faults directly in fwknop functions to see how calling functions
handle errors, and this strategy is compatible with gcov coverage results. The fiu-run tool
is also used, but more from the perspective of trying to crash one of the fwknop binaries
since we can't (yet) see code coverage results under fiu-run. Here is an example fault
introduced into the fko_get_username() function:

With the fault set (there is a special command line argument --fault-injection-tag on
the fwknopd server command line to enable the fault), the error handling code seen at the
end of the example below is executed via the test suite. For proof of error handling execution,
see the
full coverage report
(look at line 240).

Once again, it is the integration of fault injection with the test suite and
corresponding code coverage reports that extends testing efforts in a powerful way.
libfiu offers many nice features, including thread safey, the ability to enable a fault
injection tag relative to other functions in the stack, and
more.

5) Negative valgrind findings should force tests to fail

So far, a theme in this blog post has been better code coverage through integration.
I've attempted to make the case for the integration of fuzzing and fault injection with project
test suites, and code coverage stats should be produced for both styles of testing.

A third integration effort is to leverage
valgrind. That is, the test suite should run tests
underneath valgrind when possible (speed and memory usage may be constraints here depending
on the project). If valgrind discovers a memory leak, double free(), or other problem, this
finding should automatically cause corresponding tests to fail. In some
cases
valgrind suppressions will need to be created if a project depends on libraries or other
code that is known to have issues under valgrind, but findings within project sources should
cause tests to fail. For projects heavy on the crypto side, there are some instances where
code is very deliberately built in a manner that triggers a valgrind error (see Tonnerre
Lombard's write up on the old
Debian OpenSSL vulnerability), but these are not common occurrences and suppressions can
always be applied. The average valgrind finding in a large code base should cause test
failure.

Although running tests under valgrind will not expand code coverage, valgrind is a powerful
tool that should be tightly integrated with your test suite.

The 2.6.3 release of fwknop is available for
download. The emphasis in this release is maximizing
code coverage through a new python SPA packet fuzzer, and also on fault injection testing
with the excellent fault injection library libfiu
developed by Alberto Bertogli. Another important change in 2.6.3 is all IP resolution
lookups in '-R' mode now happen over SSL to make it harder for an adversary to mount a MITM
attack on the resolution lookup. As always, manually specifying the IP to allow through the
remote firewall is safer than relying on any network communication - even when SSL would be
involved.

Here is the complete ChangeLog for fwknop-2.6.3:

[client] External IP resolution via '-R' (or '--resolve-ip-http') is now
done via SSL by default. The IP resolution URL is now
'https://www.cipherdyne.org/cgi-gin/myip', and a warning is generated
in '-R' mode whenever a non-HTTPS URL is specified (it is safer just to
use the default). The fwknop client leverages 'wget' for this operation
since that is cleaner than having fwknop link against an SSL library.

Integrated the 'libfiu' fault injection library available from
http://blitiri.com.ar/p/libfiu/ This feature is disabled by default,
and requires the --enable-libfiu-support argument to the 'configure'
script in order to enable it. With fwknop compiled against libfiu, fault
injections are done at various locations within the fwknop sources and
the test suite verifies that the faults are properly handled at run time
via test/fko-wrapper/fko_fault_injection.c. In addition, the libfiu tool
'fiu-run' is used against the fwknop binaries to ensure they handle
faults that libfiu introduces into libc functions. For example, fiu-run
can force malloc() to fail even without huge memory pressure on the
local system, and the test suite ensures the fwknop binaries properly
handle this.

[test suite] Integrated a new python fuzzer for fwknop SPA packets (see
test/spa_fuzzing.py). This greatly extends the ability of the test suite
to validate libfko operations since SPA fuzzing packets are sent through
libfko routines directly (independently of encryption and
authentication) with a special 'configure' option
--enable-fuzzing-interfaces. The python fuzzer generates over 300K SPA
packets, and when used by the test suite consumes about 400MB of disk.
For reference, to use both the libfiu fault injection feature mentioned
above and the python fuzzer, use the --enable-complete option to the
test suite.

[test suite] With the libfiu fault injection support and the new python
fuzzer, automated testing of fwknop achieves 99.7% function coverage and
90.2% line coverage as determined by 'gcov'. The full report may be
viewed here: http://www.cipherdyne.org/fwknop/lcov-results/

[server] Add a new GPG_FINGERPRINT_ID variable to the access.conf file
so that full GnuPG fingerprints can be required for incoming SPA packets
in addition to the abbreviated GnuPG signatures listed in GPG_REMOTE_ID.
From the test suite, an example fingerprint is:

GPG_FINGERPRINT_ID 00CC95F05BC146B6AC4038C9E36F443C6A3FAD56

[server] When validating access.conf stanzas make sure that one of
GPG_REMOTE_ID or GPG_FINGERPRINT_ID is specified whenever GnuPG
signatures are to be verified for incoming SPA packets. Signature
verification is the default, and can only be disabled with
GPG_DISABLE_SIG but this is NOT recommended.

[server] Bug fix for PF firewalls without ALTQ support on FreeBSD. With
this fix it doesn't matter whether ALTQ support is available or not.
Thanks to Barry Allard for discovering and reporting this issue.
Closes issue #121 on github.

[server] Bug fix discovered with the libfiu fault injection tag
"fko_get_username_init" combined with valgrind analysis. This bug is
only triggered after a valid authenticated and decrypted SPA packet is
sniffed by fwknopd:

[server] Bug fix to handle SPA packets over HTTP by making sure to honor
the ENABLE_SPA_OVER_HTTP fwknopd.conf variable and to properly account
for SPA packet lengths when delivered via HTTP.

[server] Add --test mode to instruct fwknopd to acquire and process
SPA packets, but not manipulate firewall rules or execute commands that
are provided by SPA clients. This option is mostly useful for the fuzzing
tests in the test suite to ensure broad code coverage under adverse
conditions.

The 2.6.0 release of fwknop is available for
download. This release incorporates a number of
feature enhancements such as an AppArmor policy for fwknopd, HMAC authenticated encryption
support for the Android client, new NAT criteria that are independently configurable
for each access.conf stanza, and more rigorous valgrind verification powered by the
CPAN Test::Valgrind module. A few bugs were fixed as well, and similarly to the 2.5 and
2.5.1 releases, the fwknop project has a Coverity defect count of zero. As proof of this,
you can see the Coverity high-level defect stats for fwknop here (you'll need to sign up
for an account):
I would encourage any open source project that is using Coverity to publish their
scan results. At last count, it appears that over 1,100 projects are using Coverity,
but OpenSSH is still not one of them.

Development on fwknop-2.6.1 will begin shortly, and here is the complete ChangeLog for
fwknop-2.6.0:

(Radostan Riedel) Added an AppArmor policy for fwknopd that is known to
work on Debian and Ubuntu systems. The policy file is available at
extras/apparmor/usr.sbin/fwknopd.

[libfko] Nikolay Kolev reported a build issue with Mac OS X Mavericks
where local fwknop copies of strlcat() and strlcpy() were conflicting
with those that already ship with OS X 10.9. Closes #108 on github.

[libfko] (Franck Joncourt) Consolidated FKO context dumping function into
lib/fko_util.c. In addition to adding a shared utility function for
printing an FKO context, this change also makes the FKO context output
slightly easier to parse by printing each FKO attribute on a single line
(this change affected the printing of the final SPA packet data). The
test suite has been updated to account for this change as well.

[libfko] Bug fix to not attempt SPA packet decryption with GnuPG without
an fko object with encryption_mode set to FKO_ENC_MODE_ASYMMETRIC. This
bug was caught with valgrind validation against the perl FKO extension
together with the set of SPA fuzzing packets in
test/fuzzing/fuzzing_spa_packets. Note that this bug cannot be
triggered via fwknopd because additional checks are made within fwknopd
itself to force FKO_ENC_MODE_ASYMMETRIC whenever an access.conf stanza
contains GPG key information. This fix strengthens libfko itself to
independently require that the usage of fko objects without GPG key
information does not result in attempted GPG decryption operations.
Hence this fix applies mostly to third party usage of libfko - i.e.
stock installations of fwknopd are not affected. As always, it is
recommended to use HMAC authenticated encryption whenever possible even
for GPG modes since this also provides a work around even for libfko
prior to this fix.

[Android] (Gerry Reno) Updated the Android client to be compatible with
Android-4.4.

[Android] Added HMAC support (currently optional).

[server] Updated pcap_dispatch() default packet count from zero to 100.
This change was made to ensure backwards compatibility with older
versions of libpcap per the pcap_dispatch() man page, and also because
some of a report from Les Aker of an unexpected crash on Arch Linux with
libpcap-1.5.1 that is fixed by this change (closes #110).

[server] Bug fix for SPA NAT modes on iptables firewalls to ensure that
custom fwknop chains are re-created if they get deleted out from under
the running fwknopd instance.

[server] Added FORCE_SNAT to the access.conf file so that per-access
stanza SNAT criteria can be specified for SPA access.

[test suite] added --gdb-test to allow a previously executed fwknop
or fwknopd command to be sent through gdb with the same command line
args as the test suite used. This is for convenience to rapidly allow
gdb to be launched when investigating fwknop/fwknopd problems.

[client] (Franck Joncourt) Added --stanza-list argument to show the
stanza names from ~/.fwknoprc.

[libfko] (Hank Leininger) Contributed a patch to greatly extend libfko
error code descriptions at various places in order to give much better
information on what certain error conditions mean. Closes #98.

[test suite] Added the ability to run perl FKO module built-in tests in
the t/ directory underneath the CPAN Test::Valgrind module. This allows
valgrind memory checks to be applied to libfko functions via the perl
FKO module (and hence rapid prototyping can be combined with memory leak
detection). A check is made to see whether the Test::Valgrind module
has been installed, and --enable-valgrind is also required (or
--enable-all) on the test-fwknop.pl command line.

The fwknop project consistently uses valgrind
to ensure that memory leaks, double free() conditions, and other problems do not creep into the code base.
A high level of automation is built around
valgrind usage with the fwknop test suite, and a
recent addition
extends this even further by using the excellent CPAN
Test::Valgrind module. Even
though the test suite has had the ability to run tests through valgrind, previous to this change these
tests only applied to the fwknop C binaries when executed directly by the test suite. Further, some of the most
rigorous testing is done through the usage of the perl FKO extension to fuzz libfko functions, so
without the Test::Valgrind module these tests also could not take advantage of valgrind support. Now
that the test suite supports Test::Valgrind (and a check is done to see if it is installed), all fuzzing
tests can also be validated with valgrind. Technically, the fuzzing tests have been added as FKO built-in
tests in the t/ directory, and the test suite runs them through Test::Valgrind like this:

Note that all tests passed as shown above. This indicates that the test suite has not
found any memory leaks through the fuzzing tests run via Test::Valgrind. But, let's
validate this by artificially introducing a memory leak and see if the test suite can
automatically catch it. For example, here is a patch that forces a memory leak in the
validate_access_msg()
libfko function. This function ensures that the shape of the access request conforms to something fwknop
expects like "1.2.3.4,tcp/22". The memory leak happens because a new buffer is allocated
from the heap but is never free()'d before returning from the function (obviously this
patch is for illustration and testing purposes only):

This time two tests fail. The first is the test that runs the perl FKO module built-in
tests under Test::Valgrind, and the second is the "flagged functions" test which compares
test suite output looking for new functions that valgrind has flagged vs. the previous
test suite execution. By looking at the output file of the "flagged functions" test it
is easy to see the offending function where the new memory leak exists. This provides an
easy, automated way of memory leak detection that is driven by perl FKO fuzzing tests.