Navigation

Buster.JS consists of many small Git repositories/npm modules. We try to keep
things small and separated. The repositories and installable modules both
promotes reusability (e.g. you can use Buster assertions with any test
framework)
and helps us avoid tight coupling between modules. However, some people feel
that the number of repositories are a bit daunting when trying to contribute.
This document sheds some light on existing modules, what they’re for, and what
role they play in the bigger part.

Note that some of our modules contain stronger abstractions than others, and
the less obvious ones may very well change until we have something that we’re
comfortable with. If you have suggestions for how certain modules can improve
(renaming/merging/splitting/refactoring/whatever), feel free to let us
know.

Core modules are those that are installed when you type npminstallbuster. They constitute the core of the test framework, and contains
everything you need to run node tests, simple browser tests, and automated
crowd-sourced browser tests.

Buster.JS auxilliary modules

Developer tools and docs.

Buster.JS extensions and optional modules

These modules provide additional and optional features/extensions for
Buster.JS, such as linting, JsTestDriver support, and more.

A simple and generic mechanism for flagging warnings. The analyzer is an event
emitter, and provides fatal, error and warning methods, which emit
corresponding events. Additionally, the analyzer has a concept of ok/not ok.
This is decided from a threshold (i.e., a threshold of “error” means “not ok”
if any error or fatal events where flagged).

The analyzer also comes with a reporter which can be used to log events of
interest. The analyzer and the reporter is used by buster-test-cli and
extensions to provide various insight about your code. Examples of practical
usage includes linting and syntax checking (for browser tests).

The capture server captures browsers as slaves, and offers a completely generic
API for carrying out work across those slaves. A workload is known as a
“session”, and a test run is typically a session. Other uses include for
instance synced-across-devices slide shows (for which a POC has been built).

In general, the server knows nothing specifically of testing. It knows how to
accept and server resource sets, capture and command browser slaves, and
coordinate every piece using messaging (Bayeux on the HTTP level).

Somewhat arbitrary collection of functions used in several other buster
modules. Includes the event emitter implementation used throughout, some
limited flow-control utilities, and a few functional enhancements. Hopefully,
we can get rid of this one day.

A logger-like utility that simply emits events. This is useful in any number of
cases, most importantly when running tests in browsers via
ramp. In this case, we pass the events over the wire instead
of printing them to the console.

ASCII formatting of arbitrary JavaScript objects. This module is used to give
pretty feedback in certain cases. It is used to format objects in assertion
error messages, to format objects passed to buster.log (and
console.log, if captured) and may be used in more places later. Also
intended for reuse outside of the Buster.JS sphere.

Represents files in a project that may be included in a test run. For Node.js,
Buster.JS only use ramp-resources to look up which paths to require.
For browsers, Buster.JS uses ramp-resources to build a virtual file
system, send it over HTTP and mount it on the server. All of these components
are available in this module.

ramp-resources also includes intelligent caching of resources to allow
test runs that only reads changed tests from file and so on. Resources
typically map to files on disk, but really can be anything, including one-off
strings in a configuration file.

A small extension (but installed and activated by default) that provides server
side syntax checking of scripts sent for testing with
ramp. When Buster.JS loads scripts in browsers, the
browser in question will be the one responsible for the level of detail when
errors arise, Syntax checking on the server allows us to catch these errors in
one place, and produce a pretty nice report, regardless of browser intended to
run the tests.

A small library for working with ANSI escape sequences in the terminal. Mostly used
for colored output, and positional output. Also includes a “labeled list”
object that is used to progressively print multiple lines of output at once,
e.g. when running tests on multiple browsers (the dots reporter).

Implements testCase, describe (and friends), the actual test runner,
reporters and supporting objects. buster-test is centered around the
concept of “test contexts”, which is just a bag of tests, and possibly more
bags of tests. This is the shared data format produced by both the xUnit and
BDD style tests/specs.

The reason we’re considering breaking up this module is that it includes parts
that are complete and unlikely to change (such as test case and spec
definitions, the context data format and the test runner) <strong>and</strong>
parts that are likely to change and/or have abilities added, such as reporters.
The name also indicates it is what powers the buster-test binary, which is
not true.

General purpose command line argument parser. Only parses command line options,
no printing to the console, no --help generation, no flow control. Also
tries as best it can to adhere to UNIX conventions. Fails early (typically when
using non-existent options ++).

Work in progress. Extension that will allow AMD projects to use
Buster.JS without any specific configuration. It modifies the load path
of the resourceSet used to represent user files and creates
an anonymous AMD module that depends on all tests, thus loading files
using an AMD loader rather than simple script tags. Currently developed by
Joakim Ohlrogge.

Extension that automatically compile CoffeeScript files before running tests.
In its current state, this extension does not work for files that are to be
included using require(), and is thus not very useful for Node.js projects.
Currently developed by Stein Magnus Jodal.

Work in progress. Extension to calculate line coverage. Uses the
resourceSet to instrument code, and emits custom messages over the test
runner to build up the report. Currently developed by Tobias Ebnöther.

An extension that implements “markup-in-comments”, using the
/*:DOCel=...*/ format originally found in JsTestDriver. The extension
was originally developed to be API compatible with JsTestDriver in the
buster-jstestdriver extension, but works well with vanilla Buster.JS test
cases (and specs) too.

Extension that enables the integration of JsLint and JsHint by way of
autolint. Using the
buster-analyzer module, the lint extension is able to flag lint errors as
“error” in buster. This allows the end-user to choose if lint errors should
only be printed as warnings, or actually fail the build (which can be achieved
with bustertest-Ferror). Currently developed by Magnar Sveen
<https://github.com/magnars>.

This section runs through what happens when you automate browser tests from the
command line, e.g. when you type something like bustertest--browser. The
idea is to highlight roughly the flow through the various parts of Buster.JS,
and to illustrate practically how the modules depend on and interact with each
other.

bustertest executes the “binary” script buster/bin/buster This is a
small wrapper script that can print some help, and that can look for other
commands on the path called buster-<something>. In this case, it finds the
buster/bin/buster-test script in the same package. The buster package is
a “meta package”, meaning that it does not contain much implementation, it’s
just there to glue all the pieces together and give you a convenient install
target.

buster-cli/lib/buster-cli.js is not so much a real abstraction, as it is a
collection of routines useful in buster CLIs. It centralizes help text
formatting, provides helpers for adding CLI options with help text, locates the
configuration file and coordinates loading it with running.

The --browser option is a posix-argv-parser shorthand that expands to
--environmentbrowser, which is the long form for specifying environment.

The configuration file is located and “resolved” in
buster-cli/lib/buster-cli.js. It tries to find the configuration file in
one of ./, ./test/ or ./spec/. If it is not found, the parent
directory will be consulted in the same way until we’re at the root.

If the --config option was provided, only that file will be
consulted. buster-cli contains some error handling in case
configuration could not be located.

For loading the contents of the configuration file into memory, a separate
package, buster-configuration, is used. Buster.JS defers actually reading
source files from disks as long as possible, so “resolving” the configuration
file only loads relevant groups with their extensions and builds lazy resource
sets to represent files. buster-cli uses several options to filter out
the groups found in the configuration file to figure out which ones will
eventually be run.

A configuration group has a method called runExtensionHook. You call this
method with the name of a hook and some arguments. Any extension in the
configuration that has a method of the same name will then be called with the
passed in arguments.

Depending on what groups resulted from reading the configuration file and
filtering it according to command line options, the following steps may be
repeated several times. For simplicity, this example assumes only one
configuration was loaded.

Now that buster-test-cli knows that we’re running tests for the browser
environment, it loads the browser runner. The runner will have its run
method called with the configuration loaded from file, an options object, which
contains prepared options for things like color etc, and a callback that will
be called when the run is over.

The runner now uses a little abstraction that is shared between the browser and
the node runner. It creates an analyzer for general-purpose health-checks,
and fires the "beforeRun" extension hook, allowing extensions to register
analyzers.

The browser runner then proceeds to instantiate a
buster-client/lib/client.js. This object is a JavaScript interface that
speaks HTTP to a running buster server. Because the server component is
currently being reworked, this document will only briefly touch on the concepts
it implements.

The server has the ability to capture browsers as slaves. A slave is a browser
that has loaded a frameset, where one frame, “the control frame”, keeps a
persitent connection to the server, awaiting instructions. The server also
provides an HTTP API for creating a session - a piece of work to be carried out
in available slaves. When this happens, the server uses the bidirectional
connection to instruct slaves to load the session in a separate frame.

When loading a session in a browser, an index.html file is loaded in a separate
frame, and this file will include <script> tags that loads all the files
originally specified in the configuration under libs, sources,
testHelpers and tests.

Alongside the sources, a little wiring script is loaded. This file
configures a listener for new test cases and specs Finally it wires up a
test runner with a JSON proxy reporter and defines buster.run() as a
way to start the whole thing. The test runner is completely evented, and the
JSON proxy reporter is just a way of making sure the events are only data, thus
HTTP-encodable. The events from the test runner are sent directly over the
wire.

Back on the client, buster-test-cli is now ready to use its HTTP client to
create a session. The client starts by asking the server for available cache
manifests. These will be handed to the resourceSet, and will make sure
Buster does not read any files from disk that are already hosted in the same
version on the server.

Any file that isn’t already cached will now be “serialized” (i.e. read from
disk) and sent to the server as part of the HTTP POST request to create a
session. The server uses a resource set cache
to cache and look up cached resources, and a resource set middleware
to actually serve them over HTTP.

With the session readily created, buster-test-cli‘s browser runner is
listening for messages. These messages are piped into its remote runner.
This object accepts messages from multiple test runners (i.e. one per browser),
and emits messages as if it was one test runner. The originating browser is
represented as an outer context for all tests. This allows the “remote runner”
to be used directly with any existing reporter written for the regular test
runner.

After the remote runner has been initialized, but before the tests are actually
started, the client issues the "testRun" extension hook, which allows
extensions to interact with the test runner (e.g. to listen to specific
messages etc).

When the session is created and the remote runner is initialized, the browser
runner will listen for the test runner’s "suite:end" event. This event
comes with a short summary, which is passed to the browser runner’s done
callback (passed to run).

The test.js CLI interface will now use the test report to decide if the run
was successful or not and exit with a corresponding exit code. In the case
where there are more configuration groups to be run, these will be run before
exiting.