This documentation is for the prototype version of Holochain, written in Go.
There is a new version under development, for which there is new documentation and a Developer Preview pre-release.
View the future of Holochain here.
The documentation for the holochain-proto version will remain available online.

Introduction

The Holochain system includes a testing harness that allows you to test your application functions. In addition we have built integration that simultaneously runs a set of tests in separate virtual nodes. We strongly recommend using both these tools for application development.

Running basic single-node tests

Assuming your current working directory is my_app you can run all the single-node tests (test/*.json) with this command:

$ hcdev test

This command loads a new copy of my_app into the ~/.holochaindev directory, and then finds all the .json extension files in the test directory and runs them in a random order. Note that this doesn't run the test files in the scenario directories, only the top-level test files.

You can run a single test file with (note you don't need the .json extension).:

$ hcdev test <test-name>

Test Files

A test file is essentially a collection of function calls into your application with specified inputs and expected outputs, or error values. A test file consists of an object of the form:

{

Identity: string // identity to be used for Agent.String Tests: [ { Convey: string // a human readable description of the tests intent Zome: string // the zome in which to find the function FnName: string // the application function to call Input: string // the function's input Output: string // the expected output to match against (full match) Err: string // the expected error to match against ErrMsg: string // the expected error message to match against Regexp : string // the expected out to match again (regular expression) Time: int // offset in milliseconds from the start of the test at which to run this test. Wait: int // time in milliseconds to wait before running this test from when the previous ran Exposure: string // the exposure context for the test call (defaults to ZOME_EXPOSURE) Raw: bool // set to true if we should ignore fnName and just call input as raw code in the Ribosome, useful for testing helper functions and validation functions, or doing more complex testing Repeat: int // number of times to repeat this test, useful for scenario testing }, {...} ]}

Replacement Strings

Input and Output values can include special replacement values that allow you to substitute for various in-system values for your tests.

%h%,%h1%,%h2%,...: the hash of the nth entry on the source chain. Allows you to test that value was committed. %h% would mean most recently written, and %h1% second most recent, and so on.

%r1%,%r2%,%r3%: the three most recent test results treated as a stack. Useful if result is a string (not an object).

%result0%,%result1%, ... : the results from previous tests in order of the test. Useful if result was a string (not an object)

{"%result%":X},: result from the Xth test. Use this form when that result was an object, not a string.

%dna%: the DNA hash.

%agent%: the hash of the second entry on the chain which is always, the permanent agent entry.

%agenttop%: the hash of the most recent agent entry on the chain (because of key revocation or identity change).

Each test/scenario sub-directory should contain one test file for each role required to model the test. The scenario command requires the name of the scenario directory [my_app/test/<scenarioName>] as a parameter. Test filenames are automatically discovered.

Tests pass or fail on the basis of the content of messages passed between roles/nodes on the network. In order to test the content of messages passed between roles, it is necessary for tests to account for the amount of time it takes for messages to travel between nodes on the network. This is achieved with the Time parameter of the test object. If roles 1 and 2 are called back and forthrespectively, then when forth sends a message, back should wait at least 50ms for checking to see if there are messages that contain the expected content. If the message has not yet arrived, then the test will fail.

You can add a `_config.json` file to the scenario directory of the following format:

a minimum duration that all the nodes should be kept running for the test to succeed, i.e. so other nodes that still have tests pending can gossip with them.

a list of roles for which you want multiple clones to run at the same time, plus the count of clones of that role.

In the chat sample application you will find a backnforth scenario with two roles defined: person1 & person2. Check out these examples to see how things work.

Testing Error Conditions

To test error conditions you can specify your expected error in either the `Err` or `ErrMsg` attributes. `Err` matches against an entire error object, so usually it's easier to use `ErrMsg` which just matches against the `errorMessage` attributed within the error object. Note that if you have set `ErrorHandling` in the javascript zome `Config` to `"returnErrorValue"` the error object will be returned as a value from a any calls you make and thus won't show up in `Err` unless you explicitly `throw` it.

Notes on the Time parameter:

if Time is null or 0, tests are executed in the order discovered in the test file

all other values of the Time parameter are used in multi-node testing to allow sufficient space for gossip operations between nodes on the DHT.

to test a sequence of messages sent and received between Holochain nodes, between 50 to 200 millis is required for message arrival.

role1.json >> "send message, Time = 0"

role2.json >> "check for message, Time = 100"

the 50 to 200 millis number works well within the test harness network. Tests crossing external networks may require much more time for messages to be delivered.

TODO Note, it might make sense to implement both an automatic backnforth test generator, and also an asynchronous method of testing for the test message arrival (e.g. a message id)