Featured in
Architecture & Design

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Featured in
Operations & Infrastructure

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

Featured in
Enterprise Architecture

Mini-talks: The Machine Intelligence Landscape: A Venture Capital Perspective by David Beyer. The future of global, trustless transactions on the largest graph: blockchain by Olaf Carlson-Wee. Algorithms for Anti-Money Laundering by Richard Minerich.

System/Acceptance Testing with Time and Dates

Unit Testing of Dates is often talked about problem – consider that one can’t depend on an absolute date in the test case. If the test passes, then the test will be considered successful only for the respective date. Instead, the standard advice is to use dependency injection (Adam Sroka, BigVisible):

In order to test a number of interesting dates you need to pass them into the calculation. Thus, the method that does the calculation should never get the date for itself – whether it is “now,” or from a database, or from user input, etc. You should obtain the date at a higher level and pass it in to your calculation.

It's easy to see how this works for Unit Tests – but the question remains: what do for Acceptance/System tests? Andreas Ebbert-Karroum faced just this problem and came up with a list of three alternatives:

In your tests, make the expected result also date/time dependent.

Change your application, so that it does not use the system current time, but a time service, that you can control remotely to use a different time than the current system time

We use an abstraction layer over top of the real time service (read: hide the calls to the system time API behind an interface) and then replace the implementation when testing with one controllable by the test scripts. By default, the real implementation is plugged in.

Pros: Can ensure that all test conditions are covered on every run. Can be used to test time race conditions.

George Dinwiddie, Software Development Coach, also notes that a single time source avoids subtle problems with systems that run across multiple machines with multiple time references i.e. system time on one machine vs. database time on another.

Ward Cunnigham (wikipedia) takes a different approach: “In the "swim" framework we tested cases that lasted weeks. We wrote SQL with a global substitution, $now, that would, in test, accumulate an expression that would have terms for every time advance, be it hours, days or weeks. This had the advantage of looking great when we traced the SQL.”

Mike Stockdale, Developer of FitSharp, points out that it supports relative date parsing, ex: today+2. Rick Mugridge, author of FitLibrary, notes it has: “a general date generator "DSL" which allows for the selection of relative dates in different time zones, supplied in different formats. It allows for dates to be selected at the end of a month, on a Monday, etc. This has been used extensively on a booking system where date selection in the future is critical, and where the tests have to make use of the data (and dates) that exist in the system under test.” While Martin Gijsen, test automation architect, uses ANTLR to solve the same problem.

So both Andreas’s first and second approaches are in wide use. Which approach have you used? What are their pros and cons?

... uses the (much too unknown, under-utilized, under-maintained....) library timeandmoney for date and time handling. You find it on sourceforge.

Applications uses a static method on a singleton clock with injectable "timeSource" to get the current time or date(examples taken from our local fork so some trivial classes might be missing in the sourceforge version):

CalendarDate today = SystemClock.today();

For unit testing time- and progress-dependent functions, set an adjustable, fixed timesource:

Naturally we have several several TimeSource classes that can be used, including an OffsetTimeSource that follows the normal clock adjusted for some offset, which let you run integration tests with a normally progressing clock set to a specific date or time-of-day.

We have also discussed combining a TimeSource with the transactionManager in order to "freeze the time" at the start of a transaction, so transactions not only becomes wisible as a unit in the database but also contains timestamp data that are all identical to the start of the transaction. This will make virtually all aspects of the transaction appear atomic. (We haven´t done it, though, but it IS quite simple).

Interesting Eirik thanks for letting us know about another good library. Is this it: sourceforge.net/projects/timeandmoney/? With Eric Evans involved can I assume its well tested? Not just unit but also acceptance/api tests (even if they're written in JUnit)?

Unlike java and joda it has the basic concepts right, which is an enormous benefit. For example is a CalendarDate only three immutable numbers (y,m,d), but the beginning and end can be mapped to (immutable) TimePoints using a calendar and a time zone. The 'money' part will give you little, but I can assure you it is the simplest and best library for time calculations around.

There are quite some tests but I've also found a bug or two.

We have been using a local copy for years.

Unfortunately, we get risk getting fired if we let a single line of source code escape the entrance door (no, I'm not kidding), so our fixes and additions have not been possible to integrate with the sourceforge project. The example above is prose, as you can see (no curly braces).

It is really unfortunate that we cannot contribute fixes and community to a library that is a core part of our systems. It is a security related promise to our customers (we do the payment and money transfers between banks - a sector seemingly built on security through secrecy) and also a way to prevent deverlopers from working on pet projects in their paid time. I hope someone else can.

[...] Not just unit but also acceptance/api tests (even if they're written in JUnit)?

Well, we use the OffsetTimeSource in the program and have made a Servlet to adjust it from http get-urls.

This facilitates adjusting the time when the system runs inside a container. A small web page on top of the servlet lets human testers adjust the time in order to check behaviour in the easter, on new years eve, just around midnight etc. Such adjustments can also be scripted on *nix with wget or curl (and also in java using jwebunit or even just plain java.net.URL's).

It is also possible to create a TimeSource that reads the offset from a database table now and then, which is more convenient if you need to adjust the time for a cluster installation.

Otherwise, we try to run most tests in-process, embedding the webcontainer (jetty) as a library instead of the opposite. This is a whole lot faster and simpler, with tests typically running inside the IDE in 10 seconds or less. That way you can simply use the same approach as in the junit test example I provided.

Fitnesse tests use the same approach as junit, of course. Create a fixture that sets an adjustable time source and adjusts the time as specified in the test. It must probably return 'ok' so fitnesse has something to validate afterwards but that is just cosmetics.

Ted - I'm not personally a fan of option #3 although apparently Andreas is. If I'm correct in understanding you set the time on the machine to be what ever you need for the test. The benefit being that there is no risk of bug in the dependency injection layer. Frankly it seems like a lot of work for a tiny benefit. In the past I've used both options 1 and 2. The bigger issue is trying to make an exhaustive list of all the edge cases you will encounter. Inevitably you miss something which is why I like as much as possible to use 3rd party libraries to hide the complications.

Unfortunately, we get risk getting fired if we let a single line of source code escape the entrance door (no, I'm not kidding), so our fixes and additions have not been possible to integrate with the sourceforge project. The example above is prose, as you can see (no curly braces).

It is really unfortunate that we cannot contribute fixes and community to a library that is a core part of our systems. It is a security related promise to our customers (we do the payment and money transfers between banks - a sector seemingly built on security through secrecy) and also a way to prevent developers from working on pet projects in their paid time. I hope someone else can.

Quite aside from the fact this seems silly and punitive - are you allowed to report bugs and make feature requests?

Ingo - I just checked code was last committed ~600 days ago. So just under two years - Apr 2008? The projects authors are Eric Evans (of Domain Driven Design fame), Barry Hawkins (also a DDD guy) and Blaine Buxton. Based on the street cred of the authors and the fact that Eirik is using the library in production it seems like a good bet. In the worst case download it, try it and throw it away after a day if you don't like what you see.

BTW I will ask Barry Hawkins if he can jump in and tell us what he thinks of the current state of the library.

Ah, OK. The word "remotely" was what confused me. I'm not a fan of changing the machine's system time either, because all sorts of weird operating system-related things start happening. If I were really that concerned, I'd probably put the system under test into a VM and use then change the system time. However, it seems like that would be more useful in a high-level integration test and not unit tests, since the whole point of unit tests is to isolate things not to change the world.

That would be nice!The web site quoted above (sourceforge.net/projects/timeandmoney/files/) shows the last release to 0.5.1 and the date for this release to be 2006-10-26I looked at the library before and liked what it did - the above release number and date however makes it look like a dead project ....

It is dead as in: large organizations use it but are prohibited from providing anything back to the project.

Also, the lack of community is in this case not a large issue. After all, how many ways to add a week to a date will the future hold that we don´t already have today? Unlike more recently developed areas, Time is not a moving target.

option three means, that you ssh (or telnet) into your test machine and set the time and date to what you current automated acceptance test requires. even if you are in a mixed windows/linux environment this is easily achievable.

The way I see it there are three solutions 1) Adjust the time values in the expected results (#1 above) This works good if your results are in XLS format, and can contain date based formulas that get evaluated as the expected results are read into the system. 2) Adjust the Perceived 'current' time as seen by the test code(#2 #3 above both address this basically via different means) 3) Adjust the time values in the test data.

In my experience for systems that had to do any kind of 'aging' evaluation (finance stuff like wash sale rules, long-term vs short term gains, or 'stale' aging for items waiting for responses, workflow routing etc) #3 actually turned out to be an easier solution. We used a set of scripts that adjusted the test data according to the difference between when the data snapshot was taken, and the current time, so that offsets for aging calculations always yeilded the same results, something that was say 29 days old, would always be 29 days old. The system level tests commenced after the database had been restored and 'adjusted', and that approach worked great for stuff were accuracy was restricted to 'day' or 'hour' level of granularity. such as the examples given above.

There is an open source library TimeMachine which heavily simplifies creating these kinds of tests. It allows moving the current time of JVM to an arbitrary point of time in the future. At the same time it does not change the current system time (the change is only visible inside JVM).

The power of this library comes from the fact that you don't have to modify your existing code to use it as it enhaces the bytecode of loaded classes to modify the behaviour of System.currentTimeMillis/nanoTime, Object.wait and Thread.sleep methods. Therefore the various types of schedulers (java.util.Timer, Quartz, etc.) as well as embedded databases work right away.

Chuck - changing the data seems like an interesting approach, how much effort did you find you had to put into maintaining the scripts? How about the performance - was this suitable only for system tests?

We have used it to test long running processes in our workflow product and we didn't have any problems with that. The only obstacles so far came from the inappropriately written code in third-party libraries. An example of that is Bitronix transaction manager which periodically tries to do recovery of transactions. Unfortunately if it does this recovery at time T and finishes it at time T + Q, it then tries to do it at time T + some delay (instead of T + Q + some delay) even though it is in the past. So if we move to the future by a year then it tries to make up for all this time and fires lots of threads...

As for the .NET stuff I'm not aware of any kind of a tool like that. At the same time I'm not really a .NET developer anyway.