Data-driven test with xUnit

In some cases it’s very challenging to recreate all data for some use case. It much easier to run the application, create some starting point: people, products, orders, prices, amounts, etc. and use such DB during tests. Automated tests should restore such DB in a test sandbox, run tests and drop the DB. All test frameworks have possibility to run some code before and after tests. This article describes xUnit Fixtures and Collections and how they could be used to build data-driven integration tests which interact with DB. The described solution sets a data context for separate collections of automated tests, restore, upgrade and drop a test DB. The solution uses xUnit 2.0 version as of March 16, 2015.

Test Scenario for xUnit

The simplest scenario for data-driven tests should be as follows depending on your case:

create DB for a collection of tests

upgrade DB for the latest version (optional)

run automatic tests

drop DB

Probably you would need to drop DB if it’s already existed, or upgrade it automatically to the latest version. I’m not going to describe that now because they’re very specific cases. I’d like to mention only one thing about running DB scripts. ADO.NET does not allow execute batches with GO-sentence. You should plan and write your scripts to be executed separately one by one. Even SQL Server Management Objects (SMO) splits such batches by ‘GO’ and run them separately (I checked). So the 2nd point is skipped, and xUnit can help solve others with their concept of Fixtures and Collections. These concepts are parts of xUnit shared context which is fully described in their documentation.

Fixture is a class which is instantiated before running each logical block, where the logical block can represent one or several tests. Collection is a class which groups several tests, but not instantiated at all during execution of tests. The topic below describes their life cycle which is important for the case described in the article.

Fixture and Collection Life Cycle

You can find a sample source code at GitHub. In the xUnit 2.0 authors replaced IUseFixture with ICollectionFixture and IClassFixture.

In order to demonstrate how xUnit instantiates the classes I created three test suites (one suite = one class). Two of them should be ran in the same test context, one separately. In the sample code you will see that the collections group tests regardless to their suites and the moments when each of the constructors and Dispose methods is executed. For instance, below is the excerpt from the code.

C#

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

publicclassCollectionFixture:IDisposable

{

publicCollectionFixture()

publicvoidDispose()

}

publicclassClassFixture:IDisposable

{

publicClassFixture()

publicvoidDispose()

}

[CollectionDefinition("ContextOne")]

publicclassTestCollection:ICollectionFixture<CollectionFixture>

{

publicTestCollection()// TestCollection is never instantiated

}

}

[Collection("ContextOne")]

publicclassTestContainerOne:IClassFixture<ClassFixture>,IDisposable

{

publicTestContainerOne()

[Fact]

publicvoidTestOne()

[Fact]

publicvoidTestTwo()

publicvoidDispose()

}

[Collection("ContextOne")]

publicclassTestContainerTwo:IDisposable

{

publicTestContainerTwo()

[Fact]

publicvoidTestOne()

publicvoidDispose()

}

publicclassTestContainerThree

{

[Fact]

publicvoidTestOne()

}

You should run the tests in debug-mode to see something in output window. Just a little comment about xUnit parallelism. By default tests are executed synchronously within the same collection or class. In other cases tests are executed in parallel. You can find more information about xUnit parallelism here. So the actual output may vary a bit on your PC, but for the demonstration purpose I adjusted it.

C#

1

2

3

4

5

6

7

8

9

10

11

12

13

14

CollectionFixture:ctor

ClassFixture:ctor

TestContainerOne:ctor

TestContainerOne:TestOne

TestContainerOne:disposed

TestContainerOne:ctor

TestContainerOne:TestTwo

TestContainerOne:disposed

ClassFixture:disposed

TestContainerTwo:ctor

TestContainerTwo:TestOne

TestContainerTwo:disposed

CollectionFixture:disposed

TestContainerThree:TestOne

Thus you may see that ICollectionFixture can be used to put several tests in the same context and set up some environment. IClassFixture can be used to setting up environment which is specific for a suite. The class constructor also can be used for that purpose too, but it’s called for each test in the suite and you should plan carefully their execution.

Implementation Details

It should be clear now that we can create a fixture-class to encapsulate DB routines and attach it to our tests by means of ICollectionFixture or IClassFixture. I use a collection for this purpose. Its constructor restores DB before tests and the Dispose() method drops the DB in the end.

I found several pitfalls while was implementing the solution with backup files:

When DB is restored from a backup file its embedded information about files is used. Your tests will fail if they use the same backup in parallel because of the conflict with names. The moving data and log files prevents such conflicts.

Generally backups can use different number of files (data, log, file stream, etc). The script should correctly restore such backups. I assume in the script that data and log exists always. Other files can be ignored using PARTIAL keyword.

Below you can find the scripts which can be run in constructor and Dispose().

The base fixture class should be abstract and require specification of connection string and backup file only. You can use the SqlConnectionStringBuilder class to get a target DB name for the script. The scripts should be called under [master] BD context.

Also you should be aware about xUnit parallelism. If you need definitely drop DB after some tests you should call Dispose() manually.