Posts Tagged ‘unit testing’

Introduction

Last time I blogged on creating my first Swift application that would display a Koch Snowflake fractal on an iPad or and iPhone. This time I’m going to look at adding some unit tests to that program to see how that works in XCode and Swift. It appears that Apple has built some nice unit testing support into XCode so doing this was actually quite easy.

I put the Koch Snowflake Swift project on Google Drive here. Feel free to download it and play with it. Refer to this project or the previous blog post for the code we are actually testing in this article.

Test Driven Development

If I was following proper Test Driven Development (TDD) then I should have written the unit tests before I wrote the actual code. This would force me to think about the design of the APIs and the testability of the program first, and then would force quick cycles of writing tests, writing code so the tests pass and then refactoring the code to make it all better. However, in this case I ported an old Objective-C program to Swift to play with Swift and now that I’m getting more familiar with Swift, I’m finally looking at the unit testing framework. So what I’m doing is perhaps not a good practice, but adding the unit tests later is better than not adding them at all.

The danger I could have run into is that the program could have turned out to be quite hard to test due to perhaps some bad design decisions and then required quite a bit of refactoring to make it properly unit testable. Fortunately, in this case the program is fairly simple and adding a good set of unit tests was fairly straight forward and didn’t require any changes to the program.

One criticism of TDD is that there is a perception that it interferes with good program design and architecture. This isn’t true. The misconception is that the first tests need to be truly useful, whereas the first tests usually just call the API and set how the API to the various classes is used. The classes are just skeletons and the unit tests pass as long as everything compiles. Getting everything to compile is the first step and thinking about all the APIs before writing the real code that does stuff is the first step to getting a good modular design.

Apple has adopted TDD for a lot of their own projects, for instance their Core Data module was developed this way. Since Apple developers use XCode and now many are using TDD this means that a lot of good support for TDD is being baked into XCode. It also means that newer Apple technologies are much easier to use in a TDD environment than some of their older ones. But even for the older ones there are lots of clever examples on the web of how to work around the various challenges they introduce.

Testing Turtle Graphics

When you create a project with XCode you have the option to include unit tests (which I did) and this gives you a test project that you can run with some dummy tests in it. So I figured first let’s add some unit tests to my TurtleGraphics class. This is pretty simple you move the turtle and turn the turtle and it leaves a trail behind it. For drawing fractals turtle graphics makes the algorithms very easy since it fits in well with the recursive algorithms typically used.

The turtle graphics class uses Apple’s Quartz2D library to draw on the screen. Perhaps the first possible design flaw is that when I instantiate the turtle graphics library I need to pass its constructor a valid graphics context. The underlying Quartz2D library routines certainly don’t like this being nil and throw an exception if you try that (ie my first try). So I thought I would just create a graphic context object but after battling with Swift a bit, I found I couldn’t do this because the underlying class doesn’t have any constructors. This also prevented me from mocking the class (at least using the real class and replacing just a few methods). This is because Apple wants to make sure you get a valid graphics context from one of their factory methods. If you are actually in the drawing routine this is passed to you by the system, but as a unit test, we aren’t running in a screen drawing context. Fortunately there is a way to get a graphics context to draw on an in-memory image file, so I used that.

To solve this, I could have refactored the drawing routines into another class and had this class accept nil for the graphics context and just return when this happens. This is basically just making a new class that pass through to the real class when the program runs, or acts as a stub class when running unit tests. I didn’t like this so much since then if the real program did pass nil, it wouldn’t get caught and the program just wouldn’t work properly. Even though I’m not testing the actual placement of pixels on the screen, it makes me feel good to know the actual real drawing routines are being exercised during the unit tests, so if they do get an invalid argument or something the problem will be caught.

The unit test framework lets unit tests get at variables in a class to help you set up the asserts to prove correctness (you import the module via @testable import KochSnowFlake rather than just import), so to test it, I just did some simple move and turns and then tested that the internal state of the turtle graphics module was correct. Notice when I turned 45 degrees I used trigonometry to calculate the new position, for testing its usually good to test against a calculation, rather than running the program to see what the value is and then using that. Notice the use of XCTAssertEqualWithAccuracy, since we are using floating point arithmetic, rounding errors creep in, so the equals will never be exact. You can configure XCode to either stop on each error or run to completion just recording all the errors, its up to you. For this I usually just stopped on an error and then fixed the problem.

Testing the View Controller

Now let’s test things at a higher level. Generally, we don’t do UI testing in unit tests, but we can do some basic tests to ensure everything is created properly and is more or less hooked up correctly. XCode does have built in support for UI testing and perhaps we’ll have a look at that in a later blog post. In this case we are going to create the storyboard (where the UI is defined) and from that instantiate our View Controller. Then we do a dummy access for the view to get the delay loaded elements loaded and start calling the View Controllers methods starting with viewDidLoad(). For our test we set the fractal level to 3 in the text box and then call the notification methods and see if it then updated properly in our View.

func testInitialViewController()
{
// Test that the storyboard is connected to the view controller and
// that we can create and use the view and controls.
let storyboard = UIStoryboard(name: "Main", bundle: nil)
let viewController = storyboard.instantiateInitialViewController() as! ViewController
_ = viewController.view
viewController.viewDidLoad()
viewController.fractalLevelTextField.text = "3"
viewController.textChangeNot("dummy")
XCTAssertTrue(viewController.fracView.level == 3)
// This next line is just to get 100% code coverage.
viewController.didReceiveMemoryWarning()
}

Test Coverage

The big questions that is usually asked about unit tests, is how much code to they cover (or exercise). XCode can give this report which you can see in the screenshot:

Amazingly we nearly get 100% coverage. The only module with poor coverage is AppDelegate.swift which is code generated by XCode and then I never use it. The reason we get such good coverage is that this is a fairly simple program and it draws the snowflake when it initializes. But the key point here is that I am testing a lot of things fairly easily and fairly quickly. It tests the functionality of the turtle graphics library and it tests that the UI is all hooked up properly and working.

A true TDD adherent would delete all the code in AppDelegate.swift, since code shouldn’t be written unless it is to make a unit test pass. But I tend to give an exception to code generated by the tools like XCode, to me this code is Apple’s responsibility and not mine. Plus, if I ever do add code to this class then I would need to start adding unit tests and take it a bit more seriously.

Harder Cases

This program is pretty simple, so I didn’t need to do anything too fancy to unit test it. This is mainly because there is no database access, no hardware access (beyond drawing and one text field) and no communication with a server. If you add these then you would need to add mock, stub and/or fake classes to assist in testing. You want your unit tests to be fast and run on every compile. You don’t want to require databases be setup or that servers are running somewhere in a certain state. Fortunately, Swift is a very powerful object oriented language and creating all of these is fairly easy (extensions are especially helpful). The main problems you run into are in using the Objective-C libraries like UIKit, but again you aren’t the first to run into these and there are tons of sample code, blogs and lessons on how to deal with these.

Another tricky area is with asynchronous calls. When you make a call to the server, the response will be returned asynchronously. What this means that as soon as your unit test sends the server request, it will complete. Since the asynchronous request comes back on a separate thread later, you need to ensure your main thread doesn’t just end and you wait for the results, or the tests in the callback will never be executed. There are lots of examples on how to do this, but it definitely adds some complexity to your tests.

Continuous Build Server

If you have a team, then you will probably want a continuous build server that monitors your source code repository (probably Git) and then does a build and test each time something new is checked in. Apple has an XCode Server that you can purchase if you are a registered developer. Or you could use the Jenkins Build Server which is open source and free. Jenkins also has an XCode plugin to make life a bit easier. Though XCode is quite good at letting you control it from the command line. This way you can be notified as soon as something breaks and ensure problems get fixed right when they are caused, which is the easiest time to fix them.

Summary

That turned out to be quite a long posting on unit tests. XCode and Swift have a very powerful and easy to work with unit testing framework built in. If you are developing a new application, certainly consider using TDD for your development methodology. If you are porting or doing a new version of an app, then certainly try adding some unit tests. Unit tests will help you greatly over the lifetime of your product, making it more reliable, bug free and easier to maintain.

Unit Testing is a technique to test individual units of source code. Usually in the Java world this means a set of tests to test an individual class. The idea is to test the class (or unit) in complete isolation from the rest of the system, so calls to other parts of the system would be stubbed or mocked out. Unit test are typically written by the software developer to perform white box testing for their class. Unit testing is only one sort of testing there would be other types of testing for various integration testing, user testing, load testing, manual testing, etc.

The goal of unit testing is to show that the individual parts of a program are correct and then to deliver the following benefits:

Establish a well-defined written contract of what the code must do.

Find problems early in the development cycle.

Facilitate change and refactoring since you can rely on the unit tests to prove things still work.

Simplify integration since you know the building blocks work correctly.

Provide documentation, the unit tests show examples of actually using the class being tested.

JUnit is a unit testing framework for Java programs. It is integrated into most IDEs like Eclipse and it is easy to run the tests from build tools like Ant. Usually in Eclipse you have a test tree with all the test classes, usually a test class for each real class and the test class containing all the tests for the real class. JUnit support a number of annotations that you put on the methods in the test class to specify which ones are tests to run as well as optionally specifying any order or dependency requirements (the best unit tests run completely independently of each other). Then it adds a number of methods to call to test for pass and failure of the tests (via various forms of assert) and to report the results back to the test framework.

GWT

The Google Web Toolkit (GWT) supports unit testing with something called GWTTestCase which bridges GWT to JUnit. If you want to interact with GWT controls then you have to use this. However it’s relatively slow. You want unit tests to run blazingly fast. A unit test should only take a few milliseconds to execute. If it takes longer then it won’t be run often. You want it so your unit tests can be run every time you compile and so they don’t slow you down as you work.

We use GWTTestCase to test our extension of the GWT CellTable widget. These tests take 10 minutes to run. This is way too slow to expect them to be run on a regular basis. Hence they don’t provide the immediate feedback to a developer working on this widget that we would like.

Fortunately since in GWT you are writing all your classes in Java, if you structure your program correctly you can have most of your classes not interact with GWT or easily mock the GWT part of it. Then you can use JUnit directly to unit test your program.

MVC

One of the key goals of us enforcing MVC design patterns on our developers creating UIs is to facilitate good unit testing. This way if most of the code that needs testing is the Model and Controller, these can be tested with JUnit and we don’t need to use GWTTestCase. This greatly simplifies unit testing and greatly improves the productivity and speed of the tests. Usually our View part is very small and mostly implemented in the SWT framework, so the unit testing is then in framework unit tests (many of which are GWTTestCase), and not in the actual UIs.

EasyMock

EasyMock is a library that can dynamically “mock” classes that aren’t part of the test. With unit tests we only want to test the one class at a time and we want the tests to run quickly. We don’t want to require a particular database be in place and then have to wait for the tests to run as it makes database queries. We want the tests to run in milliseconds and we want the tests to run in spite of what might be happening in the rest of the system. To this end the test framework needs to replace the real classes that the class being tested calls and interacts with, with something appropriate for testing. These test replacement classes then have the ability to return unexpected or rare results such as network disconnection errors, or other rare hard to setup type of scenarios. One way to do this is to write “stub” classes that replace the real classes and then only have code to do what is required for the testing. Writing and maintaining stub classes is difficult since you need to keep them in sync with the classes they are stubbing; keeping these correct can become a major amount of work.

EasyMock offers an easier way. It generates the classes on the fly using Java’s proxy mechanism. This way the class interface is always up to date since it is generated from the real class. You first run in a “recording” mode where you record what you want the various methods to return to the test cases. Then you switch to “play” mode and run the tests.

For this to work effectively, you must design classes so classes they use can be mocked. This usually means creating them externally and then passing them to the constructor rather than creating secret internal instances. Generally this is good object oriented design anyway and usually difficulty in constructing unit tests is a “code smell” that something is wrong with the design. For simple utility classes or simple classes in the same package, it’s often easier to leave them in place rather than mock them, then they get a bit of testing as well and as long as they run quickly, shouldn’t be a problem.

Example

Below is one of the tests from the BOM Detail popup form in Order Entry. This shows the structure of a JUnit test and shows how to mock a big class (namely the Model) that this class interacts with. Notice that the Model is created externally to this class and passed into the constructor, so it is easy to mock. First we record what the mock class should do and then we switch to replay mode to do the actual test.

/**
* Tests that the callback’s onPopupClosed handler will filter on the child
* BOMs of the parent component number that was passed in at callback * creation time.
*/
@Testpublicvoid testOnPopupClosedFiltersOnChildrenOfCallerParentBOMNumber()
{
// Set up the mock model and the expected method calls that should be
// made on it by the class under test.
OEDocumentBOMDetailsModel mockModel = EasyMock.createMock(OEDocumentBOMDetailsModel.class);
mockModel.filterOnChildBOMsOf(EasyMock.eq(TEST_PARENT_COMPONENT_NUMBER), EasyMock.isA(EntryFilterCallback.class));EasyMock.expectLastCall();

// Tell EasyMock we’ve finished our setup so that any time we interact
// with the mock, EasyMock records what calls were made on the mock.EasyMock.replay(mockModel);

// Check that the expected method calls on the mock happened when we
// exercised the method under test.EasyMock.verify(mockModel);
}
}

Summary

Unit testing is an effective form of testing that does find quite a few bugs and helps keep code running smoothly as a system is developed and refactored going forwards. Sometimes the discipline of unit testing forces programmers to ensure they fully understand a problem before coding resulting in a better solution. Certainly this can’t be the only form of testing, but it is an important building block to a quality software system.