Last weekend we hosted the second theEvnt workshop in Madrid, this time on Injection. Our colleague Pedro Gómez was the best host to talk about Dependency Injection on Android and it was a total success.

Again, around 40 people attended this 4-hour workshop on a Saturday morning, and we really appreciate that guys, thank you all! We learnt how to deal with Dependency Injection on the Android platform following different approaches and using existing libraries. Before finishing the workshop, theEvnt raffled a new book about pushing the limits programming on Android that Fernando Martín received. Congrats!

As every month, we hosted the 9th PHPMad meeting in the kitchen of our central Madrid office. Francisco Santamaría talked about “Emergent design with phpspec”. We saw a brief introduction to the tool specBDD called phpspec and how it can help us designing our application, being much more than a simple testing tool.

In addition to PHPMad, we are open to hosting other user groups meetings and talks so, if you’re interested in organising a tech or design-related event, get in touch!

Asynchronous code is intrinsically hard to test. Most of the async testing solutions out there use some kind of waiting mechanism that stops the test until the async logic returns. That kind of solutions are natural and easy to use, but they introduce non-determinism in your tests. Non-deterministic tests coud be extremely dangerous in Continuous Integration environments, since they can affect the deployment pipeline, slowing down the development team, and eventually the whole company. Furthermore, unstable tests reduce the developers' confidence in the tests, and can ruin the whole suite if it they are not taken care of (see Fowler 2011).

Our iOS team at Tuenti recently released TUScheduler: a very simple Objective-C library to deal with Async Testing problem in a very elegant and robust way, and without introducing any non-determinism.

In this blogpost we explore the problem of async testing and how it affects test quality and the speed of development teams. We show the different solutions available, exploring their strengths and weaknesses. Finally, we explain the approach followed in TUScheduler, and talk a little bit about this simple library. Although TUScheduler and all the code snippets of this post are written in Objective-C, all the concepts and ideas can be directly applied to any language or framework that requieres testing asynchronous code.

The Problem of Async Testing

Asynchronous code is hard to test because in order to validate it, a test has to stop and wait for the async code to finish. Only then can it verify that its behaviour was as expected.

Inactive waiting techniques (i.e. using sleep()) are the worst possible approach in this case, since they stop the test for a fixed amount of time, which is usually several orders of magnitude higher than the real expected duration of the async task. This is why solutions based on active waits are strongly preferred.

Active Waiting Techniques

The most common approach in the async testing scene is to use some kind of polling mechanism to periodically check if the async code finished executing:

The test checks to see if the async code has finished (here represented by the testFinished flag). If it hasn't, the test sleeps for a very short amount of time, and then checks again. This process is repeated until the expected condition is met, or the total waiting time surpasses the defined timeout.

The advantage of this approach over inactive waits is that we only waste at most pollingTime seconds. That is why most iOS async testing frameworks use this technique (e.g. Kiwi, GHUnit, SenTestingKitAsync). Apple is also providing a new native mechanism for async testing in Xcode 6 which allows expectations to be defined for your tests using XCTestExpectation.

A very common variation to Polling is to design the async calls to have customisable Callbacks that get called on completion. Tests can listen for those callbacks to know when to proceed to the verification phase. Once more, we need to use some kind of active wait based on a handcrafted timeout value.

Timeouts and Continuous Integration

Polling techniques are usually enough in simple environments: they provide simplicity and a very natural syntax, and, as long as timeouts are correctly tuned, polling should be fine most of the time.

However, there is a critical problem associated with polling: how to choose the timeout value. If this value is too tight, the test suite could very easily fail due to overloaded machines, or other special circumstances, introducing uncertainty in the tests. On the other hand, very high timeout values could make the whole suite to take a huge amount of time when several tests time out in the same run.

This problem can significantly slow down teams that relies on a Continuous Integration environment, in which integration of branches is done very frequently by different members of the team (or by different teams), using an automated deployment pipeline.

How can we choose an appropriate timeout for testing this method? There is no way to know how long it would take to execute incrediblySlowMethodToLoadContacts. The only option in this case is to estimate a reasonable timeout by trial and error, which gives the tests enough time to get back from incrediblySlowMethodToLoadContacts and pass.

This estimated timeout is typically adjusted to work fine during normal deployment cycles, but there is no way to ensure that it would be enough under special circumstances. For example, during crunch times, when many teams are trying to integrate their branches, and all the systems are overloaded. In those circumstances, timeout could not be high enough, producing apparently random test failures.

This kind of non-determinism in tests is very dangerous, and can bite you in many ways (see Fowler 2011 for a very detailed analysis). But in this blogpost we are specially interested in how non-determinism can affect development teams that use an automated integration pipeline.

Tuenti Case Study

The iOS team of Tuenti is constantly changing due to the need of the company to move fast and quickly adapt to new challenges. However, a very typical configuration involves around 8 to 10 developers, distributed across an iOS Core Team and 3 to 5 product teams. Our codebase has more than 300k lines of code, that without counting more than 20 external libraries (some of them our own forks of other Open Source libraries), over 400 branches, and over 18,000 changesets.

Each team works independently on their own branches, which are integrated very frequently. We use Flow, an state-of-the-art automated deployment system created by our devops engineers to take full control of the integration process. When a branch is ready to be integrated, we send a pull request to Flow directly from Jira. Flow creates a temporary branch with the merge of the current integration branch and the pull request branch that will be merged, and enqueues the temporary branch waiting for a free slot in the build servers. When an slot is allocated, Flow creates a job in Jenkins for the branch. If the job gets blue ball, the branch is automatically merged into integration, the Jira ticket is updated, and developers are notified. Every changeset of integration is a Release Candidate, so we better make sure this whole process is robust.

In the iOS platform, a complete build (including creation of a fresh server instance, static analysis, build, testing, and report generation) takes approximately 40 minutes. On an extraordinarily busy day, when many features are scheduled to be integrated, a branch can sit in the queue for several hours. If after that time waiting the build fails because of an unstable test, the process needs to be restarted from the beginning. The owner of the affected branch has to merge all the changes of the branches that were originally queued after the affected branch, which are now being integrated before, and the branch goes back to the last position of the queue. As the servers get busier and busier, more tests timeout, and more branches are erroneously rejected. The problem grows exponentially affecting more developers, and effectively blocking the integration pipeline of the whole team.

So it is not a matter of just running the tests again. In our every day work at Tuenti, unstable tests can significantly slow our teams down, and reduce our ability to move fast and deliver on time.

TUScheduler Approach and the Humble Object pattern

TUScheduler is an alternative to the Polling/Callback approach. It is inspired in the Humble Object Pattern. The idea is to extract all the hard-to-test code (the async behavior) from your business logic into a Humble Object (TUScheduler), so you can test your business logic separately, just as if it were fully synchronous. The Humble Object must be a very thin concurrency layer, so simple that it doesn't even need to be tested.

When a class needs to perform any asynchronous job, instead of directly posting this job using any of the available concurrency APIs (GCD, NSOperationQueue, etc.), it will delegate this responsibility into a TUScheduler. The scheduler can be provided as a parameter or injected into the class during initialization.

When running tests, the class will be injected with a test-specific scheduler, called the TUFullySynchronousScheduler. This scheduler is not concurrent at all, it just immediately executes all the jobs sent to it in a purely synchronous way. This way, tests can focus on testing the business logic without having to care about timeouts or concurrency management.

Does this affect the quality of our tests in any way? The answer is no. This little scheduling trick should be completely safe: the essence of concurrent programming is to separate what has to be done from when it is done, so the class should not be making any assumption on when the scheduled jobs are being executed, and the associated tests should be just as valid no matter whether the jobs are synchronous or asynchronous. In fact, if a piece of code is in some way dependent on when the async jobs are done, then it is probably exposed to race conditions and other synchronization problems, and should be refactored ASAP.

Furthermore, the use of TUScheduler also brings a very nice side-effect: classes are not coupled to a particular concurrency API any more, so you can swap APIs at any time by simply changing the construction of your async classes. If you are using Dependency Injection like we do at Tuenti, then the change is less than one line of code, probably just one word!

TUSchedulerFactory

The concurrency mechanisms that a class uses for its internal operation are not interesting to the component that creates the class. Each class should be responsible for managing its own schedulers privately, since they are just implementation details. It would be a very bad idea if the component that creates an object is also responsible for configuring and injecting an scheduler. That would be knowing too much about the object and its internal details. Furthermore, some classes could need several schedulers to operate (e.g. a serial one to keep an internal collection synchronized and a concurrent one to post heavy background tasks).

This is why the recommended way of using TUScheduler is injecting the abstract factory <TUSchedulerFactory> into the class, and using it internally to produce and configure all the <TUScheduler> instances that the class might need.

For testing, TUScheduler provides an special concrete factory called <TUFullySynchronousSchedulerFactory>, which always returns fully synchronous schedulers. This way, the concurrency is still isolated inside the Humble Object, making the async logic testable, but at the same time each class keeps its implementation details private.

Test Coverage

TUScheduler is designed to be extremely simple (only six classes, two protocols, and very few lines of code). That is the spirit behind Humble Object Pattern: the extracted logic must be so simple that it does not even need to be tested at all.

Even so, TUScheduler comes with a suite of unit tests that enforces the contract of the <TUScheduler> protocol in both the TUGCDScheduler and TUOperationQueueScheduler concrete implementations. This way, users of TUScheduler can focus on testing their business logic, knowing that the concurrency management is tested elsewhere, and more importantly, only once.

These tests use the special helper class TUAsyncTestSentinel, which implements a Polling algorithm to test the concurrent behaviour of schedulers. Yes, timeouts. Why not? In this case there is no way to avoid non-determinism, and, on the other hand, there is no problem with unstable tests as long as they are kept isolated and out of the deployment pipeline, where they cannot bite us. We at Tuenti use TUScheduler as an external library, with its own tests that run separately on each push to ensure they still pass. That is all we need to know.

At any case, TUScheduler is so simple that one could even completely ignore the tests. Visual inspection should be more than enough to spot a bug in such a simple piece of code.

Conclusion

Testing async code is intrinsically hard. Most solutions for writing async tests include some kind of timeout system, which will introduce indeterminism to the suite of tests. While many development teams can live with this small amount of uncertainty, some of us just can't.

When a team relies on an automated build system, that aspires to maintain an evergreen integration branch, and in which branches need to be integrated frequently and independently, unstable tests could represent an important problem, slowing developers down, and eventually the whole team.

Last weekend, we hosted the first theEvnt workshop in Madrid. TheEvnt is a foundation created by one of our engineers and based on Extremadura. Its main goals are supporting local region development and providing easy access to resources related to Internet, Mobile, Software, Marketing, Social Media, SEO and so on.

Tuenti and theEvnt are going to collaborate together on achieving those goals. For that reason, we will organize a monthly workshop from now on. The first one was held last weekend at our offices and the topic was "Swift". One of our iOS senior engineers, Victor Pimentel, who has been playing with Swift since it was released, was the speaker.

We had a blast! Around 40 people attended to this 4-hours workshop, a Saturday morning, and we really appreciate that guys, thank you! Learning, networking, chatting and lots of fun. We learnt Swift basics and then we created a sample app using XCode. Before the end, theEvnt also raffled a new book about iOS and Swift that one of the attendees would enjoy. Borja López was the lucky guy!

Thanks you all again for coming and we hope to see you soon in the next theEvnt workshop about Android next month. You can check all the updates in the workshops web, following theEvnt or the Tuenti Engineering accounts on Twitter. Stay tuned! ;)

As every month, we hosted the 8h PHPMad meeting in the kitchen of our central Madrid office. Eduardo Gulias talked about “Driving your Development in PHP”. We were focused on methodologies like TDD, DDD, BDD which can help us to drive our development. Eduardo brought us the basic concepts about them and the different frameworks / libraries which are available in PHP.

In addition to PHPMad, we are open to hosting other user groups meetings and talks so, if you’re interested in organising a tech or design-related event, get in touch!