The Official Venmo Blog

November 12, 2018 · by David Gallagher

QuizTrain: A Swift framework for TestRail's API

At Venmo we love using TestRail to manage and track tests for our mobile and web apps. We wanted a way to integrate it with our automated iOS tests which are written in Swift. TestRail provides an excellent API but no Swift framework to use it. So we decided to build one and open source it under the MIT License so you can use it too!

QuizTrain allows you to do everything you can do with TestRail's API from the comforts of Swift on iOS, macOS, tvOS and watchOS. Acting as a foundation you can integrate it with your Swift testing framework or even build an app on top of it. We've been using it at Venmo since early 2018 and absolutely love it. 😀

How Venmo uses QuizTrain

On iOS we use Apple's User Interface Testing framework to run automation tests. It provides a great way to test Venmo from the perspective of users while remaining within Apple's development ecosystem. TestRail acts as our single source of truth for all test cases and results. Keeping this in mind we leveraged QuizTrain to do the following:

As tests run we log rich test case descriptions pulled from TestRail.

We track results as pass/fail based on assertion outcomes in tests.

When complete we create a test plan and submit results to TestRail.

This was accomplished by building a few objects on top of QuizTrain and extending XCTContext. You can find an example project in the QuizTrain repository containing this code.

If any assertions fail between steps 4 and 6 then the test case ID is marked as failed. Otherwise it passes. Later when results are submitted the failure message is appended to the result description so you can view it on TestRail. If multiple failures occur between steps 4 and 6 then every failure is included in the result description.