How DataRobot Optimizes Test Automation with Rainforest QA

Teams should leverage test automation. Used strategically, it increases efficiency and speeds up software delivery. Yet, not all tests are a great fit for automation, and teams often find that implementing it creates more challenges than it solves.

But that doesn't mean test automation can't be leveraged to create an optimized QA process. And the team over at DataRobot is doing just that! We sat down for a webinar with their VP of Engineering and Rainforest customer, Dustin Burke, and picked his brain on how they've optimized their QA strategy. Here is what we learned:

DataRobot Uses "Quality Assistance" to Build a Culture of Quality

Every team and product is unique, with varying testing needs. While there's no one-size-fits-all way to do QA -- especially when deciding how many tests to automate -- there are innovative approaches all successful teams share.

QEs do more high-value testing, like exploratory testing, rather than spending their time on regression testing. -Dustin Burke

With every member of the team contributing to product quality at each stage of the development cycle, QEs have more bandwidth to focus on high-impact testing initiatives. This approach doesn't just shift quality earlier in the development cycle. In DataRobot's case, this approach has created a culture of quality while optimizing resources.

While DataRobot operates within a continuous delivery pipeline and automates as many tests as possible, they are strategic about which tests they automate. Furthermore, DataRobot finds ways to automate aspects of their testing process that are not a good fit for test automation.

We are heavily invested in test automation. We even have a dedicated team to make our automated tests scalable and cost-efficient. - Dustin Burke

DataRobot Automates their Manual Testing Process with Rainforest QA

Strategically automating tests is just one piece of DataRobot's QA success. Additionally, the team automates their manual testing process with the Rainforest platform.

Rainforest provided us with a new capacity to automate the equivalent to manual regression testing for flaky UI tests, previously done by human testers on our team. - Dusin Burke

DataRobot's Delivery Pipeline with Test Automation and Rainforest QA:

Test Case Management: QA team manages Rainforest test suites, which are version controlled in their Github repository. Tests are written in plain English.

Deployment Integration: The Jenkins job is integrated with DataRobot's staging deployment pipeline. It's triggered by a 'switchover' (i.e. the automated tests passed, and a Blue/Green deployment was initiated).

Slack Notifications: The DataRobot team gets Rainforest test results through a Slack channel which is managed by their QA and Cloud Ops team, and uses the feedback to decide whether or not to ship.