How We Maintain a Trustworthy Rainforest Tester Network

Rainforest QA is committed to meeting the security standards of our customers, and are constantly evolving our processes to deliver on that promise.

On a previous post we described what measures we take to secure the VMs used to test our customers’ applications But how about the testers themselves? In this post, we’ll dive into how we secure the actual crowd to make sure they are real, trustworthy people.

Crowdsourced Work Providers

Our testers come to us through either CrowdFlower or Mechanical Turk, which constitutes our first filter. Both services are dedicated to crowd provisioning and management and provide us with the first check of workers’ identity and payment information verification.

To be eligible to become part of the Rainforest tester family, a worker must first meet the providers requirements and prove themselves qualified worker by doing a minimum required set of tasks through a period of time.

Rainforest Tester Training & Peer Reviews

Once they have meet the requirements for our crowd providers, the crowd worker becomes eligible to start training for becoming Rainforest tester. During this training period, we provide sample applications that might or might not contain bugs, and present the most common or important scenarios we have seen with our customers tests. This helps testers gain the knowledge and experience needed to execute tests up to our standards. This also serves to rule out bots or automated systems trying to get tester accounts, as bugs are inserted into the applications randomly.

In order to assure the testers are doing what is requested of them, we have implemented a peer-review mechanism, where our pool of proven and trusted testers review and flag tester work for review by our staff. These trusted testers do not have the ability to hand out rejections or block accounts — rather, they help Rainforest admins review test results and provide feedback as to whether the work was accurate or not. Every piece of feedback they give is then reviewed by a Rainforest admin before the decision to reject or block a worker is made. This not only works as quality control, but also as a filter to decide which testers shouldn’t be allowed back into the community.

Optional Rainforest Tester Security Measures

We also took an extra step for our customers’ comfort and have implemented a series of optional requirements specific to said customer account. Only testers that have meet the requirement or requirements requested would be able to access customer tests.

Malware Scans

Our testers can provide recurrent malware scans from the systems they use to access our platform. We support Windows through Windows Defender and Mac/Linux systems through ClamAV, both of which allow us to parse and verify the malware scans on the backend.

Two-factor Authentication

We provide the support needed for testers to secure their accounts using two-factor authentication, which can also be set up as a requirement for customer tests. This helps us ensure that verified tester accounts remain secure.

Customized NDAs

For customers with specific NDA requirements, we support the signature of custom, customer-provided NDA contracts with our testers.

It’s important to remember that the majority of our testers have worked with us full-time for 2 years on average, and see Rainforest as their livelihood. We take security measures to ensure that our testing network stays composed of these hardworking, trustworthy testers!

*Editor’s Note: This post was updated on August 16, 2017 to clarify the role that tester peer review plays in approving and rejecting test results. *

Nic is the head security architect for Rainforest QA. He loves new technologies (if they are open, all the better!). He has strong beliefs and preferences towards process automation: if you do something once that's fine, if you are doing it twice, automate it.