A look inside: Scenario testing for Unity 2017.2

Unity 2017.2 released earlier this month and we in QA are happy to share some results of our latest scenario test week. Previous testing sessions like this were really funny and useful, so we plan to continue running these for each release.

After nearly a month of preparation – information gathering, brainstorming, synchronizing and setting up different conditions – we had an exciting scenario test week full of testing, game development, and gaining useful information about the state of Unity 2017.2.

Help us make Unity even better: Participate in the Unity beta program. Just sign up to our beta newsletter via the form at the bottom of the page to receive notifications on new beta versions, guides for beta testing, and new features in the upcoming versions of Unity.

The main goal of each scenario test week is to experience Unity as a user or small game studio (and keep being testers in the same moment). This week our mission was to create a game with attention to new areas and features, looking for bugs and issues.

We decided to split areas of coverage down to a more granular view, and ended up with a list of features and areas with 86 items. The most covered areas were Services (Unity Teams), UI, Input, and Troubleshooter.

QA is not the only team involved; we invite all experts to join in scenario testing, so we are able to get different points of view – from the perspective of artists, developers, designers, project managers and product owners. It allows us to experience the game-making process from prototyping to game completion, the continuation of an existing scenario for several iterations, and, actually, eating our own dog food.

For this week, we had 16 scenarios – some of them were a continuation of existing projects from previous times, and some were absolutely new and built from scratch.

Scenarios we worked on during 2017.2 scenario test week.

We managed to identify 130 bugs, mostly in the Editor, VR and Timeline areas.

Bugs reported during 2017.2 scenario test week versus team size.

One of the most interesting parts of scenario test week was the example projects we made, mostly using our services like the Asset Store, Unity Collaborate, Cloud Build, Dashboard, Analytics and Multiplayer.