Speeding ROBLOX Development with Continuous Testing

Speeding ROBLOX Development with Continuous Testing

We’ve been talking a lot lately about the new features and engine improvements coming to ROBLOX. But behind the development is a relatively unseen – and extensive – process that ensures the changes don’t break the game for players: Testing.

ROBLOX undergoes constant, rigorous, automated tests. Every time a developer “checks in” a new line of code, our testing system creates a trial version of ROBLOX and runs it through a battery of automated tests. This not only ensures the new code works, but also that it didn’t inadvertently introduce new bugs somewhere else.

Testing: All day, all night

We run full test suites on ROBLOX dozens of times each day. Each test suite consists of hundreds of tests – and this number is growing daily – for multiple platforms. Do the math and that means we run tens of thousands of tests per day, with each test covering hundreds of thousands of lines of code.

Each colored block represents activity by ROBLOX’s various testing systems.
As you can see, our PC and Mac testing machines are active throughout the day.

Testing: Any type, anywhere

Our testing framework consists of three varieties of tests: manual, unit and automated. Considering the size of ROBLOX, each one is necessary for us to release regular, quality updates.

Manual testsManual testing, in the most basic sense, consists of jumping into a ROBLOX game and seeing whether a specific behavior works. ROBLOX is too complex for us to test it in this way, so we’ve built up a library of testing scripts, written with very specific steps, that we send to a dedicated testing house. Using their extensive resources, they run the tests – usually overnight – and return detailed diagnostics the next day.

For example, we wrote a script that tests whether rockets function as they should. If a code change causes them to, say, fall to the ground, the test fails and we know there’s a regression that needs fixing. We have a lot of these scripts, covering a spectrum of ROBLOX behaviors.

Unit tests
Unit testing refers to ROBLOX running tests on itself. If ROBLOX were a car, its unit tests would be checking its own oil level, transmission temperature, etc., and reporting problems – like being low on oil. We have a bank of machines that almost always have ROBLOX running tests on itself because the tests fire up each time a developer checks in a line of code.

To write unit tests, we use the boost::test framework. Here’s an example of a test that makes sure our raycasting code works:

Automated testsAutomated testing determines whether a specific behavior happens as it should. The tests actually open instances of ROBLOX and run Lua scripts – the same language ROBLOX players use to create games – to determine whether the behavior happened as desired. For instance, an automated test might open ROBLOX and run for 30 seconds to ensure a ball continues bouncing until 30 seconds have passed (as opposed to bouncing away, falling through the floor or some other buggy behavior).

Testing: You can, too

To make writing automated tests easy, we extended the boost::test framework into our Lua environment. If you’re an advanced ROBLOX Studio user, you might have noticed a new TestService feature. This lets you write sophisticated tests right inside a game. All scripts inside the TestService are executed when a test is run, but not during normal gameplay.

Here’s a script we run to make sure the BindableEvent feature is working:

Let’s say there was a bug in BindableEvent. In that case, when we run the test we might see the following output:

What good does all this testing do you? Continuously testing ROBLOX is allowing us develop new functionality more quickly, and with less risk of causing regressions in the process. This ultimately means a better gaming experience and more consistent releases for our users.

ive had problems with the connenction too. the internet status on my pc taskbar is 5 bars and i just got a new router and its in the same room as my computer. i can connect to a game, but after a while of playing, the game freezes and it says the connection was lost. please fix it as soon as possible.

I have had this issue to and all I can say so far about ideas to resolve it is to try disconnecting and then reconnecting to the router. Doing this once I went from a download rate of about 60 KBs/sec to over 1 MB/sec.

Im really excited for the upcoming features & adjustments that are being made for ROBLOX. I honestly have to say this, ROBLOX has surely gotten better over the years mostly this year [2012] without a doubt! Keep up the AMAZING work ROBLOX Staff…ROBLOX just keeps getting better & better :)

I still believe that Roblox should realease their updates sooner and at least not tell us when its in development. Cause what we dont know wont hurt us. Like if you are making a trade system and it is just being thought of dont get us all excited over lost hopes D:* , and i still belive you sould hire me :D Tweetytime :D

I don’t entirely see the truth in this post. I do appreciate the effort that you guys put into testing everything, however with each update, there’s always something, mostly scripting wise that breaks. That is truly frustrating.

That’s been doing that since Studio Mode (What you now call Edit Mode) was created. GUI’s are for the player. On Studio Mode, you are not a player (Technically, there is no player you control) during that time. If you want to work with GUI, I suggest opening ROBLOX Studio and use Solo Mode (Build Mode today) and use the tools as a player.

There is no “bug” with edit mode. Obviously what is happening in edit mode is it’s showing the GUI before any scripts are run. So if your GUI is off the screen before a script changes its position you wont see it until you hit the “play” button in tools.