Pages

Tuesday, May 25

Beta Stats and Tweaks

Here's a sweet interview with Bungie's Brian Jarrard from IGN which is kind of a post mortem on the fantastic Halo Reach beta. There's some great stats in there and some noted changes to the mechanics of Halo Reach's mulitplayer (and presumable campaign itself) as a result of the testing.

The Halo: Reach beta has closed, but the work is not done for Bungie. Now comes the task of fine tuning the game in preparation for the final product, hitting store shelves on September 14. I was able to track down Brian Jarrard, community director for Bungie, to chat about what the team learned from the beta and what sorts of crazy stats have been pulled.

IGN: Were you able to get all of the info and data you needed from the beta? How long will it take to fully analyze the feedback?

Brian Jarrard: Our team received a staggering amount of info and data thanks to the support and participation by our community. More than 2.7 million unique participants logged more than 16 million hours testing Reach and it will absolutely result in a better final game experience this September.

Thankfully a large component of the feedback we needed was in the form of networking stats and game data uploads which our team is able to process and digest fairly automatically on the back-end. There's still a ton of data to sift through but a lot of that can be handled with systems and queries that our engineers have constructed. That data was being analyzed daily as it came in and to some degree is still being evaluated by individuals on the team.

Player feedback was generally more difficult to analyze simply due to the volume and types of input being provided. The Bungie.net beta forum alone had more than 360,000 forum posts during the Beta and that doesn't include all the feedback coming in through other channels. Our team spent a lot of time wading through these posts and comments to try and filter out consistent key issues or takeaways. That's easier said than done when you have such an outspoken and passionate community. For every person who wrote a full page about something they absolutely hated, another person wrote a full page describing why they loved that same thing. At this point I think the majority of that feedback has been sifted, sorted, processed and disseminated to the appropriate people on the team and a lot of it has in fact influenced tweaks to the final game.

IGN: Can you dish some stats from the beta? What's the craziest number you've pulled from the data?

Brian Jarrard: For me personally the craziest number I've seen was the most time spent playing by a single person, which was 198 hours. That's roughly 12 hours per day for the entire duration of the beta. That's just crazy.

I think the 2.7 million total unique participants number is pretty awesome – even more so considering that the Halo 3 beta had just over 800,000 players during its lifespan (the Reach beta had surpassed that by the end of the first day).

There were more than 1.4 million pieces of user generated content (films and screenshots) uploaded during the beta which equated to roughly 1.3 Terabytes of data uploaded to fileshares.

In terms of sheer carnage, there were over 1.1 BILLION kills registered.

Over 13 billion Credits were earned by players over the course of the beta.

IGN: What did you learn through the course of the beta that you found most surprising?

Brian Jarrard: If you ask different people on the team you'd get different answers but I think one of the high level surprising takeaways from the beta was that overall the armor abilities all worked together pretty well and there wasn't one single standout that really broke the game or was exploited by players. The most surprising bugs generally resulted from weirdness caused by host migration such as Elites spawning with Spartan armor abilities or vice versa or both teams switching roles in an Invasion game (i.e. Spartans would suddenly be on offense after host migration, Elites on defense).

IGN: On Bungie.net, there have already been some news updates about tweaks that players should expect to the final product, including changes to the grenade damage and melee attack frequency. Have you sorted out any other specific areas that you plan to address which you can share?

Brian Jarrard: There are all sorts of large, medium and small sized tweaks and bug fixes that have been or are being made to the game (and that will no doubt continue up until the end of development). One of the biggest challenges of the beta was the fact that it was a static snapshot of what was about a six week old code base at the time of launch. The timing was such that a few key bugs were fixed just after the beta was cut and there wasn't time to get it into the release.

The melee combat was in fact broken and something the team knew about before players even got their hands on the beta but couldn't be fixed in time. Other bugs like the occasional "ghosting" you'd see around your weapon or the hard to read weapon reticules (which were not final) got addressed just after the cut-off for the beta release.

While there were quite a few known issues that the team had already resolved before the beta began, there were still other things that only cropped up once the beta was in action. We heard a lot of feedback from players expressing that their headshots didn't seem to be consistently registering as expected. As it turns out, a beneficial change had been made to another area of the game that had the unintended side effect of making headshots in the lower and side areas of the hit box register less often than they should. Players were right - headshots were more difficult and unpredictable to land on an opponent and it wasn't by design nor was it due to reticule bloom.

Base movement speed and jump height were areas that many players were very vocal about. While the design intent is for the Spartan IIIs to move a bit slower and jump a bit lower than the Spartan IIs of Halo 3, the team has been testing some minor tweaks in response to beta feedback.