Plan and discuss the first Extreme Drupal Games (aka the "Golden Neckbeard" competition) for GLADCamp 2014

This year's GLADCamp conference will feature the first Extreme Drupal Games! This event will run throughout the conference, beginning on Friday, March 7th and ending when winners are announced at the closing of the conference on Sunday, March 9th.

We have a sponsor who will be donating 6 identical VMs (distro and stack TBD) for us to use during the conference. I've asked that each VM have a minimum of 2 cores and 4GB of RAM, but exact specs are still being worked out. These VMs are hosted at 1 Wilshire in Downtown Los Angeles, so there will be extremely low latency.

The plan right now is for up to 6 teams to compete over the course of the weekend, tuning both the stack and the Drupal application to get the best possible performance in the allotted time. The team with the fastest Drupal site at the end of the conference will be declared the winner.

Now that we have a date and time, the question is how we're going to load test and judge the teams. What are the rules? Do we test for both anonymous and logged in users? Will contestants be permitted to rip out pieces of the stack and install completely different ones? How about the use of premade scripts? Do we allow commercial software, or go 100% free and open source?

To be fair to all participants, I think each VM should have an identical stack and preloaded Drupal site (with a couple gigabytes of kitten photos, of course), each on their own subdomain or IP address whose location is only known to the judges and the team assigned that VM.

What do you all think? Can we do this in time? We were talking about the idea this week at the High Performance Drupal Meetup but need everyone's help and crazy ideas to make it happen. Please share your thoughts and suggestions!

Test how long a large VBO operation takes (unpublish all nodes and then republish them).

Have a sabotage test where certain modules (non critical) are removed from the file system and opcode caches are clear. Then do a logged in user load test.

Do a massive 20k concurrent anonymous user load and see how the server handles it for 1 minutes.

Test for cache freshness. Update a block that is on every page; see how long it takes for that change to go out. Update the front page and see how long that takes. Update something in a view, test it.

While all of this is going on have 5 concurrent anonymous users hitting the front page all the time just to keep things interesting. Have it max out at 100 requests per second. Incomplete and non 200 response codes will be dinged. Long page loads will be dinged as well.

Yes, absolutely. I'm not sure if our event rules will insist that the server stack remains 100% open source, but I think each team should be permitted to replace and install anything they want to create a high performance stack.

What do you all think? Should we require open source components or allow the use of commercial, closed source accelerators?

a) only open source components would be an interesting benchmark for comparison
b) commercial, closed source accelerators would give vendors an extra way to show off their products, but I don't think it would be as good for the open source community, unless there was a way to encourage them to discuss what kind of changes had been made to produce their 'super-duper' version. If someone uses commercial product, they should be encouraged to discuss it in detail at one of the High Performance meetups (maybe this would be impossible ????)

Is the Xtreme D games exclusive to LA residents or can a user from New York City compete as well or else where?

+1 for only allowing the use of FOSS in the competition as commercial software might be beneficial to those that are willing to spend money on non FOSS application and those that leverage the true power of FOSS to bend it how they want.

@christefano

I think that forcing a specific LAMP stack should not be forced but do provide all TEAM members with a 'drush arb' archive to restore with 'drush arr' and with this competitors can tweak their VM however they want, even by replacing the existing linux distro on it with a custom one i.e arch linux.

@mikeytown2 are the load test going to be performed with httperf?

w.r.t to automated testing perhaps using 'New Relic' will add some eye candy and more detail statistics to performance.

I would go with cli tools eitherway. This competition sounds like it will be great!

I think it might be a good idea to only publish the details of the competition once it opens. Since performance optimisation is almost always unique to the challenges of a particular site/use case, a competition should also reflect that. That way the competition is both a reflection of drupal's capabilities and the winning team's skill level and is not just some generic benchmarking exercise. This would also allow for the competition to be run in other camps/events with different stacks, parameters, etc. and still keep the competitions interesting.

For example, a performance competition that is restricted to just pure open source options would have much different solutions from one that also allowed commercial solutions. One that favours the best performance for anonymous users (most successful page requests) would be quite different to one that favours the best performance for logged in users (highest level of concurrent users). Even highlighting different types of testing for different use cases would be important/interesting.

Thanks again, Josh, and I'm very excited that you and the team will be coming down for GLADCamp! I missed your keynote at SANDcamp earlier this year and I'm looking forward to your keynote at this year's GLADCamp.

Any team can download a fully functional 30 day license here and use our customer support for free. Our support employees are former Linux sysadmins, or used to work for Varnish or Citrix. However, for a "fair" competition we will treat any team the same as any prospective client: support will be limited to 2 hours of phone, Skype or email support.

I'm confident that we can outperform any (commercial) caching solution on the market. A competition like this would be an excellent show-case.

It's fine if companies want to showcase their commercial solutions, but as we only have 6 free VPS's available, they should be given to teams using entirely open source operating systems and application stacks.

The commercial solution providers can purchase their own identical VPS for the competition and design a stack using whatever combination of open-source and commercial products they wish.

This will provide the widest variety of solutions for the competition.

This is a great idea! I think it should also be a requirement to fully and properly document all the things that are added, tweaks that are made etc. It would be great if the specs and test results could be posted somewhere (here, GLADcamp site) for the rest of us to learn from.

Hi everybody, we're about to put the finishing touches on the rules for the Extreme Drupal Games.

We will not be announcing all the rules until the start of the games in the morning hours of March 8th, but we will have a few strict guidelines to share when we post the event announcement in the next few days.

We're playing around with the idea of allowing one remote team to participate (think of it as the "play from home" edition of the game). So, if anyone out there wants to put a team together to participate and compete with our teams at GLADCamp, start your engines.