Performance

Spark is the new kid on the block when it comes to big data processing. Hadoop is also an open-source cluster computing framework, but when compared to the community contribution, Spark is much more popular. How come? What is so special and innovative about Spark? Is it that Spark makes big data processing easy and much more accessible to the developer? Or is it because the performance is outstanding, especially compared to Hadoop?

This article gives an introduction to the advantages of current systems and compares these two big data systems in depth in order to explain the power of Spark.

Yesterday my colleagues and I ran an awesome workshop at the MIT conference in which we built a Rube Goldberg machine using Scrum and Extreme Engineering techniques. As agile coaches one would think that being an Agile team should come naturally to us, but I'd like to share our pitfalls and insights with you since "we learned a lot" about being an agile team and what an incredible powerful model a Rube Goldberg machine is for scaled agile product development.

Last week I had an interesting course by Roger Sessions on Snowman Architecture. The perishable nature of Snowmen under any serious form of pressure fortunately does not apply to his architecture principles, but being an agile fundamentalist I noticed some interesting patterns in the math underlying the Snowmen Architecture that are well rooted in agile practices. Understanding these principles may give facts to feed your gut feeling about these philosophies and give mathematical proof as to why Agile works.

The summer holidays are over. Fall is coming. Like the start of every new year, a good moment for new inspiration.

Recently, I went twice to the Boston area for a client of Xebia. I met there (I dislike the word “assessments"..) a number of experienced Scrum teams. They had an excellent understanding of Scrum, but were not able to convert this to an excellent performance. Actually, there were somewhat frustrated and their performance was slightly going down.

So, they were great teams, great team members, their agile processes were running smoothly, but still not a single winning team. Which left in my opinion only one option: a lack of Spirit. Spirit is the fertilizer of Scrum and actually every framework, methodology and innovation. But how to boost the spirit?Read more

As an Agile coach, scrum master, product owner, or team member you probably have been in the situation before in which more work is thrown at the team than the team has capacity to resolve.

In case of work that is already known this basically is a scheduling problem of determining the optimal order that the team will complete the work so as to maximise the business value and outcome. This typically applies to the case that a team is working to build or extend a new product.

The other interesting case is e.g. operational teams that work on items that arrive in an ad hoc way. Examples include production incidents. Work arrives ad hoc and the product owner needs to allocate a certain capacity of the team to certain types of incidents. E.g. should the team work on database related issues, or on front-end related issues?

If the team has more than enough capacity the answer is easy: solve them all! This blog will show how to determine what capacity of the team is best allocated to what type of incident.

In a previous blog I discussed the speed of the Spray-can web-server and mentioned some measurements I did. My co-worker Age Mooij, committer on the Spray project, pointed me at 'weighttp' (see weighttp at github) a tool for benchmarking web servers. Cool! Of course I now had to do more experiments and so I did. I found out Spray-can is way too slow for my purposes and here's why.Read more

In this blog I will show a way to do performance testing with Selenium. The reason I use Selenium for performance testing is that some applications use proprietary protocols between the application layer in the browser and the server.

So just capturing the traffic between the server and replaying modified traffic is not that simple.

An example is testing GWT applications. In a previous blog I wrote why this is difficult.

To create a test script in Selenium the first thing I do is record a test with Selenium IDE
After recording a script I export the script to JUnit3 (Remote Control). This will generate a JUnit test script which can be run to test the application.

The next thing you need is a solution to run a lot of JUnit test cases at the same moment.

The following story I find really interesting.
Bhaskar Sunkara, VP Product Experience at AppDynamics, presents "Performance on Amazon AWS".
Among other things, AppDynamics is known to be used by NetFlix to monitor thousands of JVMs on Amazon AWS.

The presentation tells you about how Netflix partnered with App Dynamics to monitor their systems in a (more than) dynamic cloud infrastructure by AWS.
A large part of the presentation explains about what the important design principles are to be aware of when developing for the cloud.

Main message: machines will come and go, systems will go down. Application availability is in the hands of the developer. Develop for availability.