The main goal of this article was to find the best framework for a very basic but highly loaded Ruby application. This is the updated version of the comparison that was first posted in Jun 2013. Now we ran all the tests again, using the latest versions of Sinatra, Padrino, Goliath, and RoR. Unfortunately, the Espresso framework that we had tested last time disappeared from all the repositories, so it is no longer included.

All the tables below contain data on the time it takes to process 1,000 requests.

Sinatra 1.4.4

Web Server

No Views and DB

Views (Slim)

Views (Slim) + MySQL (Sequel)

WEBrick 1.6.1

3.663 s

8.594 s

8.931 s

Thin 1.6.1

0.626 s

5.730 s

7.159 s

Unicorn 4.8.1

1.651 s

8.240 s

8.235 s

Let’s start with Sinatra. It turned out that Unicorn works faster with MySQL and Views than with Views only, but since the difference is very small (0.005 s), this might be due to an error.

To eliminate any inaccuracies, we re-checked all the controversial results many times and got pretty stable values. So, you should be extremely careful when building high-load applications, because any small changes may slow them down. We recommend that you always run benchmark tests to the Web server you want to use in production.

Since Thin demonstrated the best results while keeping the application very simple, we decided to use it for the rest of the tests.

Padrino 0.11.4

Cache

No Views and DB

Views (Slim)

MySQL (Sequel)

Views (Slim) + MySQL (Sequel)

No cache

0.729 s

6.663 s

1.388 s

7.372 s

Memory

0.662 s

6.218 s

1.170 s

7.140 s

Although the Padrino framework is built on Sinatra with a lot of helpful features, like in Rails, its performance is still close to that of a clean Sinatra application. This is very good news for us.

Goliath 1.0.2

No Views and DB

Views (Slim)

MySQL (Sequel)

Views (Slim) + MySQL (Sequel)

2.075 s

7.477 s

4.314 s

10.184 s

The results for Goliath look a bit strange, because we expected this framework to be the fastest, thanks to its own HTTP server. If you test it and get different results, please do not hesitate to mention that in the comments and/or propose your version of the benchmark test.

Rails 4.0.2

No Views and DB

Views (Slim)

MySQL (Sequel)

Views (Slim) + MySQL (Sequel)

1.539 s

1.790 s

2.248 s

2.501 s

We also decided to test Rails 4.0.2. For these tests, we enabled caching classes in development.rb and disabled sessions and protection from forgery. We also used the Sequel database toolkit instead of ActiveRecord to make the results more trustworthy. As you can see, they turned out to be close to Goliath’s, but slightly better.

You can take a look at the applications I created for these tests to make sure they were really similar. Hope this article helps you to choose the right Ruby framework. Feel free to leave your comments below.

p.s. Special thanks to Alexander Kuntsevich, our Ruby developer, for his assistance.

Download a more detailed version of the comparison (with 10 diagrams) here:

About the author

Eugene Melnikov is a senior developer at Altoros with 5+ years of experience in Ruby, Rails, AngularJS, and Node.js. His professional interests also include сloud computing and automated testing. You can find him on GitHub.

Did you make any attempt to test these frameworks in production mode, the performance implications in rails at least are huge…

Eugene Melnikov

> Ed> Did you make any attempt to test these frameworks in production mode, the performance implications in rails at least are huge…

Ed, most likely, the results at large scale will be similar, more or less. Though, obviously, they may differ in the production mode (depending on configuration, load, etc.), ratios will probably stay the same. That’s what matters most when comparing performance of several products, isn’t it? 😉 We’re also working on updating this benchmark with more accurate figures and diagrams, so stay tuned.

beyondludus

I believe performance of Goliath is also dramatically improved if the “production” environment is used.

Andres Colon

Would like to see this comparison.

brauliobo

Agree, it makes no sense to benchmark on development.

Owen

So Sinatra with Slim and MySQL took 7.159 s and Rails took 2.501 s? Is that correct? Unless I’ve misread something I am extremely surprised that Rails beat Sinatra.

Eugene Melnikov

Hi Owen. There is nothing strange about these results. A lot depends on how you run the applications and how you configure the Web server for RoR. If you would like to repeat the tests, you can find the applications we used here: https://github.com/melnikaite/ruby-frameworks-comparison/branches

Surely there would be some changes, but these versions are just a little bit different from those described in the post. I think that we should better wait for some major updates and then repeat the tests again.

Darío Javier Cravero

Hi Eugene,

Thanks for your benchmark :).

Padrino core dev here. The gap from 0.11.4 to 0.12.3 (released yesterday) is quite big. We’ll get our versioning right as we reach 1.0.

Regards, Darío

Eugene Melnikov

Hi Dario,

Thanks for the information. We’ll be following new Padrino releases closely.

Jerome Lanteri

Hi Eugene, i read your comment and trust you… so 2 years after, now rails-5 also, were are the new benchmark tests ? you said something, where are the act of promises ? show us please.

Captain Bats

Jerome, you are free to run your own benchmarks if you need them, no need to wait for someone else to do it and document it for you.

Jerome Lanteri

you totaly miss-understand the message. I try again for you specially (hope you will appreciate): The article talk about tests when rails will be update. Tehn, i commented for show that the credibility of the writer of this article is (in facts) engaged and now, it is also passed and contents/promises has to be relative to the facts of acts. It is an objective comment. But… yes, thank you “Captain Bats”, do you really think that i don’t know i can do my own tests ? or do you think now that in fact, you not understood my comment ? hope this would help you to understand better (it looks like you not read the end: “you said something, where are the act of promise ?”). I think Captain Bats that if you read all the comment you should be able to easily understand and have an answer who should be effective. good luck…

Dezso Papp

Great to see Padrino (my fav) on top. Yet, you have misstyped its name in the first paragraph (Podrino) 😉

Olga Kurylionak

Corrected. Thanks for noticing!

Eugene Melnikov

New comparison is here https://blog.altoros.com/performance-comparison-of-ruby-frameworks-app-servers-template-engines-and-orms-q4-2016.html

Subscribe to new posts

Subscribe to new posts

Get new posts right in your inbox!

Follow us

About Altoros

Altoros is a 300+ people strong consultancy that helps Global 2000 organizations with a methodology, training, technology building blocks, and end-to-end solution development. The company turns cloud-native app development, customer analytics, blockchain, and AI into products with a sustainable competitive advantage. Altoros assists enterprises on their way to digital transformation, standing behind some of the world's largest Cloud Foundry deployments.