We're going to carve away a high-performance service from a very simple node app. We'll do load testing to verify our bottleneck, create a protobuf file to clearly communicate the service boundary, and then integrate a golang server implemented with an RPC library called Twirp.

Throughout October 2017, we ran the first season of Productivity Quest — our Slack-based series of challenges tailored to improve the way we work. It turned out to be a huge success and kept the community wanting more. This year, we decided to take a different spin.

Reddcoin was designed to be a social currency to credit creators and others that might be losing to the ever-changing advertisement industry. We're going to see how to work with RDD coins in the sense of sending, managing, and maintaining these coins with Node.js and JavaScript.

Many articles have been written about refactoring. What I'm trying to do here is to bring to light real life's example of how together with my team we approached the problem and how do we plan to deal with it.

Tremendously fast Magento

Magento is one of the most popular PHP free e-commerce engines available on the market. It’s designed to handle and manage almost all of the common issues and features purposed for online shops, and just because of that the Magento became a really huge, heavy and slow tool.

Of course, its authors decided to create a custom cache system inside the framework, but it’s not as fast as it supposed to be. In our last project, an online shop viners.co.uk (developed in cooperation with chilid) the application could only handle several requests per second at the beginning, which is not a satisfying result when you know more details about expected website traffic- a few hundred users at once.

Based on our previous experience we decided to use Varnish as a main cashing system. We chose it because the lion’s share of the website consists of a static content, well… almost everything except for user’s basket, checkout, accounts and products availability.

Introducing Varnish

Varnish is a http server accelerator which should work as a first layer and proxy (reverse proxy cache to be precise) between end-users and Apache/nginx server. That means, Varnish handles every request at http port 80 and decides if it should be delivered from saved cache or from Apache. When Varnish is unable to find any result in resources then it calls Apache itself and returns the request for the cache and for the user.

Kris Wallsmith, during his presentation on last SymfonyCon, said that when you can see users’ requests and not Varsnish in Apache then there has to be something wrong with your configuration.

At this point it may seem that Varnish cashes only full pages. But, it works integrally with ESI and that feature allows you to use special HTML tags inside your code where you declare blocks which should be cashed. As the result you can cache only selected areas on your site without caching the rest. But be careful, each ESI tag is treated by Varnish as a new request to Apache, so they should have different expiration times.

Varnish in Magento

We did a huge research to avoid implementing a connection between the Magento and the Varnish. And I must admit, it was challenging to find a proper plugin. But finally, we’ve found it: Turpentine. From the get-go this module looked reliable and accurate for us.

Installation and configuration

Installation is quite simple, but configuration brings a lot of problems – the plugin needs to be installed and enabled in Magento administration panel and only then we will be able to configure.

First of all, there exists no proper naming and when we saw backend host in Magento plugin configuration and in Varnish config file in the server, we thought that’s the same thing…but no, it isn’t. It is the http port where the Varnish is working and handling request. In our case, in Magento plugin configuration we have to indicate port 80 as backend host option and then it equals to Varnish’s configuration in /etc/default/varnish file, where -a option determines a listened port:

Next step is changing Apache port from 80 do 81, because two different services can’t listen the same port. In our case we edited /etc/apache/ports.cfg file and changed two lines:

NameVirtualHost *:81
Listen 81

Afterwards, we had to reconfigure our virtual hosts in the same way – . After restarting both services our server should start cooperation with Varnish, and only Varnish will be able to call a request to Apache.

Varnish VCL

In default configuration /etc/varnish/default.vcl file seems to be uncomplicated and you may think that it needs only a port change. Well, you’re so wrong just as we were. You need to download the vcl file from Magento Cache Management (you can find this page in System tab in menu) and download the proper file.

After file swapping and restarting Varnish, its cache needs to be enabled on the same page in administration panel.

And that’s it – now almost full content is cached. The caching system skips reuqest with POST data sent and called from https protocol. If you don’t want to cache some of the boxes you can define them in Magento XML configuration. Please find more details in official plugin’s documentation.

Results

With default Magento cached the application handled about 60 requests per second during ab test, and after enabling Varnish that value increased to 950 requests! The end-user can feel the difference too – Firebug reports that the main page is loading about 1.7 seconds without Varnish, and 134 ms with.

It’s tremendously fast!

Conclusion

The Varnish cache seems to be a good solution for caching pages where most content is static or can be synchronized at intervals. It cuts the loading time and increases performance about ten times without overloading server’s resources.

Magento is one of the most popular PHP free e-commerce engines available on the market. It’s designed to handle and manage almost all of the common issues and features purposed for online shops, and just because of that the Magento became a really huge, heavy and slow tool.

Of course, its authors decided to create a custom cache system inside the framework, but it’s not as fast as it supposed to be. In our last project, THE e-commerce shop viners.co.uk (developed with chilid for Oneida Ltd.) the application could only handle several requests per second at the beginning, which is not a satisfying result when you know more details about expected website traffic- a few hundred users at once.

Based on our previous experience we decided to use Varnish as a main cashing system. We chose it because the lion’s share of the website consists of a static content, well… almost everything except for user’s basket, checkout, accounts and products availability.