Nginx Guts

Industrial revolution made a profound impact on our perception of time.

Before the pendulum clock was invented around middle of the seventeenth century people perceived time differently. Time was not something related to a periodic rhythm, but rather solar activity. If you asked somebody “what time is it now”, you’d get an answer like: “the sun is still very high, so I still have time to harvest my crops”.

Nginx Essentials is available for pre-order! This is my first book ever and my first book about Nginx. The book is conceived as “programmer’s view of Nginx administration” and intends to enrich web masters’ and site reliability engineers’ knowledge with subtleties known to those who have deep understanding of Nginx core.

At the same time, Nginx Essentials is a ‘from the start guide’ allowing newcomers to easily switch to Nginx under experienced guidance.

I’ve enjoyed writing this book and it’s been a great re-solidification of my knowledge and I hope you’ll enjoy reading it as much as I do enjoy writing it!

And so, 2 weeks after I’m about to complete my first project on my own. I’m working on a distributed cache module for Nginx that stores cached content in Redis. Redis is a superfast persistent key-value storage that is easy to integrate and easy to work with.

Imagine that you run an online service company and you do some service for people and your business becomes so big that your departments no longer fit in entire floors of a big building and your budget is so big, that you don’t give a damn of caring about people and raising their salaries, because you can hire someone else of the same level the next day. Well, in fact you don’t give a damn of people’s level at all, because they write throw-away code anyways and they do jobs that can be trained for in no time anyways.

Queueing systems manage and process queues of jobs. The presence of a queue implies that something cannot be done immediately and one has to wait until some resource is available. When you respond to an HTTP request you usually want to do it interactively, that is, you want to respond within some reasonable amount of time. What can prevent you from doing this? Various things: external data sources,long-running tasks. Your request might consume too much memory and the system might decide to swap out something, which is time-consuming.

From web app’s point of view external data sources are out of influence: there is no way you can make external data sources respond faster other than by making external data sources respond faster. For example, if you are unsatisfied with how long a MySQL database responds to certain query, fixing this requires changing something in the database. Fixing the query would produce a new query, which is a different story.

Few months ago I shut down the pages of nginx socketlog module and nginx redislog module. This is due to excessive volume of support they required. Some people, however, found that these are interesting pieces of technology and I got several requests to release them.

So how do you store a lot of data if there is already over your head? The simplest answer is: partition horizontally, in other words divide and conquer. In horizontal partitioning each data instance is stored on a single storage node, so you can increase your total capacity by adding more nodes.

For horizontal partitioning to work data instances have to have no dependency between each other. If this condition is satisfied, there is no need to visit multiple storage nodes in order to reconcile a single data instance.

Long time ago I disabled comments because I was getting a lot of spam. Apparently this was man-made spam, because they did go through CAPTCHA. Today I installed Social and also added Twitter and Facebook accounts.