Debian packages no longer require Ruby 1.9

Due to a bug in the package specifications, Debian packages used to require Ruby 1.9, even if you already have newer Ruby versions installed from APT (e.g. through the Brightbox repository). This bug has now been fixed. The Phusion Passenger Debian packages now require some Ruby interpreter, but it doesn’t care which version.

Phusion Passenger 4 is the current stable branch, in which we release bug fixes from time to time. At the same time there is also Phusion Passenger 5, which is the not-yet-ready-for-production development branch, with major changes and improvements and terms of performance application behavior visibility. Version 5.0 beta 3 will soon be released, but until the 5.x branch is considered stable, we will keep releasing bug fixes under the 4.x branch.

Improved Ruby 2.2 support

Version 4.0.56 already introduced Ruby 2.2 support, but due to an issue in the way we compile the Phusion Passenger native extension, it didn’t work with all Ruby 2.2 installations. In particular, 4.0.56 worked with Ruby 2.2 installations that were compiled with a shared libruby, which is the case if you installed Ruby 2.2 with RVM or though operating system packages. But it did not work with Ruby 2.2 installations that were compiled with a static libruby, which is the case if you installed manually from source, or using rbenv and chruby, or when you are using Heroku.

At first, we suspected a bug in Ruby 2.2’s build system, but after feedback from the MRI core developers, it turned out to be an issue in our own build system. The issue is caused by a commit from 4 years ago, GH-168, which attempted to fix a different issue. It seems there is no way to fix Ruby 2.2 compatibility while at the same time fixing GH-168, so we had to make a choice. Since GH-168 is quite old and was made at a time when Ruby 1.8.6 was the latest Ruby version, we believe that the issue is no longer relevant. We reverted GH-168 in favor of Ruby 2.2 compatibility.

We’ve just released version 4.0.56 of the Phusion Passenger application server for Ruby, Python and Node.js, which fixes a number of interesting and important bugs. They’re the kind of typical bugs that make me go “what the **** was I thinking?!” after I’ve analyzed them, because the fixes are very simple.

Leaking file descriptors

The first bug is a file descriptor leak. A file descriptor is number which represents a kernel resource, such as an open file or a socket. Every time you call File.open or TCPSocket.new in Ruby, you get an IO object that’s internally backed by a file descriptor. In Node.js you even often work with file descriptors directly: most fs functions return a file descriptor. Since Phusion Passenger is written in C++, we also work with file descriptors directly.

Schematically, it looks like this:

You’re probably familiar with memory leaks. A file descriptor leak is very similar. If you ever lose track of a file descriptor number, you’ve leaked it. In Ruby this is not possible because all IO objects own their file descriptor, and IO objects are garbage collected (thus closing the corresponding file descriptor). However in Node.js and in C++ this can easily happen if you’re not careful. When leaked, the kernel resource stays allocated until your process exits.

What went wrong

In Passenger, we leaked a file descriptor when creating an error report file. This file is created if your app can’t spawn for some reason (e.g. it throws an exception during startup). The code that was responsible for rendering the file looked like this, in semi C++ pseudocode:

Notice the guard variable. In C++, it is a so-called RAII object: “Resource Acquisition Is Initialization”. It is a common coding pattern in C++ to ensure that things are cleaned up when exceptions are thrown, kind of like the C++ equivalent of the ensure keyword in Ruby or the finally keyword in Javascript. When this function exits for any reason, be it a normal return or an exception, the guard destructor is called, which is supposed to close the file descriptor.

The facepalm moment was when Paul “popox” B reported that the guard was on the wrong line. The guard was created before the file descriptor was assigned to fd, so the guard did nothing all this time. Every time a report file was created, a file descriptor was leaked.

Node.js load balancing

The other issue fixed in 4.0.56 is a Node.js load balancing issue. In Passenger we load balance requests between application processes as much as possible. Traditionally, the reason for load balancing has been to minimize latency. This utilizes the concept of “application concurrency”: the maximum number of concurrent requests a single app process can handle. For Ruby apps, the concurrency is 1 (unless you configured multithreading, in which case the concurrency is equal to the number of threads). Since Ruby apps have finite I/O concurrency, Passenger load balances a request to a different process only if one process has run out of concurrency.

Node.js is different in that it’s fully asynchronous. It can effectively have an unlimited amount of concurrency.

Passenger orders processes in a priority queue by “busyness”. Load balancing is achieved by routing a new request to the process with the least busyness.

What went wrong

What went wrong with the Node.js case is the fact that we had special rules for application processes with unlimited concurrency. The busyness for such processes is calculated as follows:

if (sessions == 0) {
return 0;
} else {
return 1;
}

sessions indicates the number of requests that a process is currently handling. This piece of code effectively sorted Node.js processes in two categories only: idle processes and non-idle processes.

From a concurrency point of view, there is nothing wrong with this. Node.js apps have unlimited concurrency after all. However this resulted in lots of requests “sticking” to a few processes, as Charles Vallières reported:

Then it dawned to me that I forgot something. An even distribution of requests is desirable here, because now the reason for load balancing becomes different. It’s to maximize CPU core usage, because single Node.js process can only use 1 CPU core.

Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Phusion Passenger is under constant maintenance and development. Version 4.0.55 is a bugfix release.

Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Phusion Passenger is under constant maintenance and development. Version 4.0.53 is a bugfix release.

Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Phusion Passenger is under constant maintenance and development. Version 4.0.52 is a bugfix release.

Recent changes

Version 4.0.50 and 4.0.51 have been skipped because they were hotfixes for Enterprise customers. The changes in 4.0.50, 4.0.51 and 4.0.52 combined are as follows:

Fixed a null termination bug when autodetecting application types.

Node.js apps can now also trigger the inverse port binding mechanism by passing '/passenger' as argument. This was introduced in order to be able to support the Hapi.js framework. Please read this StackOverflow answer for more information regarding Hapi.js support.

It is now possible to abort Node.js WebSocket connections upon application restart. Please refer to this page for more information. Closes GH-1200.

Passenger Standalone no longer automatically resolves symlinks in its paths.

passenger-config system-metrics no longer crashes when the system clock is set to a time in the past. Closes GH-1276.

passenger-status, passenger-memory-stats, passenger-install-apache2-module and passenger-install-nginx-module no longer output ANSI color codes by default when STDOUT is not a TTY. Closes GH-487.

passenger-install-nginx-module --auto is now all that’s necessary to make it fully non-interactive. It is no longer necessary to provide all the answers through command line parameters. Closes GH-852.

Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Phusion Passenger is under constant maintenance and development. Version 4.0.49 is a bugfix release.

Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Phusion Passenger is under constant maintenance and development. Version 4.0.48 is a bugfix release.

Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Phusion Passenger is under constant maintenance and development. Version 4.0.46 is a bugfix release.

Logging of application spawning errors has been much improved. Full details
about the error, such as environment variables, are saved to a private log file.
In the past, these details were only viewable in the browser. This change also
fixes a bug on Phusion Passenger Enterprise, where enabling Deployment Error
Resistance causes error messages to get lost. Closes GH-1021 and GH-1175.

Passenger Standalone no longer, by default, loads shell startup files before
loading the application. This is because Passenger Standalone is often invoked
from the shell anyway. Indeed, loading shell startup files again can interfere
with any environment variables already set in the invoking shell. You can
still tell Passenger Standalone to load shell startup files by passing--load-shell-envvars. Passenger for Apache and Passenger for Nginx still
load shell startup files by default.

If you are a Union Station customer, then
Phusion Passenger will now also log application spawning errors to Union Station.
This data isn’t shown in the Union Station interface yet, but it will be
implemented in the future.

Minor changes:

The Python application loader now inserts the application root into sys.path.
The fact that this was not done previously caused a lot of confusion amongst
Python users, who wondered why their passenger_wsgi.py could not import any
modules from the same directory.

Fixed a compatibility problem with Django, which could cause Django apps to
freeze indefinitely. Closes GH-1215.

Fixed a regression in Node.js support. When a Node.js app is deployed on
a HTTPS host, the X-Forwarded-Proto header wasn’t set in 4.0.45.
Closes GH-1231.

Passenger Standalone now works properly when the HOME environment variable
isn’t set. Closes GH-713.

Passenger Standalone’s package-runtime command has been removed. It has
been broken for a while and has nowadays been obsolete by our automaticbinary generation system.
Closes GH-1133.

The passenger_startup_file option now also works on Python apps. Closes GH-1233.

Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Phusion Passenger is under constant maintenance and development. Version 4.0.45 is a bugfix release.

Recent changes

Major improvements in Node.js and Meteor compatibility. Older Phusion Passenger versions implemented Node.js support by emulating Node.js’ HTTP library. This approach was found to be unsustainable, so we’ve abandoned that approach and replaced it with a much simpler approach that does not involve emulating the HTTP library.

Introduced support for sticky sessions. Sticky sessions are useful — or even required — for apps that store state inside process memory. Prominent examples include SockJS, Socket.io, faye-websocket and Meteor. Sticky sessions are required to make the aforementioned examples work in multi-process scenarios. By introducing sticky sessions support, we’ve much improved WebSocket support and support for the aforementioned libraries and frameworks.

Due to user demand, GET requests with request bodies are once again supported. Support for these kinds of requests was removed in 4.0.42 in an attempt to increase the strictness and robustness of our request handling code. It has been determined that GET requests with request bodies can be adequately supported without degrading robustness in Phusion Passenger. However, GET requests with both request bodies and WebSocket upgrade headers are unsupported. Fixes issue #1092.

“Phusion” and “Phusion Passenger” are registered trademarks of Phusion. “Rails”, “Ruby on Rails” and the Rails logo are registered trademarks of David Heinemeier Hansson. All other trademarks are property of their respective owners.