Random little things I've learned

The mysterious “memory leak”

Recently at my workplace, we started to migrate one of our projects from
Rails 2 to Rails 3.1. Everything seemed to go pretty well. Then one day,
I started noticing my development machine crawling to a halt for no
apparent reason. I could see that the memory was full and swapping. My
initial reaction was that some sort of memory leak was introduced during
the Rails 3 migration, as I also discovered that another colleague was
experiencing the same issue. After going through a quick process of
kill -9 elimination, the culprit turned out to be Passenger
Standalone.

Checking Passenger’s memory profile

Some quick googling turned up the command passenger-memory-stats. The
tool shows details regarding the Apache/Nginx processes’ memory profiles,
as well as the Passenger processes’ memory profiles. I quickly realized
that passenger uses 6 worker processes by default.

One thing that my team quickly realized during the Rails 3 migration was
that Rails 3 has a much larger overhead in development mode than Rails
2. This quickly lead to a substantial amount of the dev machine’s
memory being utilized when passenger was run and a few requests were
made. In my case, I had also started using gnome-shell, which took up a
good chunk of memory, and that compounded with 6 memory-intensive workers
meant a machine running out of memory.

I ended up spending some time to find out documentations on how to
configure the number of worker instances. Much to my surprise, I was
not able to find any information on how to do this for standalone
Passenger.

It was time to start digging into Passenger’s internal’s.

Discovering how to configure nginx for standalone Passenger

A quick grep for passenger_max_pool_size under the passenger gem root
directory turned up this file: phusion_passenger/templates/standalone/config.erb.
In the file, I quickly saw these two hopeful looking lines:

phusion_passenger/templates/standalone/config.erb

12

passenger_max_pool_size <%=@options[:max_pool_size]%>;

passenger_min_instances <%=@options[:min_instances]%>;

I proceeded to replace the two erb tags with 2 and 1 respectively.
After restarting passenger and running passenger-memory-stats, I saw
only two passenger workers. Success!!

Looking for a better solution

Modifying the nginx config template worked. However, modifying the gem
directly was not an ideal solution. One option that I had was to fork
the gem and modify the template, but that also did not seem like a clean
solution for this problem. So I started digging a bit further into
passenger. A grep for ‘config.erb’ turned up this line:
lib/phusion_passenger/standalone/command.rb:179: template_filename = File.join(TEMPLATES_DIR, "standalone", "config.erb")

How I got to learn vim registers

One of the things that bugged me the most when I was learning Vim was
copy and pasting.

For example, I would copy a line with: yy. Then
I would go to the section I wanted to put the yanked line, and of
course, I would see a line that I had to delete first, so without
thinking, I would type: dd. I would then quickly realize that
I had lost the line that I copied, go up to the copied line, yank the
line again, go down to the section again, and finally paste it.

Another case where copy and pasting bugged me was copying to/from the
system clipboard. For example, I had no idea how to copy and paste
between different Vim windows. I also had no idea how to copy from
external pages into Vim other than highlighting the text from the
external page, going into insert mode in Vim, and middle-clicking the
mouse.

After feeling this limitation countless times, I finally decided to
learn how the Vim registers worked, and how I could copy and paste at
will.

Vim registers

Vim has something called registers, where various strings are stored,
depending on the context and what actions you have taken.

For example, the ” register is Vim’s default register, so when you yank
or delete lines, they go into that register by default. The % register
holds the relative path of the current file. You can view the contents
of these registers by typing :reg

You can perform actions with these registers by typing “, followed by
the register name, then the action you would like to perform.

For example, if you would like to paste the current file’s relative path
twice, you would type "%2p. You can achieve the effect of the
usual p with ""p.

The null register

At this point, you’re probably wondering how this helps with the first
issue I had with Vim, which is how I can avoid overwriting the content
that I yanked in Vim. This is solved by using the _ register, which
is Vim’s null register.

Let’s say you’ve copied a line with yy,
and you have a line that you want to delete without losing the line
you’ve yanked. You can do that by typing "_dd. Want to delete
lines 38 to 43 without overwriting the yanked line? No problem with
:38,43d_. Alternatively, you can select those lines in visual
mode, and type :d_.

Copy and pasting to and from the system register

To solve the second issue I had, I used two other special Vim registers,
namely the * and + registers. The * register is where content that you
highlight in linux operating systems go to, and the + register is where
content that you copy/cut with Ctrl+C or Ctrl+X go to.

Now given the knowledge acquired from using the null register, you can
safely guess that "+p would paste whatever you’ve copied using
Ctrl+C. Copying into the clipboard is as simple(?) as "+yy,
which you can test by pasting into somewhere else with Ctrl+P. If
you copy with "*yy instead, you can paste it by middle-clicking
the mouse, just as you would do with any content you’ve highlighted in
linux.

Re-mapping the default yank, delete, and paste behaviours in Vim

I personally prefer using the Vim registers directly, so that I don’t
forget how to use them, but there are definitely a lot of people who
would prefer to have copy and pasting to behave like any other editors
out there.

In that case, you could consider re-mapping the default Vim behaviours by adding something
like the following into your ~/.vimrc: