New Relic Ruby Agent 3.5.5.38 – Come and Get It!

The 3.5.5.38 release of the New Relic Ruby agent is our biggest one in a while. It includes:

* Thread Profiling
* Rails 4 Support
* Audit Logging

Thread Profiling
Concurrency has been a hot topic in the Ruby community recently. From the threading support of Ruby implementations like JRuby and Rubinius, to frameworks like Celluloid, tapping into concurrency is on Rubyists’ minds.

Ruby 1.9.x FTW
One of the most important features of Ruby 1.9 was the change from green threads to OS-backed threads. Before this, MRI essentially implemented a time-sharing scheme for threads in the interpreter. MRI 1.9 instead relies on native threads, turning over responsibility for that scheduling to the underlying operating system. However, MRI still has the GIL (Global Interpreter Lock), which means that life proceeds in a mostly synchronous fashion. But it is a step toward a better threading story.

Whether your code is concurrent or not, the backtrace method is worth a closer look. This small addition to Ruby 1.9’s thread class looks something like this:

We get back an array of strings with each line representing a frame in our call stack. But we aren’t limited to the current thread. Thread also exposes a list method so we can enumerate the Ruby threads in a process.

A Tasty Sampling
By mixing these two methods, we can peek into an app to see how it’s running. Here’s a recipe for a simple profiler.

1. Spawn a thread to run our profiling. (Yo dawg, I heard you like threads …)
2. On that thread, grab the Thread.List periodically and collect backtraces for each of them.
3. Aggregate the backtraces to figure out which methods show up the most.

Now some of you are probably thinking, “But that doesn’t catch everything.” Well, you’re right. This approach to profiling is known as sampling. It relies on statistics and a significant run time to gather meaningful data. While our Ruby agent instruments specific method to track every call to them (like database queries), watching every method this way would be prohibitively expensive.

Thankfully, it doesn’t take much sampling to build a useful picture of your app’s performance. For a reasonably trafficked app, sampling every 100 milliseconds over the course of two to 15 minutes does the trick. Taking the thread backtraces is only on the order of about a millisecond. So running that often isn’t a problem, even in production.

The Gotchas
As with many things in life, threads in Ruby aren’t without their issues. Since Thread.backtrace wasn’t introduced until 1.9, the venerable 1.8.7 and Ruby Enterprise Editions can’t do this type of profiling. There were also stability issues on MRI 1.9.1 that led us to recommend profiling only on MRI 1.9.2 and above.

JRuby is a huge part of the threaded Ruby market. And we’ve seen a few issues there as well. Under certain circumstances, JRuby’s finalizer thread gets adopted as a Ruby thread, but throws a Java exception when backtraced. There have also been cases where the backtraces appear munged in odd ways. We’re actively working on these issues and are looking into alternatives for gathering stable backtraces in JRuby.

The Payoff
New Relic already supports thread profiling for Java and Python, so our web application was primed and ready for the aggregated backtraces. Here’s what a profile trace looks like:

Rails 4 Support
A new version of Rails is on its way. As of this writing, Rails 4.0 is still in beta. But we don’t want you to lose New Relic while trying out the shiny new Rails goodness! To that point, this latest version of the Ruby agent should work properly with the Rails 4.0 beta. We’ll continue to test as Rails approaches release, but let us know if you see any issues. We want New Relic to work out of the box on any new version of Rails.

Audit Logging
As a SaaS business, our customers entrust their data to us. To help maintain that trust, we’ve added audit logging to the agent. This feature logs all communication between New Relic’s service and the agent in a human readable form so you can see what’s being sent. As an added benefit, it should make it easier to debug the rare cases where the agent sends malformed or improperly formatted data. See the Ruby Agent Audit Log documentation for all the details.