Jekyll2020-05-16T14:59:58-04:00https://nasseri.io/feed.xmlnasseri.ioNassredean Dean Nasseridean@nasseri.iohttps://nasseri.ioI Did Some Podcasts2018-12-01T00:00:00-05:002018-12-01T00:00:00-05:00https://nasseri.io/2018/12/01/i-did-some-podcasts<p>Hello folks! I just wanted to give a PSA that I recently did a couple of podcasts. One with the fine folks of <a href="https://devchat.tv/ruby-rogues/">Ruby Rogues</a>, which you can find <a href="https://www.youtube.com/watch?app=desktop&amp;v=UNdv6Yh4I9Q&amp;list=PLJesql-aSfX7ZlZvJ7KZnyluVOVTSNHW1&amp;index=101">here</a>, wherein we discuss the open source ruby repl <a href="https://github.com/dnasseri/fir">Fir</a> I have been working on. I also did an episode of <a href="https://devchat.tv/my-ruby-story/">My Ruby Story</a>, where we talk a little bit more about the project and also about myself and who I am generally. If you are curious, go check them out!</p>Nassredean Dean Nasseridean@nasseri.iohttps://nasseri.ioHello folks! I just wanted to give a PSA that I recently did a couple of podcasts. One with the fine folks of Ruby Rogues, which you can find here, wherein we discuss the open source ruby repl Fir I have been working on. I also did an episode of My Ruby Story, where we talk a little bit more about the project and also about myself and who I am generally. If you are curious, go check them out!Introducing Fir, the Friendly, Interactive Ruby REPL2018-01-23T00:00:00-05:002018-01-23T00:00:00-05:00https://nasseri.io/2018/01/23/introducing-fir<p>I haven’t been posting here much because most of my free time has been spent working on a ruby REPL that I call <a href="https://github.com/dnasseri/fir" title="fir github page">fir</a>, which stands for friendly-interactive-ruby. The ruby community has no shortage of great REPL’s — the default irb is pretty good, and <a href="https://github.com/pry/pry" title="pry github page">pry</a> offers a fantastic repl as well as an interactive debugger that really highlights some of the great features of the ruby language. In my daily work, I also really enjoy some of the autosuggestion features that the <a href="https://fishshell.com/" title="the fish shell">fish-shell</a> offers, so much so that I got to thinking about integrating those features into a ruby repl. If your not familiar with the fish shell, this is what some of those features look like in action:</p>
<p><img src="/assets/images/posts/2/fish.gif" alt="image-title-here" /></p>
<!--more-->
<p>As you can see, fish offers as you type autocompletion based on your history, programs in your path, and directories on your machine. It’s really quite clever, and anytime I’m using another shell, I tend to miss it. Fir does something similar, by suggesting lines from your .irb_history file:</p>
<p><img src="/assets/images/posts/2/fir.gif" alt="image-title-here" /></p>
<p>Fir not only offers suggestions as you type, it also indents and dedents code as you type it. Take a look at how it handles this begin/rescue block:</p>
<p><img src="/assets/images/posts/2/raise-rescue.gif" alt="image-title-here" /></p>
<p>As you can see, fir is smart enough to dedent the rescue keyword as soon as you type it. The difference between fir and a repl like pry (or most others) is that fir parses the syntax and re-renders the screen every time it receives a key input, as opposed to only when the program receives select inputs like tab or enter. Since every keystroke triggers a loop and a full re-render, we can continuously update the screen with new indents, suggestions, and in the future, colorization. Like other repl’s, fir only triggers a full evaluation of the inputted code when it forms a self contained executable block, and the user presses enter.</p>
<h4 id="implementing-as-you-type-updates-with-raw-input">Implementing as you Type Updates with Raw Input</h4>
<p>In order to achieve the functionality described above inside our command line programs, we need to leverage “raw” terminal drivers, as opposed to what is called “cooked”, or canonical, input. Terminals and repls are usually cooked, whereby characters are buffered internally until a carriage return is inputted to the program at which point the line is processed. This “cooked” functionality is actually what allows for certain characters to be treated as special e.g. control sequences or backspace. It is on this system that programs usually base their line editing. If instead we want to manually handle characters as the program receives them, and also prevent them from being echoed to the screen, than we need to force the terminal to use a raw driver. In ruby we can force stdin or stdout into raw mode as follows:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">while</span> <span class="kp">true</span>
<span class="vg">$stdin</span><span class="p">.</span><span class="nf">raw</span> <span class="k">do</span> <span class="o">|</span><span class="n">raw_input</span><span class="o">|</span>
<span class="n">raw_input</span><span class="p">.</span><span class="nf">getc</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>If you run the above program and start typing, you will see nothing happens on the screen, and that’s good! The program is fetching the raw characters, and preventing their default behavior, which in this case, is rendering them to the screen. This is essentially what Fir is based on, fetching raw input, processing it, and then manually drawing to the screen using <a href="https://en.wikipedia.org/wiki/ANSI_escape_code" title="ansi escape code wiki">ANSI escape sequences</a> to do things like erase or move the cursor. It also means that we can handle the characters as we receive them, instead of waiting for a carriage return.</p>
<p>Now as I mentioned before, you may have noticed that trying to ctrl-d or exit the program using the standard process control characters did not work. Sorry about that! You see one of the downsides of raw terminal drivers is that you have to manually handle everything. In this case, those standard process control characters are doing nothing to end the program, so you are going to have to send that runaway ruby process a kill signal ;).</p>
<h4 id="capturing-escape-keys">Capturing Escape Keys</h4>
<p>Certain keys, such as the arrow keys, are prefixed by an “escape” sequence. These escape characters are actually not characters at all, but strings that begin with the escape character “\e”. For instance, the left arrow character is represented by “\e[D”. Let’s modify our raw input example to inspect and display the character we have trapped, and see what a left arrow key looks like.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">while</span> <span class="kp">true</span>
<span class="vg">$stdin</span><span class="p">.</span><span class="nf">raw</span> <span class="k">do</span> <span class="o">|</span><span class="n">raw_input</span><span class="o">|</span>
<span class="vg">$stdin</span><span class="p">.</span><span class="nf">puts</span> <span class="s2">"CHARACTER:"</span><span class="p">:</span> <span class="n">raw_input</span><span class="p">.</span><span class="nf">getc</span><span class="p">.</span><span class="nf">inspect</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Run the program and then hit the left arrow key, and you should see something like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">$</span> <span class="n">ruby</span> <span class="n">raw_mode_example</span><span class="p">.</span><span class="nf">rb</span>
<span class="no">CHARACTER</span><span class="p">:</span> <span class="s2">"</span><span class="se">\e</span><span class="s2">"</span>
<span class="no">CHARACTER</span><span class="p">:</span> <span class="s2">"["</span>
<span class="no">CHARACTER</span><span class="p">:</span> <span class="s2">"D"</span>
</code></pre></div></div>
<p>What happened here? Well if we read the <a href="https://ruby-doc.org/core-2.3.0/IO.html#method-i-getc" title="getc ruby docs">documentation for getc</a> we see that getc “Reads a one-character string from ios”, and this is not a one character string! This posed a substantial challenge for Fir, since the left arrow was broken down into its component characters, my program behaved as if the user had just hit escape, and then left bracket, and then d. Not what we want.</p>
<p>I stumbled into a solution <a href="http://www.alecjacobson.com/weblog/?p=75">here</a>. The solution is to spawn an additional thread when the program detects an escape key where we call getc on the input twice, and then kill the thread very quickly. If the the escape was the beginning of the long character sequence, as in the case of left arrow, we are able to trap it, and if not the thread is killed and the program isn’t blocked waiting for more input. Here’s what that solution looks like inside the <a href="https://github.com/dnasseri/fir/blob/master/lib/fir/key.rb">fir key class</a>.</p>
<h4 id="conclusion">Conclusion</h4>
<p>All in all, this has been a really fun project for me to pursue, and I have learned a ton. I am looking for any contributions, so if you would like to contribute, open a pull request, and submit a PR!</p>Nassredean Dean Nasseridean@nasseri.iohttps://nasseri.ioI haven’t been posting here much because most of my free time has been spent working on a ruby REPL that I call fir, which stands for friendly-interactive-ruby. The ruby community has no shortage of great REPL’s — the default irb is pretty good, and pry offers a fantastic repl as well as an interactive debugger that really highlights some of the great features of the ruby language. In my daily work, I also really enjoy some of the autosuggestion features that the fish-shell offers, so much so that I got to thinking about integrating those features into a ruby repl. If your not familiar with the fish shell, this is what some of those features look like in action:The Structure and Interpretation of Ruby Programs2017-08-16T00:00:00-04:002017-08-16T00:00:00-04:00https://nasseri.io/2017/08/16/the-structure-and-interpretation-of-ruby-programs<p>Whenever a ruby program is run, the program is first lexed into tokens, then the tokens are assembled into an abstract syntax tree, and finally the AST is compiled into virtual machine instructions. In this post, we will explore each of these steps in detail.</p>
<p><img src="/assets/images/posts/1/tokenize-parse-compile.png" alt="tokenize-parse-compile" /></p>
<!--more-->
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Math</span>
<span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="n">x</span> <span class="o">+</span> <span class="n">y</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">Math</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">add</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
</code></pre></div></div>
<p>Whenever you run a ruby program, ruby steps through the characters in the program one at a time and groups them into special words called “tokens”. These tokens are not guaranteed to be valid Ruby. At this point the stream of tokens could be invalid. It is the responsibility of the parser to determine whether the inputted program is valid or not.</p>
<p>We can actually see how Ruby’s tokenization would treat this program using the built in tool ‘ripper’. Lets take a look at how the definition of the add method is tokenized using the following code:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'ripper'</span>
<span class="nb">require</span> <span class="s1">'pp'</span>
<span class="n">code</span> <span class="o">=</span> <span class="o">&lt;&lt;</span><span class="no">CODE</span><span class="sh">
def add(x, y)
x + y
end
</span><span class="no">CODE</span>
<span class="n">pp</span> <span class="no">Ripper</span><span class="p">.</span><span class="nf">lex</span><span class="p">(</span><span class="n">code</span><span class="p">)</span>
</code></pre></div></div>
<p>This results in the following output:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"def"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="ss">:on_ident</span><span class="p">,</span> <span class="s2">"add"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span> <span class="ss">:on_lparen</span><span class="p">,</span> <span class="s2">"("</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">8</span><span class="p">],</span> <span class="ss">:on_ident</span><span class="p">,</span> <span class="s2">"x"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">9</span><span class="p">],</span> <span class="ss">:on_comma</span><span class="p">,</span> <span class="s2">","</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">10</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">11</span><span class="p">],</span> <span class="ss">:on_ident</span><span class="p">,</span> <span class="s2">"y"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">12</span><span class="p">],</span> <span class="ss">:on_rparen</span><span class="p">,</span> <span class="s2">")"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">13</span><span class="p">],</span> <span class="ss">:on_ignored_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="ss">:on_ident</span><span class="p">,</span> <span class="s2">"x"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="ss">:on_op</span><span class="p">,</span> <span class="s2">"+"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">5</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">6</span><span class="p">],</span> <span class="ss">:on_ident</span><span class="p">,</span> <span class="s2">"y"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span> <span class="ss">:on_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_kw</span><span class="p">,</span> <span class="s2">"end"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="ss">:on_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">]]</span>
</code></pre></div></div>
<p>Each nested array in above output represents a single token. The structure of the array is as follows:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[[</span><span class="n">line</span> <span class="n">number</span><span class="p">,</span> <span class="n">text</span> <span class="n">column</span><span class="p">],</span> <span class="n">token</span> <span class="nb">name</span><span class="p">,</span> <span class="n">characters</span><span class="p">]</span>
</code></pre></div></div>
<p>The token name symbols in the ouput map to token types defined inside of the file “parse.y” of the ruby source code. Though the symbol names are not one to one with the C definition inside parse.y, the output of ripper captures what Ruby does internally as it encounters tokens. For instance “def” is mapped to :on_kw indicating that it is ruby keyword. Similarly “end” is also mapped to :on_kw. However inside of ruby “def” would be treated as the token of type “keyword_def”, and “end” would be treated as the token of type “keyword_end”. There are some other differences in the output of ripper and the internal represenation of each token, but ripper gives us a good idea of how ruby tokenization works. It is important to remember that tokenization happens in a character by character streaming fashion. This gif should help visualize how a stream of characters is turned into a stream of tokens:</p>
<p><img src="/assets/images/posts/1/tokenization-animation.gif" alt="tokenization-animated" /></p>
<h4 id="parsing">Parsing</h4>
<p>After ruby has finished converting the text of the program into a stream of tokens, the tokens are then grouped into logical units that ruby can understand. This is the parsing step, and it is at this step that the program is determined to be valid ruby or not. Ruby does this by determining if the stream of tokens conform the grammar rules defined in the parse.y file. If so, the stream of tokens are converted into a corresponding abstract syntax tree. The AST is a data structure representation of the syntactical meaning of your ruby program. The handy ripper tool can again help us visualize what this AST looks like to ruby, this time using the “sexp” function:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'ripper'</span>
<span class="nb">require</span> <span class="s1">'pp'</span>
<span class="n">code</span> <span class="o">=</span> <span class="o">&lt;&lt;</span><span class="no">CODE</span><span class="sh">
def add(x, y)
x + y
end
</span><span class="no">CODE</span>
<span class="n">pp</span> <span class="no">Ripper</span><span class="p">.</span><span class="nf">sexp</span><span class="p">(</span><span class="n">code</span><span class="p">)</span>
</code></pre></div></div>
<p>Output:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="ss">:program</span><span class="p">,</span>
<span class="p">[[</span><span class="ss">:def</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:@ident</span><span class="p">,</span> <span class="s2">"add"</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]],</span>
<span class="p">[</span><span class="ss">:paren</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:params</span><span class="p">,</span>
<span class="p">[[</span><span class="ss">:@ident</span><span class="p">,</span> <span class="s2">"x"</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">8</span><span class="p">]],</span> <span class="p">[</span><span class="ss">:@ident</span><span class="p">,</span> <span class="s2">"y"</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">11</span><span class="p">]]],</span>
<span class="kp">nil</span><span class="p">,</span>
<span class="kp">nil</span><span class="p">,</span>
<span class="kp">nil</span><span class="p">,</span>
<span class="kp">nil</span><span class="p">,</span>
<span class="kp">nil</span><span class="p">,</span>
<span class="kp">nil</span><span class="p">]],</span>
<span class="p">[</span><span class="ss">:bodystmt</span><span class="p">,</span>
<span class="p">[[</span><span class="ss">:binary</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:var_ref</span><span class="p">,</span> <span class="p">[</span><span class="ss">:@ident</span><span class="p">,</span> <span class="s2">"x"</span><span class="p">,</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">]]],</span>
<span class="p">:</span><span class="o">+</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:var_ref</span><span class="p">,</span> <span class="p">[</span><span class="ss">:@ident</span><span class="p">,</span> <span class="s2">"y"</span><span class="p">,</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">6</span><span class="p">]]]]],</span>
<span class="kp">nil</span><span class="p">,</span>
<span class="kp">nil</span><span class="p">,</span>
<span class="kp">nil</span><span class="p">]]]]</span>
</code></pre></div></div>
<p>This output is a bit terse, so I’ve included a visual representation of the AST below. Each node in the tree represents a construct in the program. For instance, the addition of the identifier’s x and y is represented by a “binary” node. A binary node encodes an operation that performs on two elements of a set to derive a third element.</p>
<p><img src="/assets/images/posts/1/ast.png" alt="ast" /></p>
<p>The generated AST encapsulates everything ruby needs to understand your program, and with that, ruby can move on to the third and final step.</p>
<h4 id="compilation">Compilation</h4>
<p>Ruby is a compiled language in much the same way that Java is. While ruby is not compiled down to native machine code, it is compiled into a set of bytecode instructions that are interpreted by a virtual machine. In the case of Java the VM is JVM, in the case of Ruby it is YARV, which stands for “Yet another ruby virtual-machine”.
In order to compile your program, ruby recursively iterates over the nodes in the AST from the top down and compiles each node into corresponding YARV instructions. Once more, we can use built in tools to examine how ruby compiles our AST into YARV instructions.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">code</span> <span class="o">=</span> <span class="o">&lt;&lt;</span><span class="no">CODE</span><span class="sh">
def add(x, y)
x + y
end
</span><span class="no">CODE</span>
<span class="nb">puts</span> <span class="no">RubyVM</span><span class="o">::</span><span class="no">InstructionSequence</span><span class="p">.</span><span class="nf">compile</span><span class="p">(</span><span class="n">code</span><span class="p">).</span><span class="nf">disasm</span>
</code></pre></div></div>
<p>Output:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">==</span> <span class="ss">disasm: </span><span class="c1">#@&gt;================================</span>
<span class="mo">0000</span> <span class="n">trace</span> <span class="mi">1</span> <span class="p">(</span> <span class="mi">1</span><span class="p">)</span>
<span class="mo">0002</span> <span class="n">putspecialobject</span> <span class="mi">1</span>
<span class="mo">0004</span> <span class="n">putobject</span> <span class="ss">:add</span>
<span class="mo">0006</span> <span class="n">putiseq</span> <span class="n">add</span>
<span class="mo">000</span><span class="mi">8</span> <span class="n">opt_send_without_block</span> <span class="p">,</span>
<span class="mo">0011</span> <span class="n">leave</span>
<span class="o">==</span> <span class="ss">disasm: </span><span class="c1">#&gt;=======================================</span>
<span class="n">local</span> <span class="n">table</span> <span class="p">(</span><span class="ss">size: </span><span class="mi">2</span><span class="p">,</span> <span class="ss">argc: </span><span class="mi">2</span> <span class="p">[</span><span class="ss">opts: </span><span class="mi">0</span><span class="p">,</span> <span class="ss">rest: </span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="ss">post: </span><span class="mi">0</span><span class="p">,</span> <span class="ss">block: </span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="ss">kw: </span><span class="o">-</span><span class="mi">1</span><span class="err">@</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="ss">kwrest: </span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
<span class="p">[</span> <span class="mi">2</span><span class="p">]</span> <span class="n">x</span> <span class="p">[</span> <span class="mi">1</span><span class="p">]</span> <span class="n">y</span>
<span class="mo">0000</span> <span class="n">trace</span> <span class="mi">8</span> <span class="p">(</span> <span class="mi">1</span><span class="p">)</span>
<span class="mo">0002</span> <span class="n">trace</span> <span class="mi">1</span> <span class="p">(</span> <span class="mi">2</span><span class="p">)</span>
<span class="mo">0004</span> <span class="n">getlocal_OP__WC__0</span> <span class="mi">4</span>
<span class="mo">0006</span> <span class="n">getlocal_OP__WC__0</span> <span class="mi">3</span>
<span class="mo">000</span><span class="mi">8</span> <span class="n">opt_plus</span> <span class="p">,</span>
<span class="mo">0011</span> <span class="n">trace</span> <span class="mi">16</span> <span class="p">(</span> <span class="mi">3</span><span class="p">)</span>
<span class="mo">0013</span> <span class="n">leave</span> <span class="p">(</span> <span class="mi">2</span><span class="p">)</span>
</code></pre></div></div>
<p>YARV is a stack oriented virtual machine, so most of the instructions invlove putting an object onto the stack, and then executing an operation against the values on the stack. The top block of instructions are used to define the “add” method. Essentially the instructions put the method name on the stack, and then calls “define_method” a C function that is used by YARV to create a new ruby method. In the second block, a local table is defined which represent the arguments our function can accept.</p>
<h4 id="conclusion">Conclusion</h4>
<p>In this post we explored how ruby translates the text of a program, first into tokens, then into a structure called an AST, and finally into intstructions usable by the virtual machine. These three passes are what allow ruby to be interpreted by the virtual machine. Hopefully after reading this post you will have a little bit better understanding about what exactly happens when you boot up a ruby process.</p>Nassredean Dean Nasseridean@nasseri.iohttps://nasseri.ioWhenever a ruby program is run, the program is first lexed into tokens, then the tokens are assembled into an abstract syntax tree, and finally the AST is compiled into virtual machine instructions. In this post, we will explore each of these steps in detail.Understanding Posrgres Window Functions2016-06-02T00:00:00-04:002016-06-02T00:00:00-04:00https://nasseri.io/2016/06/02/understanding-postgres-window-functions<p>Recently I stumbled into a very handy feature of Postgres — window functions. A window function gives you a “window” into other rows which are somehow related to the current row. You can then preform a calculation across that set of rows. Window functions behave much like regular aggregate functions, with the caveat that the rows do not become grouped into a single row! Let’s explore this feature, and why it might be useful.</p>
<!--more-->
<p>Let’s imagine that we have an application that is used to record sales data for different stores. To keep things simple, we will say that the application’s database has only three tables: sales, users, and stores.</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>SALES STORES
+------+-----------+------------+----------+--------------+ +------+---------------+
| id | user_id | store_id | amount | entered_at | | id | name |
|------+-----------+------------+----------+--------------| |------+---------------|
| 1 | 1 | 1 | 10000.0 | 2015-11-07 | | 1 | Gucci |
| 2 | 1 | 1 | 30000.0 | 2015-11-29 | | 2 | Louis Vuitton |
| 3 | 1 | 2 | 11000.0 | 2015-11-08 | | 3 | Fendi |
| 4 | 1 | 2 | 19000.0 | 2015-11-26 | +------+---------------+
| 5 | 1 | 3 | 14000.0 | 2015-11-15 |
| 6 | 1 | 3 | 14000.0 | 2015-11-23 | USERS
| 7 | 1 | 1 | 30000.0 | 2015-12-08 | +------+--------+
| 8 | 1 | 1 | 40000.0 | 2015-12-29 | | id | name |
| 9 | 1 | 2 | 10000.0 | 2015-12-01 | |------+--------|
| 10 | 1 | 2 | 11400.0 | 2015-12-14 | | 1 | Sarah |
| 11 | 1 | 3 | 21000.0 | 2015-12-26 | +------+--------+
| 12 | 1 | 3 | 21000.0 | 2015-12-26 |
+------+-----------+------------+----------+--------------+
</code></pre></div></div>
<p>In our application, users enter sales data for a given store. The sales table has a column for “entered_at” which represents when the sales data was entered.</p>
<h4 id="using-windows-functions-with-aggregate-functions">Using Windows Functions with Aggregate Functions</h4>
<p>Lets say we wanted to aggregate the sales for each store. We could do this easily using a <code class="highlighter-rouge">GROUP BY</code> statement.</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>SELECT store_id, sum(amount) as amt FROM sales GROUP BY sales.store_id
</code></pre></div></div>
<p>which yields the following rows:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>+------------+----------+
| store_id | amt |
|------------+----------|
| 1 | 110000.0 |
| 2 | 51400.0 |
| 3 | 70000.0 |
+------------+----------+
</code></pre></div></div>
<p>But what if we wanted to return each row from the sales table individually, with the corresponding stores total sales amount. We can do this using a window function.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="k">sum</span><span class="p">(</span><span class="n">amount</span><span class="p">)</span> <span class="n">OVER</span> <span class="p">(</span><span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">store_id</span><span class="p">)</span> <span class="k">FROM</span> <span class="n">sales</span>
</code></pre></div></div>
<p>which results in:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>+------------+----------+
| store_id | sum |
|------------+----------|
| 1 | 110000.0 |
| 1 | 110000.0 |
| 1 | 110000.0 |
| 1 | 110000.0 |
| 2 | 51400.0 |
| 2 | 51400.0 |
| 2 | 51400.0 |
| 2 | 51400.0 |
| 3 | 70000.0 |
| 3 | 70000.0 |
| 3 | 70000.0 |
| 3 | 70000.0 |
+------------+----------+
</code></pre></div></div>
<p>Let’s break this down a bit. The store_id column is self explanatory. The sum aggregates the amount from every row that has the same store_id as the current row. The OVER keyword is what causes the aggregate function to be treated as a window function, and compute the aggregate across the rows in the window without grouping.</p>
<p>But how do we define the window? How does Postgres know which subset of rows we want to aggregate? The answer lies in the PARTITION keyword. The partition is what divides the table rows into distinct window frames. In this case we partition the table by the store_id.</p>
<p>One thing to note is that we can use an aggregate function as a window function. We could have just as easily written</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="n">OVER</span> <span class="p">(</span><span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">store_id</span><span class="p">)</span> <span class="k">FROM</span> <span class="n">sales</span>
</code></pre></div></div>
<h4 id="using-the-rank-function">Using the Rank Function</h4>
<p>Postgres provides a number of window specific functions which you can find <a href="https://www.postgresql.org/docs/9.3/functions-window.html">here</a>. One of the most common use cases for window functions is to rank a window of rows using the rank function.</p>
<p>Lets say we wanted to rank each sales record by the amount entered for a given store_id. We can do this easily with the rank function:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="n">amount</span><span class="p">,</span> <span class="n">rank</span><span class="p">()</span> <span class="n">OVER</span> <span class="p">(</span><span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">store_id</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">amount</span> <span class="k">DESC</span><span class="p">)</span> <span class="k">AS</span> <span class="n">amount_rank</span> <span class="k">FROM</span> <span class="n">sales</span>
</code></pre></div></div>
<p>The resulting table would look like this:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+------------+----------+---------------+
| store_id | amount | amount_rank |
|------------+----------+---------------|
| 1 | 40000.0 | 1 |
| 1 | 30000.0 | 2 |
| 1 | 30000.0 | 2 |
| 1 | 10000.0 | 4 |
| 2 | 19000.0 | 1 |
| 2 | 11400.0 | 2 |
| 2 | 11000.0 | 3 |
| 2 | 10000.0 | 4 |
| 3 | 21000.0 | 1 |
| 3 | 21000.0 | 1 |
| 3 | 14000.0 | 3 |
| 3 | 14000.0 | 3 |
+------------+----------+---------------+
</code></pre></div></div>
<p>You may notice that if a two or more rows have the same amount, they receive the same rank. The rank function will assign a unique number to each distinct row, but it leaves a gap when there are multiple rows with the same rank. Take a look at the rows with a store_id of 1. We can see that there are two ranks of 2, so the next rank is incremented not to 3, but to 4. the dense_rank function does not do this and instead the row with a rank of 4 would have a rank of 3.</p>
<p>These have been some pretty simple examples of window functions. Let’s move onto something a little bit more interesting.</p>
<h4 id="complex-window-partitions">Complex Window Partitions</h4>
<p>Let’s say that in our application we want to be able to get the most recent sales data entered for a given month/year combination and a given store. This problem can be solved handily using window functions.</p>
<p>Let’s start by crafting a query to give us the store_id, the month of the entered date, the year of the entered_date, and the amount of the sale.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="k">month</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">mm</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="nb">year</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">yyyy</span><span class="p">,</span> <span class="n">amount</span> <span class="k">FROM</span> <span class="n">sales</span>
</code></pre></div></div>
<p>Which yields:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>+------------+------+--------+----------+
| store_id | mm | yyyy | amount |
|------------+------+--------+----------|
| 1 | 11 | 2015 | 10000.0 |
| 1 | 11 | 2015 | 30000.0 |
| 2 | 11 | 2015 | 11000.0 |
| 2 | 11 | 2015 | 19000.0 |
| 3 | 11 | 2015 | 14000.0 |
| 3 | 11 | 2015 | 14000.0 |
| 1 | 12 | 2015 | 30000.0 |
| 1 | 12 | 2015 | 40000.0 |
| 2 | 12 | 2015 | 10000.0 |
| 2 | 12 | 2015 | 11400.0 |
| 3 | 12 | 2015 | 21000.0 |
| 3 | 12 | 2015 | 21000.0 |
+------------+------+--------+----------+
</code></pre></div></div>
<p>Note that we are using the EXTRACT function which allows us to retrieve subfields such as year, hour, day, month from date/time values.</p>
<p>The next thing we want to do is to partition the table. The original intent of our query was to grab the most recent sales data entered for a given month/year combination and a given store. So our partition should be defined as follows “for each row, get me the rows with the same month/year in the entered date, and the same store_id”. In other words, we need to partition by the extracted month, year, and store_id.</p>
<p>Now here’s the problem. Unlike a GROUP BY statement, we can’t partition over several columns. So how can we get the window frames we want?</p>
<p>The solution I found was to use the string function concat, and grouping by the concatenated value of the extracted month/year, and the store_id.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="k">month</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">mm</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="nb">year</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">yyyy</span><span class="p">,</span> <span class="n">amount</span><span class="p">,</span> <span class="n">rank</span><span class="p">()</span> <span class="n">OVER</span> <span class="p">(</span><span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">concat</span><span class="p">(</span><span class="k">extract</span><span class="p">(</span><span class="k">month</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">),</span> <span class="k">extract</span><span class="p">(</span><span class="nb">year</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">),</span> <span class="n">store_id</span><span class="p">)</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">entered_at</span> <span class="k">DESC</span><span class="p">)</span> <span class="k">AS</span> <span class="n">entered_at_rank</span> <span class="k">FROM</span> <span class="n">sales</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">store_id</span>
</code></pre></div></div>
<p>This is effectively grouping by the month of the entered date, the year of the entered date, and the store id. It gives us the exact partitions that we want!</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>+------------+------+--------+----------+-------------------+
| store_id | mm | yyyy | amount | entered_at_rank |
|------------+------+--------+----------+-------------------|
| 1 | 11 | 2015 | 30000.0 | 1 |
| 1 | 11 | 2015 | 10000.0 | 2 |
| 1 | 12 | 2015 | 40000.0 | 1 |
| 1 | 12 | 2015 | 30000.0 | 2 |
| 2 | 11 | 2015 | 11000.0 | 2 |
| 2 | 12 | 2015 | 11400.0 | 1 |
| 2 | 12 | 2015 | 10000.0 | 2 |
| 2 | 11 | 2015 | 19000.0 | 1 |
| 3 | 11 | 2015 | 14000.0 | 1 |
| 3 | 11 | 2015 | 14000.0 | 2 |
| 3 | 12 | 2015 | 21000.0 | 1 |
| 3 | 12 | 2015 | 21000.0 | 1 |
+------------+------+--------+----------+-------------------+
</code></pre></div></div>
<p>Now the next step is to apply a where clause that selects only the sales with the highest rank. Since we can’t apply a WHERE clause to an aliased column that is not on the table itself, we have to use a subquery in our FROM statement:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="n">mm</span><span class="p">,</span> <span class="n">yyyy</span><span class="p">,</span> <span class="n">amount</span><span class="p">,</span> <span class="n">entered_at_rank</span> <span class="k">FROM</span> <span class="p">(</span><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="k">month</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">mm</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="nb">year</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">yyyy</span><span class="p">,</span> <span class="n">amount</span><span class="p">,</span> <span class="n">rank</span><span class="p">()</span> <span class="n">OVER</span> <span class="p">(</span><span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">concat</span><span class="p">(</span><span class="k">extract</span><span class="p">(</span><span class="k">month</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">),</span> <span class="k">extract</span><span class="p">(</span><span class="nb">year</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">),</span> <span class="n">store_id</span><span class="p">)</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">entered_at</span> <span class="k">DESC</span><span class="p">)</span> <span class="k">AS</span> <span class="n">entered_at_rank</span> <span class="k">FROM</span> <span class="n">sales</span> <span class="k">ORDER</span> <span class="k">by</span> <span class="n">entered_at_rank</span><span class="p">)</span> <span class="k">as</span> <span class="n">t</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">store_id</span>
</code></pre></div></div>
<p>This results in exactly the same table as above, but now we can use a WHERE clause on entered_at_rank like this:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="n">mm</span><span class="p">,</span> <span class="n">yyyy</span><span class="p">,</span> <span class="n">amount</span><span class="p">,</span> <span class="n">entered_at_rank</span> <span class="k">FROM</span> <span class="p">(</span><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="k">month</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">mm</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="nb">year</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">yyyy</span><span class="p">,</span> <span class="n">amount</span><span class="p">,</span> <span class="n">rank</span><span class="p">()</span> <span class="n">OVER</span> <span class="p">(</span><span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">concat</span><span class="p">(</span><span class="k">extract</span><span class="p">(</span><span class="k">month</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">),</span> <span class="k">extract</span><span class="p">(</span><span class="nb">year</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">),</span> <span class="n">store_id</span><span class="p">)</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">entered_at</span> <span class="k">DESC</span><span class="p">)</span> <span class="k">AS</span> <span class="n">entered_at_rank</span> <span class="k">FROM</span> <span class="n">sales</span> <span class="k">ORDER</span> <span class="k">by</span> <span class="n">entered_at_rank</span><span class="p">)</span> <span class="k">as</span> <span class="n">t</span> <span class="k">WHERE</span> <span class="n">entered_at_rank</span> <span class="o">=</span> <span class="mi">1</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">store_id</span>
</code></pre></div></div>
<p>Which results in</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>+------------+------+--------+----------+-------------------+
| store_id | mm | yyyy | amount | entered_at_rank |
|------------+------+--------+----------+-------------------|
| 1 | 12 | 2015 | 40000.0 | 1 |
| 1 | 11 | 2015 | 30000.0 | 1 |
| 2 | 12 | 2015 | 11400.0 | 1 |
| 2 | 11 | 2015 | 19000.0 | 1 |
| 3 | 12 | 2015 | 21000.0 | 1 |
| 3 | 11 | 2015 | 14000.0 | 1 |
| 3 | 12 | 2015 | 21000.0 | 1 |
+------------+------+--------+----------+-------------------+
</code></pre></div></div>
<p>There is a problem though! The store with an id of 3 has multiple sales for the same month with the same rank (since they have the same amount). What to do?</p>
<h4 id="row-number-to-the-rescue">Row Number to the Rescue</h4>
<p>One thing we can do is use the row_number() window function which will give the number of the current_row within its partition.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="n">mm</span><span class="p">,</span> <span class="n">yyyy</span><span class="p">,</span> <span class="n">amount</span><span class="p">,</span> <span class="n">entered_at_row</span> <span class="k">FROM</span> <span class="p">(</span><span class="k">SELECT</span> <span class="n">store_id</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="k">month</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">mm</span><span class="p">,</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="nb">year</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">)</span> <span class="k">as</span> <span class="n">yyyy</span><span class="p">,</span> <span class="n">amount</span><span class="p">,</span> <span class="n">row_number</span><span class="p">()</span> <span class="n">OVER</span> <span class="p">(</span><span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">concat</span><span class="p">(</span><span class="k">extract</span><span class="p">(</span><span class="k">month</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">),</span> <span class="k">extract</span><span class="p">(</span><span class="nb">year</span> <span class="k">from</span> <span class="n">entered_at</span><span class="p">),</span> <span class="n">store_id</span><span class="p">)</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">entered_at</span> <span class="k">DESC</span><span class="p">)</span> <span class="k">AS</span> <span class="n">entered_at_row</span> <span class="k">FROM</span> <span class="n">sales</span> <span class="k">ORDER</span> <span class="k">by</span> <span class="n">entered_at_row</span><span class="p">)</span> <span class="k">as</span> <span class="n">t</span> <span class="k">WHERE</span> <span class="n">entered_at_row</span> <span class="o">=</span> <span class="mi">1</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">store_id</span>
</code></pre></div></div>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>+------------+------+--------+----------+------------------+
| store_id | mm | yyyy | amount | entered_at_row |
|------------+------+--------+----------+------------------|
| 1 | 11 | 2015 | 30000.0 | 1 |
| 1 | 12 | 2015 | 40000.0 | 1 |
| 2 | 11 | 2015 | 19000.0 | 1 |
| 2 | 12 | 2015 | 11400.0 | 1 |
| 3 | 11 | 2015 | 14000.0 | 1 |
| 3 | 12 | 2015 | 21000.0 | 1 |
+------------+------+--------+----------+------------------+
</code></pre></div></div>
<p>Now we have exactly one sale for each store in a given month/year!</p>Nassredean Dean Nasseridean@nasseri.iohttps://nasseri.ioRecently I stumbled into a very handy feature of Postgres — window functions. A window function gives you a “window” into other rows which are somehow related to the current row. You can then preform a calculation across that set of rows. Window functions behave much like regular aggregate functions, with the caveat that the rows do not become grouped into a single row! Let’s explore this feature, and why it might be useful.How One Ruby Method Let us Delete Thousands of Lines of Code.2016-03-05T00:00:00-05:002016-03-05T00:00:00-05:00https://nasseri.io/2016/03/05/how-one-ruby-method-let-us-delete-thousands-of-lines-of-code<p>At VTS we have a saying: code is a liability. As an application grows, legacy code develops. This code might not ever be covered in your production application, but you still incur the cost of maintaining it. Unfortunately, knowing what code is legacy and safe to remove can be difficult in a complex application. In order to try and determine what code was being hit we opted to use a gem called coverband. Coverband is a ruby gem to measure production code coverage. When enabled it logs every file and line number which has been hit to redis. Coverband can be used as a rack middleware which means that, once configured, we were able to trace the execution path of requests on our production servers.</p>
<!--more-->
<p>Up until very recently, <a href="https://github.com/danmayer/coverband">coverband</a> worked by using a method on the Kernel module called <code class="highlighter-rouge">set_trace_func</code>. The method <code class="highlighter-rouge">set_trace_func</code> takes a blocks as an argument with 6 parameters: The event name, filename, line number, object id, binding, and the name of a class. When called, <code class="highlighter-rouge">set_trace_func</code> will invoke the passed in block whenever one of the following events occurs:</p>
<ul>
<li>c-call: call a C-language routine</li>
<li>c-return: return from a C-language routine</li>
<li>call: call a Ruby method</li>
<li>class: start a class or module definition</li>
<li>end: finish a class or module definition</li>
<li>line: execute code on a new line</li>
<li>raise: raise an exception</li>
<li>return: return from a Ruby method</li>
</ul>
<p>Here is an example, taken from the documentation:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Test</span>
<span class="k">def</span> <span class="nf">test</span>
<span class="n">a</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">b</span> <span class="o">=</span> <span class="mi">2</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="nb">set_trace_func</span> <span class="nb">proc</span> <span class="p">{</span> <span class="o">|</span><span class="n">event</span><span class="p">,</span> <span class="n">file</span><span class="p">,</span> <span class="n">line</span><span class="p">,</span> <span class="nb">id</span><span class="p">,</span> <span class="nb">binding</span><span class="p">,</span> <span class="n">classname</span><span class="o">|</span>
<span class="nb">printf</span> <span class="s2">"%8s %s:%-2d %10s %8s</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">file</span><span class="p">,</span> <span class="n">line</span><span class="p">,</span> <span class="nb">id</span><span class="p">,</span> <span class="n">classname</span>
<span class="p">}</span>
<span class="n">t</span> <span class="o">=</span> <span class="no">Test</span><span class="p">.</span><span class="nf">new</span>
<span class="n">t</span><span class="p">.</span><span class="nf">test</span>
</code></pre></div></div>
<p>Output:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">line</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">11</span> <span class="kp">false</span>
<span class="n">c</span><span class="o">-</span><span class="n">call</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">11</span> <span class="n">new</span> <span class="no">Class</span>
<span class="n">c</span><span class="o">-</span><span class="n">call</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">11</span> <span class="n">initialize</span> <span class="no">Object</span>
<span class="n">c</span><span class="o">-</span><span class="k">return</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">11</span> <span class="n">initialize</span> <span class="no">Object</span>
<span class="n">c</span><span class="o">-</span><span class="k">return</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">11</span> <span class="n">new</span> <span class="no">Class</span>
<span class="n">line</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">12</span> <span class="kp">false</span>
<span class="n">call</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">2</span> <span class="nb">test</span> <span class="no">Test</span>
<span class="n">line</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">3</span> <span class="nb">test</span> <span class="no">Test</span>
<span class="n">line</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">4</span> <span class="nb">test</span> <span class="no">Test</span>
<span class="k">return</span> <span class="n">prog</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">4</span> <span class="nb">test</span> <span class="no">Test</span>
</code></pre></div></div>
<p>In the case of coverband, the relevant code looks like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">set_tracer</span>
<span class="k">unless</span> <span class="vi">@tracer_set</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">.</span><span class="nf">set_trace_func</span> <span class="nb">proc</span> <span class="p">{</span> <span class="o">|</span><span class="n">event</span><span class="p">,</span> <span class="n">file</span><span class="p">,</span> <span class="n">line</span><span class="p">,</span> <span class="nb">id</span><span class="p">,</span> <span class="nb">binding</span><span class="p">,</span> <span class="n">classname</span><span class="o">|</span>
<span class="n">add_file</span><span class="p">(</span><span class="n">file</span><span class="p">,</span> <span class="n">line</span><span class="p">)</span>
<span class="p">}</span>
<span class="vi">@tracer_set</span> <span class="o">=</span> <span class="kp">true</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>From a super high level and ignoring many implementation details the set_tracer method is called on init and calls set_trace_func on the current thread, passing in a block. The block itself calls the add_file method which will log the file and line to redis. Overtime you build up a set of key value pairs in redis where the key is the filename and the value is an array of line numbers. You can then generate a SCOV style report and get a nice output making it easy to see what lines of code are covered in your application!</p>
<p>But how does set_trace_func actually work? Seeing set_trace_func in action can feel a bit like that classic “ruby magic”. The thing that is most confusing is, how ruby knows when to call the proc. To understand this, we have to take a little detour into how ruby actually runs programs.</p>
<h4 id="tokenization">Tokenization</h4>
<p>Let’s take the following very simple program:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="mi">2</span> <span class="o">+</span> <span class="mi">2</span>
</code></pre></div></div>
<p>When you run the program, ruby first tokenizes the input. During tokenization, ruby steps through the characters one at a time, and groups them together into tokens. Tokenization is the first step in turning the text into an actual program. We can actually see how ruby does its tokenization using the built in ripper tool:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'ripper'</span>
<span class="nb">require</span> <span class="s1">'pp'</span>
<span class="n">code</span> <span class="o">=</span> <span class="o">&lt;&lt;</span><span class="no">CODE</span><span class="sh">
2 + 2
</span><span class="no">CODE</span>
<span class="n">pp</span> <span class="no">Ripper</span><span class="p">.</span><span class="nf">lex</span><span class="p">(</span><span class="n">code</span><span class="p">)</span>
</code></pre></div></div>
<p>Output:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="ss">:on_int</span><span class="p">,</span> <span class="s2">"2"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">""</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="ss">:on_op</span><span class="p">,</span> <span class="s2">"+"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="ss">:on_sp</span><span class="p">,</span> <span class="s2">" "</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="ss">:on_int</span><span class="p">,</span> <span class="s2">"2"</span><span class="p">],</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">],</span> <span class="ss">:on_nl</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">]]</span>
</code></pre></div></div>
<p>Each line represents a single token. The first array value is an array which consists of the line and column number. The second array value is the token itself, which corresponds to the actual C parse code. For instance the :on_int token corresponds to the actual tINTEGER token that would be found in ruby’s source code. The third value is the string of characters in the ruby code that correspond to the token.</p>
<p>Tokenization is responsible only for the process of turning characters into tokens, it is not responsible for determining whether the passed in characters are actually valid ruby. That is the responsibility of the parser.</p>
<h4 id="parsing">Parsing</h4>
<p>After the tokenization step, ruby then parses the tokens, grouping them into sentences and phrases as defined by ruby’s grammar rules. Ruby uses the LALR (Look-ahead left reversed rightmost derivation) algorithm to parse the code and generate an AST(abstract syntax tree). I will not detail the algorithm here, but you can see a textual representation of the AST ruby generates for our code by running the following program:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'ripper'</span>
<span class="nb">require</span> <span class="s1">'pp'</span>
<span class="n">code</span> <span class="o">=</span> <span class="o">&lt;&lt;</span><span class="no">CODE</span><span class="sh">
2 + 2
</span><span class="no">CODE</span>
<span class="n">pp</span> <span class="no">Ripper</span><span class="p">.</span><span class="nf">sexp</span><span class="p">(</span><span class="n">code</span><span class="p">)</span>
</code></pre></div></div>
<p>Output:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="ss">:program</span><span class="p">,</span>
<span class="p">[[</span><span class="ss">:binary</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:@int</span><span class="p">,</span> <span class="err">“</span><span class="mi">2</span><span class="err">”</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">]],</span>
<span class="p">:</span><span class="o">+</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:@int</span><span class="p">,</span> <span class="err">“</span><span class="mi">2</span><span class="err">”</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]]</span>
<span class="p">]]</span>
<span class="p">]</span>
</code></pre></div></div>
<p>Ruby has compiled the stream of tokens into an AST, a description of the actual program. The AST is a tree data structure which represents the structure of the program code. The textual output might look a bit confusing, so here is a corresponding graphical output:</p>
<p><img src="/assets/images/posts/4/ast.png" alt="ast" /></p>
<p>We start here with the top level program node. Every ruby generated AST begins with this node. The + operation is contained in a binary node, as in a binary operation. You can see that walking the AST top down and then left to right yields a description of the actual program.</p>
<h4 id="compiling-to-yarv-instructions">Compiling to YARV Instructions</h4>
<p>As of Ruby 1.9, Ruby uses a virtual machine called YARV(Yet Another Ruby Virtual Machine). This means there is an additional compile step after generating an AST. Ruby recursively iterates over the AST from the top down and compiles each node into YARV instructions. We can see the compiled YARV instructions for our example using built in library tools:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">code</span> <span class="o">=</span> <span class="o">&lt;&lt;</span><span class="no">CODE</span><span class="sh">
2 + 2
</span><span class="no">CODE</span>
<span class="nb">puts</span> <span class="no">RubyVM</span><span class="o">::</span><span class="no">InstructionSequence</span><span class="p">.</span><span class="nf">compile</span><span class="p">(</span><span class="n">code</span><span class="p">).</span><span class="nf">disasm</span>
</code></pre></div></div>
<p>The compiled YARV instructions of this very simple program look like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">==</span> <span class="ss">disasm: </span><span class="o">&lt;</span><span class="no">RubyVM</span><span class="o">::</span><span class="no">InstructionSequence</span><span class="ss">:&lt;</span><span class="n">compiled</span><span class="o">&gt;</span><span class="err">@</span><span class="o">&lt;</span><span class="n">compiled</span><span class="o">&gt;&gt;==========</span>
<span class="mo">0000</span> <span class="n">trace</span> <span class="mi">1</span> <span class="p">(</span> <span class="mi">1</span><span class="p">)</span>
<span class="mo">0002</span> <span class="n">putobject</span> <span class="mi">2</span>
<span class="mo">0004</span> <span class="n">putobject</span> <span class="mi">2</span>
<span class="mo">0006</span> <span class="n">opt_plus</span> <span class="o">&lt;</span><span class="n">callinfo!mid</span><span class="p">:</span><span class="o">+</span><span class="p">,</span> <span class="n">argc</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span> <span class="no">ARGS_SIMPLE</span><span class="o">&gt;</span>
<span class="mo">000</span><span class="mi">8</span> <span class="n">leave</span>
</code></pre></div></div>
<p>Let’s examine these instructions carefully.</p>
<p>First we start with the trace instruction. The trace instruction is exactly what allows us to use set_trace_func. It indicates that a new event has occurred. In this case, the event is line.</p>
<p>Next YARV pushes the object 2 on the stack as a receiver.</p>
<p>Then another object 2 is pushed onto the stack as an argument.</p>
<p>The final instruction, opt_plus says “send the plus message to the receiver (the first 2 object pushed onto the stack) with one argument (the next 2 object on the stack). The ARGS_SIMPLE indicates that the arguments are simple values.</p>
<p>Now let’s confirm that the trace YARV instruction corresponds to the events hooked into by <code class="highlighter-rouge">set_trace_func</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">set_trace_func</span> <span class="nb">proc</span> <span class="p">{</span> <span class="o">|</span><span class="n">event</span><span class="p">,</span> <span class="n">file</span><span class="p">,</span> <span class="n">line</span><span class="p">,</span> <span class="nb">id</span><span class="p">,</span> <span class="nb">binding</span><span class="p">,</span> <span class="n">classname</span><span class="o">|</span>
<span class="nb">printf</span> <span class="err">“</span><span class="o">%</span><span class="mi">8</span><span class="n">s</span> <span class="sx">%s:%-2d %10s %8s\n”, event, file, line, id, classname
}
2 + 2
</span></code></pre></div></div>
<p>Output:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">c</span><span class="o">-</span><span class="k">return</span> <span class="n">example2</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">1</span> <span class="nb">set_trace_func</span> <span class="no">Kernel</span>
<span class="n">line</span> <span class="n">example2</span><span class="p">.</span><span class="nf">rb</span><span class="p">:</span><span class="mi">4</span>
</code></pre></div></div>
<p>Curiously, we see two events here, this is because <code class="highlighter-rouge">set_trace_func</code> actually traces itself! So we see the first event, c-return or return from a c function call, in this case the built in <code class="highlighter-rouge">set_trace_func</code> method, and then the line event we expected to see. The important thing to remember is this. When ruby executes your program it first translates the source code into a stream of tokens. Ruby then parses the stream of tokens into an AST. Finally each node scope in the AST is compiled into a snippet of YARV instructions. These YARV instructions include calls to trace which allow us to hook into internal ruby events.</p>
<p>This is just one of the many beauties of Ruby. The dynamic nature of the language allows us to do things which would be impossible in other languages, and which can generate real business value. In this case, understanding one simple built in method allowed us to delete thousands of lines of unused code from our production application, as well as the corresponding tests. For us at VTS it means we have faster test suite, and fewer lines of code to maintain. For more information I would highly recommend checking out the coverband gem for yourself. For more information on how ruby tokenizes, parses, and compiles programs, I would recommend Pat Shaughnessy’s <a href="http://patshaughnessy.net/ruby-under-a-microscope">“Ruby Under A Microscope”</a>, a spectacular dive into Ruby’s internals. In a follow up blogpost I would like to detail how we were able to speed up coverband by swapping out <code class="highlighter-rouge">set_trace_func</code> for a new ruby class called TracePoint which provides the same functionality as <code class="highlighter-rouge">set_trace_func</code>, but which allows us to hook into specific events. Thanks for reading!</p>Nassredean Dean Nasseridean@nasseri.iohttps://nasseri.ioAt VTS we have a saying: code is a liability. As an application grows, legacy code develops. This code might not ever be covered in your production application, but you still incur the cost of maintaining it. Unfortunately, knowing what code is legacy and safe to remove can be difficult in a complex application. In order to try and determine what code was being hit we opted to use a gem called coverband. Coverband is a ruby gem to measure production code coverage. When enabled it logs every file and line number which has been hit to redis. Coverband can be used as a rack middleware which means that, once configured, we were able to trace the execution path of requests on our production servers.Threading in MRI Ruby for Fun and Performance2015-12-24T00:00:00-05:002015-12-24T00:00:00-05:00https://nasseri.io/2015/12/24/threading-in-mri-ruby-for-fun-and-performance<p>Frequently in our applications as software engineers, we run into situations where we want our code to do multiple things at once. This is the problem of concurrency. Concurrency has an infinite number of use cases. One such use case is that of a webserver. If you were to use a non-concurrent webserver, it would only be able to handle one request at a time. Imagine your site received several requests at the same time. Now the webserver must process each request one at a time, unable to move on to the next request in the queue until the previous one has completed. In order to alleviate this, we might use a concurrent webserver which is capable of processing multiple requests at once.</p>
<!--more-->
<h4 id="concurrency-vs-parallelism">Concurrency vs Parallelism</h4>
<p>Before I continue any further, I would like to clear up a point of confusion. This is well worn territory, but to quote Rob Pike, (concurrency is not parallelism)[https://blog.golang.org/waza-talk]. Concurrency means multiple paths of execution can be run and complete in the same time period. Concurrency does not mean that these tasks are running at the exact same time. In a concurrent program utilizing threads and running on a computer with a single CPU core, the OS would switch between the threads, and at no point would the threads actually be executing simultaneously, even if they appeared to. Parallelism means the threads are literally executing simultaneously. Imagine an ATM with 2 queues. Only one person uses the ATM machine at a time, and the next person to use the ATM alternates between the two queues. This is concurrency. Now imagine two ATM machines, each with its own queue. This is parallelism.</p>
<h4 id="global-interpreter-lock">Global Interpreter Lock</h4>
<p>MRI has a global interpreter lock (GIL). The GIL means that in a multi-threaded context, only one thread can execute Ruby code at any moment in time. Some posit that, due to the GIL, true concurrency is not possible in ruby programs. Even if we were to run our program on multiple cores, the GIL would prevent more than one thread executing at a time. This is a significant tradeoff of the language, but the GIL also means that we don’t have to worry about corrupting data, or race conditions within C extensions. For these reasons, threading is much easier to deal with in Ruby than in many other languages, even if it is less powerful.</p>
<p>“But hold on!” You might be exclaiming. “If only one thread can execute at a time, how will my code run any faster when using multiple threads of execution?”</p>
<p>Great question! The answer is that the GIL only applies to ruby operations, and the work that ruby is doing. Things like I/O or querying a database are free to execute on multiple threads at the same time.</p>
<h4 id="basic-threading">Basic Threading</h4>
<p>In Ruby, concurrency can be achieved using threads. Since version 1.9, MRI Ruby has supported native (OS level) threads.</p>
<p>Take a look at the following code.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">thr</span> <span class="o">=</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="nb">puts</span> <span class="s2">"Hello, thread world!"</span> <span class="p">}</span>
</code></pre></div></div>
<p>Running this program will yield… nothing. The issue is that the main thread creates a new thread, but as soon as it does the program terminates, and the second thread is terminated alongside it. What we have to do is suspend the execution of the main thread and wait for the second thread to terminate. To do this we have to join the thread.</p>
<h4 id="threadjoin">Thread.join</h4>
<p>Thread.join is what allows us to wait for the second thread to terminate. Typically, when a ruby program exits, all running threads will be killed. However, when we join a thread, the execution of the calling thread is suspended until the joined threads have finished running.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">thr</span> <span class="o">=</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="nb">puts</span> <span class="s2">"Hello, thread world!"</span> <span class="p">}</span>
<span class="n">thr</span><span class="p">.</span><span class="nf">join</span>
</code></pre></div></div>
<p>Running the program now will yield “Hello, thread world!” to STDOUT as expected.</p>
<p>Now lets move on to another example and try to really visualize what happens when we start creating and joining threads.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">thr1</span> <span class="o">=</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="nb">puts</span> <span class="s1">'a'</span>
<span class="nb">sleep</span> <span class="mi">2</span>
<span class="nb">puts</span> <span class="s1">'b'</span>
<span class="nb">sleep</span> <span class="mi">2</span>
<span class="nb">puts</span> <span class="s1">'c'</span>
<span class="k">end</span>
<span class="n">thr2</span> <span class="o">=</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="nb">puts</span> <span class="s1">'1'</span>
<span class="nb">sleep</span> <span class="mi">2</span>
<span class="nb">puts</span> <span class="s1">'2'</span>
<span class="nb">sleep</span> <span class="mi">2</span>
<span class="nb">puts</span> <span class="s1">'3'</span>
<span class="k">end</span>
<span class="n">thr1</span><span class="p">.</span><span class="nf">join</span>
<span class="n">thr2</span><span class="p">.</span><span class="nf">join</span>
</code></pre></div></div>
<p>Running this program on my machine a few times yields a few different outcomes. Take a look.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">$</span> <span class="n">ruby</span> <span class="n">ruby</span><span class="o">-</span><span class="n">threading</span><span class="o">-</span><span class="n">ex2</span><span class="p">.</span><span class="nf">rb</span>
<span class="n">a</span>
<span class="mi">1</span>
<span class="n">b</span>
<span class="mi">2</span>
<span class="mi">3</span>
<span class="n">c</span>
<span class="err">$</span> <span class="n">ruby</span> <span class="n">ruby</span><span class="o">-</span><span class="n">threading</span><span class="o">-</span><span class="n">ex2</span><span class="p">.</span><span class="nf">rb</span>
<span class="mi">1</span>
<span class="n">a</span>
<span class="mi">2</span>
<span class="n">b</span>
<span class="mi">3</span>
<span class="n">c</span>
</code></pre></div></div>
<p>The inconsistent results lead to some interesting conclusions. Though we initialized thr1 first, ruby makes no guarantees that it will finish initializing before thr2. Furthermore, there is not guarantee in what order the blocks passed to each thread will execute.</p>
<h4 id="thread-and-fiber-local-variables">Thread and Fiber Local Variables</h4>
<p>Often times, we need a way to reach into a given thread and pull out a value. We can do so using one of two methods. The first way is by riding the thread like so:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">.</span><span class="nf">thread_variable_set</span><span class="p">(</span><span class="s2">"foo"</span><span class="p">,</span> <span class="s2">"bar"</span><span class="p">)</span><span class="o">.</span>
</code></pre></div></div>
<p>We can also ride the current fiber:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">[</span><span class="s2">"foo"</span><span class="p">]</span> <span class="o">=</span> <span class="s2">"bar"</span>
</code></pre></div></div>
<p>The difference is a matter of scoping. Here is an example from the docs:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">.</span><span class="nf">thread_variable_set</span><span class="p">(</span><span class="s2">"foo"</span><span class="p">,</span> <span class="s2">"bar"</span><span class="p">)</span> <span class="c1"># set a thread local</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">[</span><span class="s2">"foo"</span><span class="p">]</span> <span class="o">=</span> <span class="s2">"bar"</span> <span class="c1"># set a fiber local</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span>
<span class="no">Fiber</span><span class="p">.</span><span class="nf">yield</span> <span class="p">[</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">.</span><span class="nf">thread_variable_get</span><span class="p">(</span><span class="s2">"foo"</span><span class="p">),</span> <span class="c1"># get the thread local</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">[</span><span class="s2">"foo"</span><span class="p">],</span> <span class="c1"># get the fiber local</span>
<span class="p">]</span>
<span class="p">}.</span><span class="nf">resume</span>
<span class="p">}.</span><span class="nf">join</span><span class="p">.</span><span class="nf">value</span> <span class="c1"># =&gt; ['bar', nil]</span>
</code></pre></div></div>
<p>Thread locals are carried along with threads, and do not respect fibers. That is the only difference we will concern ourselves with here. There will be more on fibers in part 2 of this series of blog posts. For now though, know that we can set fiber locals. The beauty is that we can access the locals after the thread has finished executing. So we might do some computationally expensive work, set the value to a fiber local, and then access them once the thread has finished executing. For instance:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">threads</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">threads</span> <span class="o">&lt;&lt;</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="nb">sleep</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># do something expensive</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">[</span><span class="ss">:foo</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'result1'</span>
<span class="k">end</span>
<span class="n">threads</span> <span class="o">&lt;&lt;</span> <span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="nb">sleep</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># do something expensive</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">[</span><span class="ss">:foo</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'result2'</span>
<span class="k">end</span>
<span class="n">threads</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&amp;</span><span class="ss">:join</span><span class="p">).</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">thr</span><span class="o">|</span>
<span class="nb">puts</span> <span class="n">thr</span><span class="p">[</span><span class="ss">:foo</span><span class="p">]</span>
<span class="k">end</span>
<span class="err">$</span> <span class="n">ruby</span> <span class="n">fiber</span><span class="o">-</span><span class="n">local</span><span class="o">-</span><span class="n">ex</span><span class="p">.</span><span class="nf">rb</span>
<span class="n">result1</span>
<span class="n">result2</span>
</code></pre></div></div>
<p>This is an extremely common pattern when implementing concurrency in ruby.</p>
<h4 id="a-practical-example">A Practical Example</h4>
<p>Recently at VTS we rolled out a new filtering system across the entire site. Upon each request, the filters were performing a series of expensive aggregate queries. Each query looked something like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">deal_types</span>
<span class="no">Deal</span>
<span class="p">.</span><span class="nf">where</span><span class="p">{</span> <span class="n">deals</span><span class="p">.</span><span class="nf">id</span><span class="p">.</span><span class="nf">in</span><span class="p">(</span><span class="n">my</span><span class="p">{</span> <span class="n">deal_ids</span> <span class="p">}</span> <span class="p">)</span> <span class="p">}</span>
<span class="p">.</span><span class="nf">group</span><span class="p">{</span> <span class="n">type</span> <span class="p">}.</span><span class="nf">pluck_all</span><span class="p">(</span>
<span class="s1">'deals.type as item_id'</span><span class="p">,</span>
<span class="s1">'deals.type as label'</span><span class="p">,</span>
<span class="s2">"sum(case when deal.id in (</span><span class="si">#{</span> <span class="n">deal_id_subset</span> <span class="ss">exclude: :types</span> <span class="si">}</span><span class="s2">) then 1 else 0 end) as count"</span>
<span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The result set of all the queries was serialized as a hash as follows:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">as_json</span>
<span class="p">{</span>
<span class="ss">deal_types: </span><span class="n">deal_types</span><span class="p">,</span>
<span class="err">…</span>
<span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The queries ran one at a time, and the filter response began timing out. After tuning the queries as best we could, we still had timeout issues. One of our engineers came up with the solution to use threads and run the queries concurrently.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">parallelize</span> <span class="o">&amp;</span><span class="n">block</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">current</span><span class="p">[</span><span class="ss">:output</span><span class="p">]</span> <span class="o">=</span> <span class="k">yield</span>
<span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">clear_active_connections!</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The new queries were then called and serialized as follows.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">as_json_paralellized</span>
<span class="p">{</span>
<span class="ss">deal_types: </span><span class="n">parallelize</span> <span class="p">{</span> <span class="n">deal_types</span> <span class="p">},</span>
<span class="p">}.</span><span class="nf">each_value</span><span class="p">(</span><span class="o">&amp;</span><span class="ss">:join</span><span class="p">).</span><span class="nf">map</span><span class="p">{</span><span class="o">|</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="o">|</span> <span class="p">[</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">[</span><span class="ss">:output</span><span class="p">]]</span> <span class="p">}.</span><span class="nf">to_h</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Each query operates in its own thread. We then join the threads and return the resultant fiber locals. Benchmarks revealed this method to be significantly faster, with about a third as much User CPU time (the number on the far left), and wall clock time (the number on the far right). These benchmarks were taken on a local server, and not on our production application.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">viewthespace</span><span class="p">(</span><span class="n">dev</span><span class="p">)</span><span class="o">&gt;</span> <span class="nb">puts</span> <span class="no">Benchmark</span><span class="p">.</span><span class="nf">measure</span> <span class="p">{</span> <span class="n">as_json</span> <span class="p">}</span>
<span class="mf">0.310000</span> <span class="mf">0.020000</span> <span class="mf">0.330000</span> <span class="p">(</span> <span class="mf">0.415927</span><span class="p">)</span>
<span class="kp">nil</span>
<span class="n">viewthespace</span><span class="p">(</span><span class="n">dev</span><span class="p">)</span><span class="o">&gt;</span> <span class="nb">puts</span> <span class="no">Benchmark</span><span class="p">.</span><span class="nf">measure</span> <span class="p">{</span> <span class="n">as_json_paralellized</span> <span class="p">}</span>
<span class="mf">0.100000</span> <span class="mf">0.020000</span> <span class="mf">0.120000</span> <span class="p">(</span> <span class="mf">0.154479</span><span class="p">)</span>
<span class="kp">nil</span>
</code></pre></div></div>
<p>The reason the concurrent version runs faster is because, as I said earlier, the GIL only cares about ruby work. After the SQL query is called, the ruby process releases the GIL as it waits for the database to respond. This is an ideal issue for concurrent ruby since I/O operations do not factor into the GIL.</p>
<p>Notice that we must call</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">clear_active_connections!</span>
</code></pre></div></div>
<p>Additionally, when we first deployed this code, we ran into errors where our unicorn workers would run out of ActiveRecord connections. We had stuck with the default limit of 5 maximum connections per worker. But in this case we were going to surpass that, because each thread we spawned required its own database connection. Since there were 7 aggregate queries, we bumped the limit up to 8. One connection for each possible thread running an aggregate query, and one for the main thread.</p>
<p>The takeaway here is that, while we did see significant performance gains, it was obvious that we were going “outside the framework” in a way that added complexity, and might seem un-rubyish. When it comes to your own application, consider these tradeoffs before deciding which solution to implement.</p>
<p>I hope this article has made threading in ruby a little easier to understand and shown how it can be used practically in a modern application. Thanks for reading!</p>Nassredean Dean Nasseridean@nasseri.iohttps://nasseri.ioFrequently in our applications as software engineers, we run into situations where we want our code to do multiple things at once. This is the problem of concurrency. Concurrency has an infinite number of use cases. One such use case is that of a webserver. If you were to use a non-concurrent webserver, it would only be able to handle one request at a time. Imagine your site received several requests at the same time. Now the webserver must process each request one at a time, unable to move on to the next request in the queue until the previous one has completed. In order to alleviate this, we might use a concurrent webserver which is capable of processing multiple requests at once.Ruby Metaprogramming by Example2015-11-19T00:00:00-05:002015-11-19T00:00:00-05:00https://nasseri.io/2015/11/19/ruby-metaprogramming-by-example<p>In Ruby the term metaprogramming refers to the dynamic nature of the language, which allows you to define and redefine methods and classes at runtime. Metaprogramming is often presented as a very nebulous and dangerous concept, one that is difficult to explain and hard to wrap your head around, and thus should be avoided. I’d like to to take some time to show a few powerful uses of metaprogramming techniques in real live code.</p>
<!--more-->
<p>One of the most common, and most misunderstood, aspects of ruby programming is the monkey patch. The monkey patch refers to the ability for ruby to dynamically define and override methods on existing classes and modules at runtime. For instance:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">to_s</span>
<span class="o">=&gt;</span> <span class="s2">"[1, 2, 3, 4]"</span>
<span class="k">class</span> <span class="nc">Array</span>
<span class="k">def</span> <span class="nf">to_s</span>
<span class="s1">'[]'</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">to_s</span>
<span class="o">=&gt;</span> <span class="s2">"[]"</span>
</code></pre></div></div>
<p>This is admittedly not a very useful monkey patch, but it demonstrates that we can reach into any class, including core classes like Array, and modify it on the fly. The monkey patch is actually just a symptom of a more general paradigm in ruby.</p>
<h4 id="open-classes">Open Classes</h4>
<p>In Ruby, class definitions are quite flexible.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">A</span>
<span class="vi">@color</span> <span class="o">=</span> <span class="s1">'red'</span>
<span class="nb">puts</span> <span class="vi">@color</span>
<span class="k">end</span>
<span class="o">=&gt;</span> <span class="n">red</span>
<span class="o">=&gt;</span> <span class="kp">nil</span>
<span class="k">class</span> <span class="nc">A</span>
<span class="vi">@color</span> <span class="o">=</span> <span class="s1">'blue'</span>
<span class="nb">puts</span> <span class="vi">@color</span>
<span class="k">end</span>
<span class="o">=&gt;</span> <span class="n">blue</span>
<span class="o">=&gt;</span> <span class="kp">nil</span>
</code></pre></div></div>
<p>Ruby executes the body of the class definition as it would any other code. Interestingly the class keyword does not create the C class twice, which is what allows our earlier example of monkey patching to work. Take a look at the following example:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># case 1</span>
<span class="k">class</span> <span class="nc">B</span>
<span class="k">def</span> <span class="nf">method1</span>
<span class="mi">1</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1"># case 2</span>
<span class="k">class</span> <span class="nc">B</span>
<span class="k">def</span> <span class="nf">method2</span>
<span class="mi">2</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">B</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">method1</span>
<span class="o">=&gt;</span> <span class="mi">1</span>
<span class="no">B</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">method2</span>
<span class="o">=&gt;</span> <span class="mi">2</span>
</code></pre></div></div>
<p>In the first case, class B does not exist, so Ruby defines the class and method1. The second time class B is referenced, the class exists, so instead of redefining class B, it opens the existing class and defines method2 there. The method1 is still accessible because it is still the same class.</p>
<p>When opening a class we can even reference class level instance variables.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="k">class</span> <span class="nc">C</span>
<span class="vi">@color1</span> <span class="o">=</span> <span class="s1">'red'</span>
<span class="nb">puts</span> <span class="vi">@color1</span>
<span class="k">end</span>
<span class="o">=&gt;</span> <span class="n">red</span>
<span class="o">=&gt;</span> <span class="kp">nil</span>
<span class="k">class</span> <span class="nc">C</span>
<span class="vi">@color2</span> <span class="o">=</span> <span class="s1">'blue'</span>
<span class="nb">puts</span> <span class="vi">@color1</span>
<span class="nb">puts</span> <span class="vi">@color2</span>
<span class="k">end</span>
<span class="o">=&gt;</span> <span class="n">red</span>
<span class="o">=&gt;</span> <span class="n">blue</span>
<span class="o">=&gt;</span> <span class="kp">nil</span>
</code></pre></div></div>
<p>Now its time to see some examples of metaprogramming in action. Lets take a look at the <a href="https://github.com/michaeldv/awesome_print">awesome_print</a> gem. awesome_print is a gem for pretty printing Ruby objects. Awesome Print works with IRB as well as Pry, but for the time being, we will be focused on the IRB code path. For those that might now know IRB is the Ruby REPL distributed with MRI, and written in ruby.</p>
<p>To use awesome_print, first install the gem.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>gem <span class="nb">install </span>awesome_print
</code></pre></div></div>
<p>Then in your ~/.irbrc</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="err">“</span><span class="n">awesome_print</span><span class="err">”</span> <span class="no">AwesomePrint</span><span class="p">.</span><span class="nf">irb!</span>
</code></pre></div></div>
<p>Taking a look at the project we see that awesome_print.rb has two main files, inspector and formatter, and then a number of core extensions which do various monkey patches on core ruby classes. Lets take a look at the irb! method defined on the inspector class. This function is responsible for intercepting IRB output and formatting it appropriately, in other words it is the “hook” into IRB’s output, intercepting what is outputted and replacing it with its own formatting logic.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/awesome_print/inspector.rb (26)</span>
<span class="k">def</span> <span class="nf">irb!</span>
<span class="k">return</span> <span class="k">unless</span> <span class="k">defined?</span><span class="p">(</span><span class="no">IRB</span><span class="p">)</span>
<span class="k">unless</span> <span class="no">IRB</span><span class="p">.</span><span class="nf">version</span><span class="p">.</span><span class="nf">include?</span><span class="p">(</span><span class="s2">"DietRB"</span><span class="p">)</span>
<span class="no">IRB</span><span class="o">::</span><span class="no">Irb</span><span class="p">.</span><span class="nf">class_eval</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">output_value</span>
<span class="n">ap</span> <span class="vi">@context</span><span class="p">.</span><span class="nf">last_value</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="o">...</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1"># http://rxr.whitequark.org/mri/source/lib/irb.rb</span>
<span class="k">def</span> <span class="nf">output_value</span> <span class="c1"># :nodoc:</span>
<span class="nb">printf</span> <span class="vi">@context</span><span class="p">.</span><span class="nf">return_format</span><span class="p">,</span> <span class="vi">@context</span><span class="p">.</span><span class="nf">inspect_last_value</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Looking through the IRB source on line 661 we see the method output_value. If we monkey patch it like so, check what happens:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">IRB</span>
<span class="k">class</span> <span class="nc">Irb</span>
<span class="k">def</span> <span class="nf">output_value</span>
<span class="nb">puts</span> <span class="s1">'cow'</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="s1">'dog'</span>
<span class="o">=&gt;</span> <span class="n">cow</span>
</code></pre></div></div>
<p>So we can see that this is the method that prints to STDOUT. Returning back to the inspector irb! method, we see that this is not quite the same code as we had. Instead of opening up the class using the class keyword we use a class_eval.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">IRB</span><span class="o">::</span><span class="no">Irb</span><span class="p">.</span><span class="nf">class_eval</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">output_value</span>
<span class="nb">puts</span> <span class="s1">'cow'</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The class_eval method allows us to execute code in the context of a class. This allows us to open up classes without the class keyword. We can’t have a class definition in a method body, so instead we can use class_eval, which allows us to monkey patch the output_value method dynamically. The class_eval method is very powerful, and allows us to completely redefine classes at runtime.</p>
<p>Lets take a look at another metaprogramming trick used in AwesomePrint. Open up lib/awesome_print/formatter.rb and take a look at the format method:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">format</span><span class="p">(</span><span class="n">object</span><span class="p">,</span> <span class="n">type</span> <span class="o">=</span> <span class="kp">nil</span><span class="p">)</span>
<span class="n">core_class</span> <span class="o">=</span> <span class="n">cast</span><span class="p">(</span><span class="n">object</span><span class="p">,</span> <span class="n">type</span><span class="p">)</span>
<span class="n">awesome</span> <span class="o">=</span> <span class="k">if</span> <span class="n">core_class</span> <span class="o">!=</span> <span class="ss">:self</span>
<span class="nb">send</span><span class="p">(</span><span class="ss">:"awesome_</span><span class="si">#{</span><span class="n">core_class</span><span class="si">}</span><span class="ss">"</span><span class="p">,</span> <span class="n">object</span><span class="p">)</span> <span class="c1"># Core formatters.</span>
<span class="k">else</span>
<span class="n">awesome_self</span><span class="p">(</span><span class="n">object</span><span class="p">,</span> <span class="n">type</span><span class="p">)</span> <span class="c1"># Catch all that falls back to object.inspect.</span>
<span class="k">end</span>
<span class="n">awesome</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The format method is the main entry point to format an object. After determining the class of the object it is about to format, the method dynamically calls the corresponding formatter using send.</p>
<p>The final trick I’d like to show is located lib/awesome_print/core_ext/kernel.rb. This file contains extensions on the core Kernel class. This example comes from the book Metaprogramming Ruby by Paolo Perrota.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">Kernel</span>
<span class="k">def</span> <span class="nf">ai</span><span class="p">(</span><span class="n">options</span> <span class="o">=</span> <span class="p">{})</span>
<span class="n">ap</span> <span class="o">=</span> <span class="no">AwesomePrint</span><span class="o">::</span><span class="no">Inspector</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">options</span><span class="p">)</span>
<span class="n">awesome</span> <span class="o">=</span> <span class="n">ap</span><span class="p">.</span><span class="nf">awesome</span> <span class="nb">self</span>
<span class="k">if</span> <span class="n">options</span><span class="p">[</span><span class="ss">:html</span><span class="p">]</span>
<span class="n">awesome</span> <span class="o">=</span> <span class="s2">"&lt;pre&gt;</span><span class="si">#{</span><span class="n">awesome</span><span class="si">}</span><span class="s2">&lt;/pre&gt;"</span>
<span class="n">awesome</span> <span class="o">=</span> <span class="n">awesome</span><span class="p">.</span><span class="nf">html_safe</span> <span class="k">if</span> <span class="k">defined?</span> <span class="no">ActiveSupport</span>
<span class="k">end</span>
<span class="n">awesome</span>
<span class="k">end</span>
<span class="k">alias</span> <span class="ss">:awesome_inspect</span> <span class="ss">:ai</span>
<span class="k">def</span> <span class="nf">ap</span><span class="p">(</span><span class="n">object</span><span class="p">,</span> <span class="n">options</span> <span class="o">=</span> <span class="p">{})</span>
<span class="nb">puts</span> <span class="n">object</span><span class="p">.</span><span class="nf">ai</span><span class="p">(</span><span class="n">options</span><span class="p">)</span>
<span class="n">object</span> <span class="k">unless</span> <span class="no">AwesomePrint</span><span class="p">.</span><span class="nf">console?</span>
<span class="k">end</span>
<span class="k">alias</span> <span class="ss">:awesome_print</span> <span class="ss">:ap</span>
<span class="kp">module_function</span> <span class="ss">:ap</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Here we can see that AwesomePrint has opened up the Kernel module, and is defining two methods ai and ap. Why define them on the Kernel module? Since Kernel is included in Object, and object is in the ancestor chain of nearly every Ruby class, defining a method within Kernel is a handy way of making it available nearly everywhere.</p>
<p>I hope you enjoyed this little intro on Ruby metaprogramming. To gain a deeper insight into Ruby metaprogramming, I highly suggest <a href="https://pragprog.com/book/ppmetr2/metaprogramming-ruby-2">Palo Perrotta’s book</a>.</p>Nassredean Dean Nasseridean@nasseri.iohttps://nasseri.ioIn Ruby the term metaprogramming refers to the dynamic nature of the language, which allows you to define and redefine methods and classes at runtime. Metaprogramming is often presented as a very nebulous and dangerous concept, one that is difficult to explain and hard to wrap your head around, and thus should be avoided. I’d like to to take some time to show a few powerful uses of metaprogramming techniques in real live code.