run executes an external command and :out tells it to capture the
command’s output, making it available as $proc.out. The command is
a list, with the first element being the actual executable and the rest of
the elements are command line arguments to this executable.

Here git log gets the options --date short --pretty=format:%ad!%an, which
instructs it to produce lines like 2017-03-01!John Doe. This line
can be parsed with a simple call to $line.split: '!', 2, which splits
on the ! and limits the result to two elements. Assigning it to a
two-element list ( $date, $author ) unpacks it. We then use hashes to
count commits by author (in %total), by author and date (%by-author),
and finally by date. In the second case, %by-author{$author} isn’t
even a hash yet and we can still hash-index it. This is due to a feature
called autovivification, which automatically creates (“vivifies”) objects
where we need them. The use of ++ creates integers, {...} indexing creates
hashes, [...] indexing, .push creates arrays, and so on.

To get from these hashes to the top contributors by commit count, we can
sort %total by value. Since this sorts in ascending order, sorting
by the negative value returns the list in descending order. The list contains
Pair objects, where we only want the
first five, and only their keys:

my @top-authors = %total.sort(-*.value).head(5).map(*.key);

For each author, we can extract the dates of their activity and their
commit counts like this:

To make this run, you have to install Python 2.7 and matplotlib1. You can do
this on Debian-based Linux systems with apt-get install -y python-matplotlib.
The package name is the same on RPM-based distributions such as CentOS or SUSE
Linux. MacOS users are advised to install Python 2.7 through homebrew and
macports and then use pip2 install matplotlib or pip2.7 install
matplotlib to get the library. Windows installation is probably easiest
through the conda package manager, which offers
pre-built binaries of both Python and matplotlib.

When you run this script with python2.7 dates.py, it opens a GUI window,
showing the plot and some controls, which allow you to zoom, scroll, and
write the plot graphic to a file:

Bridging the Gap

The Rakudo Perl 6 compiler comes with a handy library for calling foreign
functions – called NativeCall – which allows you to
call functions written in C, or anything with a compatible binary interface.

The Inline::Python library uses
the native call functionality to talk to Python’s C API and offers
interoperability between Perl 6 and Python code. At the time of writing, this
interoperability is still fragile in places, but can be worth using for
some of the great libraries that Python has to offer.

To install Inline::Python, you must have a C compiler available, and then
run

The arguments that you pass to call are Perl 6 objects, such as the three Int
objects in this example. Inline::Python automatically translates them into
the corresponding Python built-in data structure. It translates numbers,
strings, arrays, and hashes. Return values are also translated in the opposite
direction, though since Python 2 does not distinguish properly between
byte and Unicode strings, Python strings end up as buffers in Perl 6.

Objects that Inline::Python cannot translate are handled as opaque objects
on the Perl 6 side. You can pass them back into Python routines (as shown
with the print call above) and you can call methods on them:

say $date.isoformat().decode; # 2017-01-31

Perl 6 exposes attributes through methods, so Perl 6 has no syntax for
accessing attributes from foreign objects directly. For instance, if you try to access
the year attribute of datetime.date through the normal method
call syntax, you get an error:

say $date.year;

dies with

'int' object is not callable

Instead, you have to use the getattr builtin:

say $py.call('__builtin__', 'getattr', $date, 'year');

Using the Bridge to Plot

We need access to two namespaces in Python, datetime and matplotlib.pyplot,
so let’s start by importing them and writing some short helpers:

The Perl 6 call plot('subplots') corresponds to the Python code
fig, subplots = plt.subplots(). Passing arrays to Python functions needs
a bit of extra work, because Inline::Python flattens arrays. Using an extra $
sigil in front of an array puts it into an extra scalar and thus prevents
the flattening.

Now we can actually plot the number of commits by author, add a legend, and
plot the result:

Stacked Plots

I am not yet happy with the plot, so I want to explore using stacked
plots for presenting the same information. In a regular plot, the
y-coordinate of each plotted value is proportional to its value. In a
stacked plot, it is the distance to the previous value that is
proportional to its value. This is nice for values that add up to a
total that is also interesting.

Matplotlib offers a method called
stackplot
for this task. Contrary to multiple plot calls on a subplot object, it
requires a shared x-axis for all data series. Hence we must construct
one array for each author of git commits, where dates with no value are
set to zero.

This time we have to construct an array of arrays where each inner array
has the values for one author:

Comparing this to the previous visualization reveals a discrepancy:
There were no commits in 2014, and yet the stacked plot makes it appear
this way. In fact, the previous plots would have shown the same
“alternative facts” if we had chosen lines instead of points. It comes
from matplotlib (like nearly all plotting libraries) interpolates
linearly between data points. But in our case, a date with no data
points means zero commits happened on that date.

To communicate this to matplotlib, we must explicitly insert zero values
for missing dates. This can be achieved by replacing

my @dates = %dates.keys.sort;

with the line

my @dates = %dates.keys.minmax;

The minmax method
finds the minimal and maximal values, and returns them in a
Range. Assigning the range to an
array turns it into an array of all values between the minimal and the
maximal value. The logic for assembling the @stack variable already
maps missing values to zero.

The result looks a bit better, but still far from perfect:

Thinking more about the problem, contributions from separate days should
not be joined together, because it produces misleading results.
Matplotlib doesn’t support adding a legend automatically to stacked
plots, so this seems to be to be a dead end.

Since a dot plot didn’t work very well, let’s try a different kind of
plot that represents each data point separately: a bar chart, or more
specifically, a stacked bar chart. Matplotlib offers the bar plotting
method where the named parameter bottom can be used to generate the
stacking:

This produces the first plot that’s actually informative and not
misleading (provided you’re not color blind):

If you want to improve the result further, you could experiment with
limiting the number of bars by lumping together contributions by week or
month (or maybe $n-day period).

Idiomatic Use of Inline::Python

Now that the plots look informative and correct, it’s time to explore how
to better emulate the typical Python APIs through Inline::Python.

Types of Python APIs

Python is an object-oriented language, so many APIs involve method
calls, which Inline::Python helpfully automatically translates for us.

But the objects must come from somewhere and typically this is by
calling a function that returns an object, or by instantiating a class.
In Python, those two are really the same under the hood, since
instantiating a class is the same as calling the class as if it were a
function.

An example of this (in Python) would be

from matplotlib.pyplot import subplots
result = subplots()

But the matplotlib documentation tends to use another, equivalent
syntax:

import matplotlib.pyplot as plt
result = plt.subplots()

This uses the subplots symbol (class or function) as a method on the
module matplotlib.pyplot, which the import statement aliases to plt.
This is a more object-oriented syntax for the same API.

Mapping the Function API

The previous code examples used this Perl 6 code to call the subplots
symbol:

This makes the functions’ usage quite nice, but comes at the cost
of duplicating their names. One can view this as a
feature, because it allows the creation of different aliases, or as a
source for bugs when the order is messed up, or a name misspelled.

How could we avoid the duplication should we choose to create wrapper
functions?

This is where Perl 6’s flexibility and introspection abilities pay off.
There are two key components that allow a nicer solution: the fact that
declarations are expressions, and that you can introspect variables for
their names.

The first part means you can write mysub my ($a, $b), which
declares the variables $a and $b, and calls a function with those
variables as arguments. The second part means that $a.VAR.name returns
a string '$a', the name of the variable.

Let’s combine this to create a wrapper that initializes subroutines for
us:

This avoids duplicating the name, but forces us to use some lower-level
Perl 6 features in sub pysub. Using ordinary variables means that accessing their
.VAR.name results in the name of the variable, not the name of the
variable that’s used on the caller side. So we can’t use slurpy
arguments as in

sub pysub(Str $namespace, *@subs)

Instead we must use
|args to obtain the rest of the arguments in a
Capture. This doesn’t
flatten the list of variables passed to the function, so when we iterate
over them, we must do so by accessing args[0]. By default, loop
variables are read-only, which we can avoid by using <-> instead of
-> to introduce the signature. Fortunately, that also preserves the
name of the caller side variable.

An Object-Oriented Interface

Instead of exposing the functions, we can also create types that emulate
the method calls on Python modules. For that we can implement a class with a
method FALLBACK, which Perl 6 calls for us when calling a method that is not
implemented in the class:

Class PyPlot inherits directly from Mu, the root of the Perl 6 type
hierarchy, instead of Any, the default parent class (which in turn inherits
from Mu). Any introduces a large number of methods that Perl 6 objects
get by default and since FALLBACK is only invoked when a method is not
present, this is something to avoid.

The method TWEAK is another method that
Perl 6 calls automatically for us, after the object has been fully
instantiated. All-caps method names are reserved for such special purposes.
It is marked as a submethod, which means it is not inherited into subclasses.
Since TWEAK is called at the level of each class, if it were a regular
method, a subclass would call it twice implicitly. Note that TWEAK is only
supported in Rakudo version 2016.11 and later.

There’s nothing specific to the Python package matplotlib.pyplot in class
PyPlot, except the namespace name. We could easily generalize it to any
namespace:

Passing in any wrapped Python module other than matplotlib.pyplot results
in a type error.

Summary

We’ve explored several ways to represent commit occurrence in plots and
utilized Inline::Python to interface with a Python based plotting library.

A bit of Perl 6 meta programming allowed us to emulate different kinds of
Python APIs pretty directly in Perl 6 code, allowing a fairly direct
translation of the original library’s documentation into Perl 6 code.

1: The reason why Python 2.7 has to be used is that, at the time of writing, Inline::Python does not yet support Python 3.

Site Map

Contact Us

Legal

PerlTricks.com and the authors make no representations or warranties with respect to the accuracy or completeness of the contents of all work on this website and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended. The advice and strategies published on this website may not be suitable for every situation. All work on this website is provided with the understanding that PerlTricks.com and the authors are not engaged in rendering legal, accounting, or other professional services. Neither PerlTricks.com nor the authors shall be liable for damages arising herefrom.