A week ago today, I gave a talk at PyCon Australia in Sydney. I discussed refactoring
in Python, and how to build refactoring tools using nothing but the standard library,
building up from concepts to syntax trees to how the lib2to3 module works. The talk
finished with the announcement of the open source refactoring framework I built at
Facebook, Bowler, which I built using those same concepts.
I really appreciated the questions from the audience and in the halls afterwards, and
we really enjoyed our time in Sydney. Thanks to everyone that made PyCon so great!

This past Friday, I presented a talk at PyCon US 2018 – the first time I’ve
been fortunate enough to attend. The talk was focused on achieving high
performance from modern Python services through the use of AsyncIO and the
multiprocessing modules. The turnout was better than I could have ever
expected, and I was really happy to hear from everyone that stopped by the
Facebook booth to ask questions, discuss Facebook engineering practices, or
even just say “hello”. Thank you to everyone who made my first PyCon amazing!

I’m currently preparing a talk for PyCon 2018 on using asyncio with
multiprocessing for highly-parallel monitoring and/or scraping workloads.
To go with this talk, I’m working on some simple example code that I hope
to publish on Github. This will be my first major conference talk, so I’m
both excited and absolutely terrified! 😅

I’m looking forward to giving that talk, and will post a video here afterwards!

Posted on January 20, 2015, updated on January 21, 2015»Tagged as
games,
notes

It’s no secret that I’m a huge fan of space, video games, or video games set
in space. I was a very early backer of Star Citizen, and am still eagerly
awaiting the release of the persistent universe. But I also backed
Elite: Dangerous as well, and although the initial scope was far more
limited than Star Citizen’s, Elite has already seen a “1.0” release, and I’ve
been playing it an awful lot, all the way up from the first beta.

And damn, it’s good!

Voyager 1, found in the Sol system, ~2M Ls out

The discipline and focus of the team at Frontier Development has produced one
of the most polished space sim games I’ve ever played, and while there are
plenty of gaps in features compared to the competition, everything that’s there
is done so well that I continue to be impressed every time I get in that cockpit
and launch into the void.

After putting dozens of hours into the final release, I’ve gathered a few
notes here that will hopefully help others in their journies to the stars.
I will attempt to keep this up to date for as long as I’m playing the game.

When I was in elementary school, I discovered a series of books in the library
that detailed, in glorious full page color, photos of the outer solar system as
captured by Voyager 1 and Voyager 2 on their epic journeys to places never
before explored by humanity. I was so stunned and enthralled at the thoughts
of visiting those distant worlds; it indeed captured my imagination, and
continues to inspire me even now, two decades after I first learned of their
amazing quests.

Jupiter’s Great Red Spot, as photographed by Voyager 1.

I’ve spent countless hours dreaming of visiting the planets and stars of our
universe: in games like Wing Commander, Freespace, Freelancer,
Mass Effect, and X3, exploring fictional galaxies, and in
Orbiter Sim, recreating my own journeys to the planets and moons of our own
solar system. I can’t wait for Star Citizen and X Rebirth to give me
further ways to escape the confines of Earth and experience worlds that
I will never be able to visit otherwise.

Volcanic eruption on Io, as photographed by Voyager 1

And I have NASA to thank for inspiring so many of these wonderful outlets, from
the Mercury, Gemini, and Apollo projects that took us to the Moon, the Voyager
missions for showing us just how beautiful our own back yard really is, to
the Mars rovers for exploring our closest neighbor, and to future NASA
missions like New Horizons that are leading the way towards understanding
our tiny place in this vast universe.

I simply find it incredible that not only is Voyager still working after four
decades of exceptional service, but that it’s still capable of aiming its
antennas at Earth, sending a coherent signal using only 23W of power, and
actually sending useful data back to us. That alone is a hugely amazing
achievement, and I think everyone at NASA should be eternally proud of their
accomplishment. The fact that we now have even more data points regarding the
boundaries of our solar system is just pure icing on top of that cake, and I
congratulate everyone involved in the project.

If you asked me what my favorite spacecraft is, fictional or not, it wouldn’t
be the Enterprise, the Millenium Falcon, the Normandy, or even the Apollo
rockets; it would be Voyager. And it will forever hold a special place in my
heart.

I set out to write an article about how I feel Android provides unique
affordances that create a unique cohesive mobile experience (more on that
below) rather than talking openness, features and apps. However, the more
time I spent living with Android it became obvious that being able to do
anything and suit a variety of needs is a pillar of the Android experience.

I was trying to find out why Android felt so remarkably different to use,
beyond aesthetics. I found there are a few pieces that help contribute to
this magical user experience: the global back button, intents and
Google Now.

Delight. It’s what designers strive to produce in the experiences they
craft. Google Now has hit the nail on the head. I can’t even count the
number of times friends have pulled out their phone to show me the smart
things Google Now did for them.

Armin Ronacher, on Firefox’s removal of the option to disable Javascript:

My immediate response to change of removing the switch was:
“Thank god, that should have happened ages ago”.

…

You don’t get extra privacy by disabling JavaScript. I can fully track you
even without JavaScript. At the same time I can enhance your browser
experience through better written JavaScript code that allows me to do things
with your browser that plain HTML does not allow.

If you’re really paranoid about your privacy, use Firefox, enable the
Do Not Track flag in your options, enable click to play for plugins
like Flash or Silverlight, and install the Ghostery and Adblock
extensions. Anything else is just going to break the web at your own expense.

The answer is to make digital games so attractive that players will abandon
physical discs on their own. (One might call this the Steam strategy.)
Microsoft could have avoided this whole fiasco by maintaining the old
disc-based ecosystem while softly undermining it with three moves that
create an alternate digital future.

…

Combined, these three changes would destroy the traditional retail market.
The $40 price would make digital games cheaper at release; the ongoing heavy
sales would undercut the used games market; and persistence would make digital
games easier to maintain across multiple devices. Microsoft needs to make
buying games digitally a better deal for the consumer than buying them
physically.

I was extremely disappointed to hear Microsoft cave in to rabid demands to
maintain the status quo. I was really looking forward to their plans for
combined physical and digital ownership, where I could get all the benefits of
buying physical copies, including special and collectors’ editions of my
favorite titles, while simultaneously retaining all the benefits of a digital
copy, like the ability to forego disc-swapping.

One of my favorite projects that I’ve created has to be ZNC Push, a plugin
for ZNC that generates push notifications for IRC highlights and private
messages. It supports a wide array of push services for all of the major mobile
platforms, and gives the user deep flexibility in choosing how and when to
trigger these notifications.

However, I have very little insight into usage statistics for the plugin.
Pushover is the only service that gives me gross usage data, but
considering ZNC Push supports six other networks as well as custom push URLs,
this only represents an unknown portion of the total usage. But I will
continue to draw baseless conclusions from it regardless! Do note that I
can’t see anything more than raw volume of notifications; I do not have access
to who is sending/receiving these, nor the contents of those messages.

So without further ado, some pretty graphs straight from the Pushover dashboard:

ZNC Push reached a milestone of sending 15,000 push notifications in May
through the Pushover service!

Push volume has been growing steadily since Pushover support was added, with a
small dip in December and January that is likely attributeable to the holidays
for most users. May marked the second occasion that ZNC Push has exceeded
the volume “cap”, and Pushover has again graciously increased that in support
of this open source project. At this sort of grawth rate, I would expect the
next volume cap (25,000/mo) will be reached by October at the latest. I’m
excited to see such growth for a small side project with such a niche use case.

Sometimes, it’s amazing to stop and realize just how far this project has come.
What started in January of 2011 as a basic module for the Notifo push service,
has blossomed into a mature system supporting multiple push services and far
more customization options than I had ever envisioned from the beginning.
Knowing that it’s being used by so many people, and helping them stay more
connected with their online communities, makes me a very happy engineer.

Many thanks to Pushover for granting such a large volume of free
notifications to ZNC Push. If you’re not already using them for your push
service of choice, I highly recommend them. They have apps for both
Android and iPhone, and they continue
to improve them and add new features over time.

And as always, my eternal appreciation to everyone who has contributed time
and energy to helping me improve the module over the past two and a half
years! The project wouldn’t be where it is today without your support.

Eighteen months ago, I wrote an article about my Favorite Android Apps,
but a lot has changed with Android and its app ecosystem since then. Developers
have started putting more focus on design and usability when creating apps, and
of course newer apps have come along that I have come to rely upon. I myself
have upgraded to newer devices, with a Nexus 7 replacing my Galaxy Tab 10.1,
and a Galaxy Nexus from work has sadly replaced my beloved but aging Nexus S.
So what follows are some of my favorite new apps from 2012, and some follow-up
on what my previous favorite apps look like in modern attire. All screenshots
are taken from devices running Android 4.2.1 (CM 10.1 nightlies).

Along a similar vein as my previous post on Why I Prefer Android,
Ralf Rottmann, a self-described “Apple fanboy”, has described why I prefer
Android better than I could in my own words:

The latest version of Android outshines the latest version of iOS in almost
every single aspect. I find it to be better in terms of the performance,
smoothness of the rendering engine, cross-app and OS level integration,
innovation across the board, look & feel customizability and variety of the
available apps.

On the topic of app and system integration:

Another great example: Sharing stuff on social networks. On iOS, I have to
rely on the developers again. Flipboard, as one of the better examples,
gives me the ability to directly share with Google+, Twitter and Facebook.
On my Nexus 4, I have 20+ options. That is, because every app I install can
register as a sharing provider. It’s a core feature of the Android operating
system.

…

All of this is entirely impossible on iOS today. I’ve stopped counting how
often I felt annoyed because I clicked a link to a location in Mobile Safari
and would have loved the Google Maps app to launch. Instead, Apple’s own Maps
app is hardcoded into the system. And there’s no way for me to change it.

Regarding possibilities for app developers:

On iOS, many things I always wished to see being developed, simply cannot be
done because of the strict sandbox Apple enforces around apps.

…

I also have apps [on Android] that give me great insight into the use of
mobile data across the device and all apps. Or the battery consumption.
Or which apps talk home and how frequently.

None of it is available for iOS. And possibly won’t be at any time in the
near future.

And summing up the way I’ve felt for a long time when using iOS devices:

… whenever I grab my iPhone for testing purposes, iOS feels pretty old,
outdated and less user friendly. For me, there currently is no way of going
back. Once you get used to all of these capabilities, it’s hard to live
without them.

There are many things Apple got right with iOS, like making a consistent user
experience, and encouraging users to spend money in the market for quality apps.
But when they place so many restrictions and limitations on how you can use your
phone, and what your software is allowed to do on your own device, I gladly
give up those things that make iOS so great for the freedom to run apps that
can do what I want — and expect — from a modern computing device.

For the last couple years, this site has been managed and updated using the open
source static site generator called Poole. It is an excellent and simple
system, comprised of two Python files — the Poole source code and a
site-specific macros.py — a simple page template, input documents and
static content. This is more than enough for a small static website, but
more complicated sites, like blogs, require quite a few macros in order to
generate things like archives, tag pages, or even RSS feeds. Over the 18
months that I worked with Poole, my macro file had gotten a bit disorganization,
and the limitation of working from only a single template file was starting to
strain on what I could do with it.

Now keep in mind, I still like many of the features that drew me to Poole in the
first place, such as the use of a document-centric build process, with YAML
headers for defining page metadata rather than an inline, proprietary format
used by projects like Jekyll. I also enjoy the simplicity of the content
representations, but the lack of an extensible content pipeline is my biggest
complaint. Supporting formats other than Markdown requires adding yet another
hook to the macro file; generating archives and tag pages required hacking in a
multi-phase build using os.exec(); and inlining another page’s content
post-render was just not possible, resulting in Markdown named-link collisions
when rendering multiple posts to a single page.

I wanted something new. So, for the past few months, I’ve been working on
Nib.

The resulting design is based heavily on the concepts of Poole, but built
around a primary goal of producing a proper content pipeline that is
simultaneously aware of the differentiation between resources and documents, and
defines multiple stages where plugins can hook into the process and alter or
generate page contents at build time. Indeed, most of the actual functionality
of Nib is contained within a handful of plugins, while the main module merely
defines a framework for the content pipeline.

In effect, adding support for more content or resource formats should be as
simple as adding a new plugin attached to the appropriate file extensions.
Advanced content manipulation, generating “virtual pages”, and aggregating
pages or documents into multiple locations are all possible as well.

The Markdown plugin is 14 lines of code; the LessCSS plugin is 13; even the
blog plugin — which generates the archive pages, tag listings, and Atom feed —
takes only 86 lines to do a better job than the old Poole macros that required
double the effort.

Today, I’ve thrown the switch. While it looks nearly identical to the old
site generated by Poole, the page you are now reading has officially been generated
by Nib. The news feed sees the biggest makeover, having changed formats from
RSS to Atom, and now offers full content posts. Link-style posts, a format that
I’ve been wanting to experiment with, are also a new option. Both features
would have required more effort on Poole than I was interested in expending,
but the new architecture has allowed me to indulge myself.

Nib is still in an extremely early, and unstable, phase though. It works for
the needs of my site, and does come with some basic documentation and a
sample site to start from, but it’s far from complete. Near term goals
include adding support for an intelligent menu, as well as support for more
content and resource formats, like reStructuredText or SASS. Contributions are
always welcome though, even at this early stage. Nib is liberally licensed,
and I would love to hear feedback from anyone trying to use it. Hopefully it
will be useful for someone other than myself.

I prefer Android because it allows apps to do more things for the user, and
allows them to better integrate with the system as a whole.

I can replace the on screen keyboard with one that has a full five-row keyboard
for times when I SSH into a machine. On a similar vein, when I SSH into a
machine, I can actually leave the SSH session running in the background while
I switch to another app, without fearing that the OS is about to kill my SSH
app while I’m looking something up or responding to a text message. I can also
leave my IRC client running in the background without it constantly needing to
reconnect when I switch back to it.

Intents in Android, especially in combination with the global Share mechanism,
allow any app to receive arbitrary data from any other app, meaning apps don’t
need to know about specific apps or services in order to integrate with them.
Clicking on a URL allows you to choose which browser (or set a default) to open
the link in, allowing you to use alternate browsers (or alternate email clients,
SMS apps, dialers, etc); tapping Share in the browser allows you to send the
current URL to any application that can receive a URL, making apps like
Instapaper, Pinboard, and 3rd party Twitter clients have the same capabilities
as first party applications.

Sideloading apps means I’m not limited to installing programs from the Android
Market/Play Store, and can do things like buy apps directly from the Humble
Indie Bundle and install them on my own.

Proper background service support, and allowing apps to affect things outside
their sandbox, lets me run programs like Locale that can monitor the phone’s
status, location, etc, and modify the phone’s settings automatically based on a
set of conditions that I’ve pre-arranged. My phone automatically silences itself
at night time and while I’m physically at work, turns on my Wifi when I’m at
home or work while defaulting it off when I’m out and about, and more.

The past two months have been intense. At the beginning of May, I was let go
from BioWare and Electronic Arts in a round of budget cuts for my studio.
Since then, I’ve played far too many video games (and beaten multiple titles),
slogged through innumerable interviews, redesigned my website from the ground
up, and celebrated my seventh wedding anniversary in some of the best summer
weather I’ve ever experienced.

Oh, and I got a job at Facebook.

On May 3rd, BioWare informed me that my position in the studio was being
eliminated. My terrible luck had struck again. But this time was different;
within a few days, a recruiter from Facebook found this website and my open
source work, reached out to me, and encouraged me to apply for an engineering
role. Four weeks later, after phone screens and an on-site interview, I was
offered a position as a Production Engineer. My luck had not only reversed,
it had gone off the scale in the opposite direction.

Alongside Facebook, all my other leads paled in comparison. I wanted to stay
in gaming, but here was the opportunity of a lifetime, to join one of the
biggest and best engineering companies in the world, and the chance to learn
from some of the brightest minds in the industry. The position is similar to
the role I had at BioWare: writing and maintaining software to manage and
automate the vast array of infrastructure that supports the front-end
applications and engineers. It was an offer I couldn’t refuse.

Monday was my orientation, and I was immediately impressed by the level of
passion, vision and dedication from the entire team. Boz discussed the
culture of “hack”, and told a story about building a loft in a war room when it
couldn’t fit everyone in just two dimensions. Chris Cox gave an inspiring
talk about where Facebook is heading, and I got a surreal sense of wonder when
he mentioned our long term goals or the level of impact we have on global
society and the way people communicate. It feels amazing to be part of a
company that truly wants to change the world and improve the lives of hundreds
of millions of people.

Even the training process embodies those same ideals. The first few weeks,
every engineer goes through “bootcamp”, attending learning and development
sessions, sitting among other bootcamp engineers, and working with veteran
teammates on real tasks to get familiar with the infrastructure, tools and
codebase. Everyone is given the chance to progress at their own pace, with as
much or as little help as needed. I’ve already learned so much in a few days,
and I’ve also had the chance to share my own knowledge and experience with my
fellow newcomers. And yet there’s still so much left to cover.

Wednesday was my first day at a desk. Taped to the monitor was a sheet of
paper printed with bold, red lettering.

I’ve been spending time migrating my server from Ubuntu 10.04 to Arch Linux,
and in the process I thought very deeply about every PHP application installed
on the old server. Five out of the six PHP apps were only there to support the
one that really stood on its own: MantisBT, my long-standing bug tracker of
choice.

Now, I’ve been a core developer, and de facto release manager, for the project
for many years — since I was still in university and getting paid by my
then-employer to contribute features and plugins that they wanted to use for
their engineering team. Those plugins, like Source Integration, wouldn’t be
free without me fighting to license and release them for the community. And
I wouldn’t be where I am today without the experience and help I received in
turn from the very same community.

I know I haven’t been as involved in the project these days as I would like to
be; there is an endless list of features and improvements to make to both the
core system and the array of plugins I’ve created for it. Some great community
members have stepped up and filled my place at times, while I have at least
tried to stay active on the mailing lists and in the IRC channels. I’ve still
guided and cut the last couple releases, but I haven’t played a part in shaping
the future of the project.

There are multiple competing visions charting new paths for the aging project,
with conflicting goals and revision histories, and it really needs a stronger
leader to take the reins and guide the project to its next milestones. I’m
unfortunately not the person to fill this role, for many reasons. Maybe a few
years ago it would have been better timing.

Lately, I’ve come to the realization that I can no longer bring myself to work
with PHP for personal projects. I don’t like the syntax, I don’t like where
the language is heading, I don’t like how much memory and CPU it requires to
run on a web server, and I just spend the whole time wishing I was writing
Python code instead.

This blog hasn’t run on PHP or a database for just over a year now, and with my
MantisBT install being used mainly for projects I don’t have the time or will
to work on, it just seems to be dead weight. Github can serve my needs well
enough for the few remaining projects I work on, and without needing a
complicated setup on my end. Turning off MantisBT means I no longer need
MySQL, PHPMyAdmin, APC, or even mod_php at all. The remaining apps can easily
be replaced with external services.

So basically, this is me announcing what I’ve already been practicing for many
months now: I will no longer be a developer for Mantis Bug Tracker, but I will
remain involved as a mentor for other core developers, or for those seeking
some advice on my plugins or creating their own. I won’t be maintaining any
of my plugins, but I will look at and accept pull requests until someone else
wants to step up as maintainer. I will be removing MantisBT from my site, but
will keep a database dump in case I ever need to reference it in the future.

This is not me withdrawing from open source; I have many other projects that
I’ve been working on, most of which are written in Python or C++. I find them
more enjoyable to deal with, and most importantly, they allow me to break out
of the realm of writing web applications. IRC, as old as it is, has been my
point of intrigue lately, and is at the core of my current
“pet” project.

Regardless of language, you can still find me on Github,
where all my toys are available for the public to point and laugh at. And as
always, I will answer questions on Freenode as “jreese”, or via email, although
there may sometimes be a long delay before I can reply.

Thank you to everyone who’s contributed to MantisBT or its plugins, and thank
you to everyone who helped me on the way to where I’m going.