https://www.geewax.org/Ghost 0.11Wed, 06 Mar 2019 15:32:24 GMT60For my projects, I like to have a single "boot script" that runs at start-up time. That script then looks at some metadata in order to decide "what to do next".

This has some cool side-effects, but the biggest "feature" for me is that I can publish a single .tar.

]]>https://www.geewax.org/dealing-with-meta-data-in-google-compute-engine/69cc53ad-5d6a-435f-bf5b-9439d718bcf7Sun, 31 Jan 2016 14:36:37 GMTFor my projects, I like to have a single "boot script" that runs at start-up time. That script then looks at some metadata in order to decide "what to do next".

This has some cool side-effects, but the biggest "feature" for me is that I can publish a single .tar.gz package of the application (using npm pack for example), and the boot script can rely on the instance's metadata to run any of the services needed for my app.

For those of you just looking for "how do I safely get metadata into a shell variable":

boot.sh

This should run as the "startup script" for all machines. In my case, it's also deployed to GCS every time code hits the master branch in a Git repository.

This isn't a problem because this main boot script doesn't do anything specific with the rest of the code. It only figures out the version of the package to download and runs the boot script for the service (which lives inside the package just downloaded).

In other words, the only job of this main boot script is to download the right package. The rest of the work is done by the scripts inside that package (which are version controlled).

]]>https://www.geewax.org/copyright-in-jamaica/7301907c-914e-4318-92e0-d53f9935685eFri, 03 Apr 2015 16:43:07 GMTDISCLAIMER: I’m not a lawyer. Get your legal advice from someone smarter. If you're interested in this topic, pick up a copy of Cyber Law in Jamaica.

TL;DR

The Paymaster v GKRS case should be a reminder to all of us that being explicit about ownership and confidentiality is important. This means we should be really careful in a few situations:

If you are writing software or having software written for you (or writing software for your start-up), have your attorney draft an IPIAA (intellectual property and invention assignment agreement) outlining exactly who owns what.

The longer version

Recently, the Court of Appeal ruled on a case involving Paymaster and GraceKennedy Remittance Services which had some pretty interesting implications related to confidentiality and intellectual property ownership. This is even more interesting to me because the IP in question was actually software, so it means that we in the tech industry should see this as a friendly reminder of what we should be doing already.

But before I get into specifics, I think it's worthwhile to quickly recap this 'work-for-hire' thing.

What is a 'work-for-hire'?

Put really simply: in the US, there's a law somewhere that says "if I pay you to produce some work, I own it, not you." There are all sorts of caveats on the type of work it can be, what the working arrangement is (employee versus contractor), and some common law tests that courts use when it's unclear who owns the work produced, but the underlying implication is actually pretty important:

If there's no contract specifying who owns what, ownership is subject to debate.

First,

]]>https://www.geewax.org/net-neutrality-in-jamaica/0d0bbf04-2620-45e6-b755-7c2349f0b015Tue, 01 Jul 2014 00:49:16 GMTI'm not sure if you've been keeping up on the net-neutrality debates happening in the US, but I wanted to take a moment to propose a less severe version of this for Jamaica based on an article I read about Digicel blocking certain services they deem to be competitive.

First, some background.

The net-neutrality debates in the US are focused on whether or not ISPs should be allowed to expose "internet fast lanes" for a surcharge. The debate centers around the issue of smaller companies (ie, startups) not being able to afford those faster speeds, and therefore being at a competitive disadvantage.

For example...

If Big Company had a video sharing website that was terrible but they paid the extra $100m USD per year to keep it fast, even a better video sharing website (think YouTube when it was starting) wouldn't be able to compete because it would be considered too slow compared to Big Company's site.

So where is the debate?

The debate I think we should be having in Jamaica is slightly less extreme than the one happening in the US.

I think we should consider drafting legislation similar to the Netherlands’ Telecommunications Act that says that ISPs cannot deliberately throttle traffic based on the content or purpose of that traffic (barring a few exceptions including throttling for congestion, integrity and security of the network, spam, or legal reasons).

If Digicel, LIME, or Flow decide to add a "faster lane" of traffic, we should debate the legality of charging for that later. For now, I'm saying no one should be slowed down.

Why does this even matter?

If Jamaica is serious about relying on innovation and investment in technology to level the playing field when competing on a global market, we cannot be at the whim of an Irish Telecom's top line revenue or quarterly targets.

It would be terrible to see a Digicel executive decide, in an attempt to meet revenue targets for the quarter, say "Let’s block WhatsApp so that people have to spend a bit more on text messages this month", or "Let’s charge WhatsApp a fee to allow their app to be used on our network".

Investment is already risky enough, and without any protections that Jamaican software companies won't be arbitrarily cut-off (or worse: held hostage) by an ISP, it isn't reasonable to continue investing in technology in the country.

Put simply: I won't be able to reasonably invest in tech companies targeting Jamaica.

As an investor in 10 Pound Pledge, I have to face the very real possibility of losing Caribbean customers based on an ISP not willing to deliver high-quality video content.

As an investor in EduFocal, I have to face the very real possibility that educational videos won't be delivered to customers without a hefty "transfer fee" (to be decided at the whim of the ISP).

As an investor in Blaze, I have to face the very real possibility that Digicel will build their own mobile money solution, and Blaze (being now found competitive to Digicel) will find themselves blocked from Digicel subscribers.

So what now?

I hope that Jamaican regulators and policymakers understand the implications of what they are allowing their telecom providers to do -- and the unintended and potentially disastrous consequences.

As an investor, I can certainly say that this move will make Jamaican-focused tech companies less competitive and less attractive for investment. It seems quite hypocritical for this to happen at a time when the government and the IMF are telling the Jamaican people that their main focus is on growing the economy.

Similarly, I hope that Jamaicans take a stand and tell their representatives that they won't tolerate discrimination of their data; that blocking traffic an ISP feels is "competitive" is unacceptable; that without treating all data the same, true innovation will be stifled just when we're starting to see great progress in Jamaica.

]]>In a previous article I wrote about the need to update your Datastore indexes on App Engine before deploying code that relies on them. To simplify the deployment process, I put together a script that waits for your indexes to be built before deploying new code.

#!/usr/bin/env python

]]>https://www.geewax.org/safe-deployment-on-app-engine/8e26946d-e69b-4af8-b5d4-105582cbae22Sat, 03 May 2014 16:19:00 GMTIn a previous article I wrote about the need to update your Datastore indexes on App Engine before deploying code that relies on them. To simplify the deployment process, I put together a script that waits for your indexes to be built before deploying new code.

Every so often, I get an e-mail asking me for feedback on an idea, with a generic NDA attached that I'm asked to sign before I can hear what the idea is... I won't do that. And I'll let others much more experienced than I am explain why:

(The short version of each these is basically: "I can't and won't sign an NDA to hear your pitch or see your business plan.")

I’m happy to give feedback (who knows if it’ll even be useful to you), however the points in Carol Roth's article summarize the theme very well:

Your idea already exists (or being new can be a problem)

The value is not in the idea

Sharing ideas makes them better

If you’re going to share anything generally considered “secret sauce” (like access to your code repository or the special formula for the Red Bull competitor you’re starting), asking for an NDA is a reasonable request -- you have to protect your property (however, a first e-mail typically wouldn’t be the place to discuss sharing that level of detail though).

If you're just sharing the high level idea, see it for what it is: something that hasn’t materialized yet and will likely change a lot as you work more on it.

]]>I’m a little embarrassed to say that I haven’t yet used these in my own code -- they seem so obvious now that I look back on it.

>>> d = {'a': 1, 'b': 2', 'c': 3}
>>> keys = ['a', 'c']
>>> { k:

]]>https://www.geewax.org/python-dictionary-comprehensions/c341a674-c14b-49e2-b504-668f78e488b3Thu, 21 Mar 2013 16:00:00 GMTI’m a little embarrassed to say that I haven’t yet used these in my own code -- they seem so obvious now that I look back on it.

]]>As a long-time user of GNU screen, I knew it would be difficult to switch, but I’ve made the move and am so far pretty happy. tmux does all the same things that screen does and was a lot easier to customize. I figured I would share my tmux]]>https://www.geewax.org/getting-started-with-tmux/67c4f73b-e10b-442e-b3d2-ee10798cb5bbThu, 21 Mar 2013 16:00:00 GMTAs a long-time user of GNU screen, I knew it would be difficult to switch, but I’ve made the move and am so far pretty happy. tmux does all the same things that screen does and was a lot easier to customize. I figured I would share my tmux configuration file and some of the things that made the switch relatively painless.

Keyboard bindings

I changed the default keyboard bindings to be more like screen:

Ctrl-b is the default for tmux; I prefer Ctrl-a.

Ctrl-(left/right) arrow to move between windows. No screen equivalent (that I know of).

Ctrl-a twice to go “back”.

Setting Ctrl-a, r to reload the config file made it very easy to debug.

Status bar at the bottom

I set my status bar to use the session name and my machine name on the far left, open tab names centered in the middle, and the system time and date on the far right.

Project-specific configurations (and attach tool)

There's a cool project called tat that does some neat things to make project-specific configurations really easy to set up, and attach to them later on. It also includes tab-completion for existing sessions and your project directory names.

With tat you can create a .tmux file in your project directory, which will be run with the session name as first argument. You can use this to configure your project work space easily and consistently. Here’s one of my example .tmux project files:

Tab-completion

]]>A while back I wrote some (now deprecated) code that allowed you to easily test your Python App Engine applications and their interaction with the App Engine APIs. When we got acquired by Google, I started working with some of the awesome App Engine engineers on making that code part]]>https://www.geewax.org/task-queue-support-in-app-engines-ext-testbed/d66d7810-fd1f-47bd-806a-482ed4c284d0Wed, 20 Mar 2013 16:00:00 GMTA while back I wrote some (now deprecated) code that allowed you to easily test your Python App Engine applications and their interaction with the App Engine APIs. When we got acquired by Google, I started working with some of the awesome App Engine engineers on making that code part of the official App Engine codebase.

We launched that, but one of the APIs that was noticeably lacking helper-methods was the Task Queue. I added one of the old methods (get_filtered_tasks()) but a bit later I noticed a pretty serious bug where timezones weren’t handled properly.

I wrote some extra code to fix it, but that didn’t get merged into the main repository for quite a while.

But now it’s here.

If you’re curious about the change, feel free to check out the diff (scroll to the bottom). The method is still called get_filtered_tasks() and takes several arguments as “filters” for various properties of tasks — and it should properly handle timezones when you set the eta or countdown properties when creating tasks.

Sorry for the delay. And enjoy.

PS: Since this isn’t documented on the official App Engine page, the API may change. I don’t intend to change it, but I can’t promise it won’t. Apologies in advance if you use this helper method and have it break out from under you.

]]>If you're building a web-app in Python that requires people to sign in, you are probably pretty familiar with the concept of a @login_required decorator. For those that might not be, it's pretty straight forward: the decorator wraps a handler method with a check for whether the current request]]>https://www.geewax.org/login-required-should-be-the-default/ef7554e6-bd15-45f6-a76e-84f7a35ed822Tue, 19 Mar 2013 16:00:00 GMTIf you're building a web-app in Python that requires people to sign in, you are probably pretty familiar with the concept of a @login_required decorator. For those that might not be, it's pretty straight forward: the decorator wraps a handler method with a check for whether the current request is coming from someone who has been through your sign in process.

When I first saw this however many years ago, I was pretty new to Python decorators and thought it was pretty awesome. I still think the whole concept is pretty awesome. This seems to fit the mold of a "decorator" pretty perfectly... almost as if decorators were built with this in mind...

But what if you forget your @login_required decorator? It’s so easy to do. When you're testing that "Edit your account" page, you're authenticated, you assume everything should go to plan. You could (and should) add a functional test to check that an unauthenticated request is redirected, but this is just a quick little side project...

So why don’t we just make @login_required the default? I think that it should be, so here’s some code to save somebody else some time.

It works by wrapping all local methods with your login_required method during object creation (aka, __new__). If you want a method to be public, you decorate it with the @login_not_required decorator -- which sets a flag on the method saying... that login is not required.

Here's the BaseHandler (and Meta class) which does the wrapping:

import types
import webapp2
from auth import login_required
class BaseHandlerMeta(type):
"""Meta class for all request handlers.
This automatically wraps all handler methods with the login_required
decorator. If something should be exposed publicly, it should be wrapped
with the login_not_required decorator.
"""
def __new__(cls, name, bases, local):
if name != 'BaseHandler':
for func_name, func in local.iteritems():
if isinstance(func, types.FunctionType):
local[func_name] = login_required(func)
return type.__new__(cls, name, bases, local)
class BaseHandler(webapp2.RequestHandler):
"""Base class for all RequestHandlers."""
__metaclass__ = BaseHandlerMeta
def user_is_logged_in(self):
# Do some magic here to check if someone is logged in
return False

Here are the login_required and login_not_required decorators:

def login_not_required(handler_method):
"""Allows a user to *not* be logged in.
The login_required attribute is inspected by BaseHandlerMeta and is used
as the flag for whether to wrap a method with login_required or not.
"""
handler_method.login_required = False
return handler_method
def login_required(handler_method):
"""Requires that a user be logged in."""
required = getattr(handler_method, 'login_required', True)
already_wrapped = getattr(handler_method, 'wrapped', False)
# If the method doesn't require a login, or has already been wrapped,
# just return the original.
if not required or already_wrapped:
return handler_method
def check_login(self, *args, **kwargs):
if not self.user_is_logged_in():
uri = self.uri_for('login', redirect=self.request.path)
self.redirect(uri, abort=True)
else:
return handler_method(self, *args, **kwargs)
# Let others know that this method is already wrapped to avoid wrapping
# it more than once...
check_login.wrapped = True
return check_login

Notice that if you are in a public method (@login_not_required) and you call a method that does not have that decorator, you’ll get redirected to a login page. If you want something to be public, it and all of the functions it calls (inside the handler) should be explicitly defined as public.

Ever since webapp2 came around, I've been really into the URL routing helpers, which make it much easier to keep long lists of URLs organized. I started using a new project layout that makes routing really simple -- maybe even fun.

Before I get to the interesting part I need to digress slightly. A while back, I posted a question on Stack Overflow asking whether it was better to break your app apart into separate routes in app.yaml or inside a WSGIApplication. Put simply: should you create one WSGIApplication object and route everything there? Or many separate ones, each with their own routing in app.yaml?

The short answer was: It doesn't really matter.

The longer answer was: if you're particularly sensitive about start-up time or memory, you might break things apart.

I decided to use one big app. Having a single app makes unit testing with ext.testbed a bit easier.

In addition to making testing easier, I've been using the Python 2.7 runtime which lends itself nicely to a single WSGIApplication: I put my code in main.py and then route /.* to main.app and don’t think about it anymore. But how are you supposed to keep track of all the URLs, Handlers, methods, and routes in your app as it grows larger and larger?

The method can be 'GET', 'POST', None, or any other method (or list of methods). None simply means "I don't care".

URL patterns allow fancy matching (<id:\d+>).

Rather than having many handlers (CreateUserHandler, DeleteUserHandler), I use one handler per thing and have the methods correspond to the actions (UserHandler.create, UserHandler.delete). If necessary, I can use self.request.method to distinguish between intent (ie, "show me the account creation form" versus "create my account and rediect me somewhere").

The handler method name (AccountHandler.list) and the action part of the route name (account.list) are identical. This makes linking easy -- you know the route's name because it lines up with your code.

I break from PEP8 about lining data up vertically. This is data you'd put in a table. I don't like jagged tables.

Now to tie this all together, I just put some configuration for webapp2 into config.py:

Extra bonus: If you delete a URL and the template tries to look it up, you’ll get an exception — and so will your unit tests. I guess this could be a scary thing if you don’t have tests, but hopefully it is helpful.