Tag Archives: python

I’m working on a project where the client wants a cool sliding navigation effect. We’re implementing this with JavaScript/AJAX/DHTML.

One of the constraints is that pages can only be reached via certain other pages. For example, if you’re on the /portland/contact page and want to go to the /seattle/contact page, you’ll first slide up to /portland, then over to /seattle, then finally down to /seattle/contact.

After a while, it occurred to me that there were some similarities with another project I’ve been working on off and on for the last few years, byCycle.org, which is a bicycle trip planner a la Google Maps.

I had written a Python version of Dijkstra’s Single-Source Shortest-Paths (SSSP) for byCycle.org. That’s available on Bitbucket as Dijkstar (so named because it also does has the potential to do A*). I figured it wouldn’t be too hard to port the Python version to JavaScript, and it wasn’t.

There were a few snags, though. Most of it was just syntactic and semantic differences between the two languages. The biggest issue was that I use “heapq“ in the Python version to maintain the costs to previously visited nodes in sorted order. JavaScript has no priority queue implementation that I could find, so I came up with a different solution that involves updating an Object (AKA hash) with costs to newly visited nodes and sorting the keys to pick the next node to visit. I’m assuming/hoping the underlying sort implementation is highly optimized.

Interestingly, I think I found at least one bug in the Python version, although I’ve been using that version for a couple years now with no known problems, so it must only be applicable in certain edge (no pun intended) cases (or maybe it’s due to some difference in the languages–need to take a closer look). I think the JS version came out cleaner, too.

If anyone’s interested, I’m releasing this under an MIT license. You can get it from here. Note that it depends on the util module that you can get from here. The util module contains some other Python-inspired JavaScript, in particular a couple of functions for generating namespaces and classes. I might write another post about that at some point.

Update: It wasn’t long before the project got too complex on the back end (SOAP blech) for my limited Ruby knowledge. I switched it back to Python/Pylons and never looked back. The Pylons => Rails migration was straightforward. I guess I could have pushed through with Ruby/Rails, but with deadlines looming, it made more sense for me to go with what I knew best. Being familiar Python and its ecosystem was far more pertinent than the deficiency of any particular library. There’s probably another blog post or two in here…

I’ve been working on a fairly big Web site project lately. My partner and I initially decided to use Django to build the site, mainly because I’m a Python “expert” and Django is (apparently) the #1 Python Web framework. We were also lured by the easy admin interface.

After trying to use Django and not really enjoying it, I tried switching to Pylons because I’ve had a good amount of experience with it in the building of byCycle.org. It’s gone through two fairly major releases since then, and so have a bunch of the libraries that tend to get used with it, like SQLAlchemy, Elixir, etc.

I was having a hard time with the Pylons docs, and so I ended screwing around with Grok (which actually looks fairly interesting) and even took a look at the Zope 3 site. I’m sure Zope is really awesome or whatever, but it might as well suck. Every time I look at that site, I’m just like “WTF! This shit has been around for like five years!” Anyway, I might just not be smart enough for Zope.

This led us back toward Rails (even if it is a ghetto). I used Rails a bit last year but never did anything too serious with it. Diving into it today was quite a pleasure. There are issues to be sure, but overall I’m enjoying it by far over any of the other options we had tried. I’m also enjoying learning/relearning Ruby.

If Pylons had good docs, we’d probably be using that.

So, I don’t know if this is a particularly useful post, since I didn’t get into much in the way of reasons (what, i have back this up?!). This subject’s been hashed and rehashed, but I just wanted (needed) to make a qualitative statement about my/our experience, which, of course, is purely personal.

Yesterday, my copy of Programming Erlang arrived in the mail1. w00t! Iâ€™m already part way through chapter three. I donâ€™t know what it is about this language â€“ maybe itâ€™s all hype and a passing fad â€“ but I havenâ€™t been this interested in learning a new language since I started with Python over two and half years ago.

The day before yesterday, I took a shot at implementing a linked list in Erlang. I had one basic rule, which was that I wasnâ€™t allowed to use the built in list type. Getting started was fairly difficult, but once I started to â€œget itâ€ (e.g., pattern matching, recursion), the task got much easier.

Previously, I had only played around in the shell, so this is the first Erlang module Iâ€™ve written. Erlang modules are similar to Python modules, though to make functions available outside an Erlang module, they must be explicitly exported. I havenâ€™t yet come across whether Erlang modules can be organized into packages, although I imagine there must be some kind of higher level system for organizing Erlang programs.

Recently, I did a similar exercise with Python as way to experiment with Behavior Driven Development. That version uses the familiar â€œdestructive assignmentâ€ operation throughout. Erlang allows single assignment only, so I had to think about the problem in a different way. For example, an item canâ€™t be appended to a list by manipulating a couple of object references as in Pythonâ€”instead I used recursion to build up a new list.

After I get further into the book, it will be fun to come back to this implementation and see how it can be improved given a better understanding of the language. Iâ€™m also looking forward to exploring Concurrency Oriented Programming in depth. With Python, I tend to not think in terms of concurrency, though Iâ€™m sure Iâ€™ve got code that could be improved by using it.

Finally, hereâ€™s the code. It was written in Emacs, which has a nice Erlang mode. There are two other IDEs available, one based on Eclipse and the other on NetBeans.

I’ve been hearing about Erlang for several months now. It has seemed interesting, but up until recently I hadn’t felt any great need to look into it.

Last week, however, my interest was piqued, and I spent much of the weekend reading about Erlang and playing around in its interactive shell. That continued into this week when I decided to buy the new book.

While I’ve been sitting around waiting for the book to show up, I’ve been wondering if there’s an Erlang for Python Programmers tutorial floating around anywhere (I haven’t come across one yet). I’m thinking I might try to put something like that together. I think that will be an interesting challenge as there’s not necessarily an obvious one-to-one correspondence between the two languages.

It’s a functional language, which makes it more interesting in some ways than learning, say, Rubyâ€”in fact, I wouldn’t really need to learn much to start using Ruby, whereas using Erlang will require new ways of thinking about program structure (in terms of functions and processes)

I was testing my algorithm/data structures chops by implementing a Linked List from scratch using no reference material (not very difficult, I know), and I must say that using BDD was incredibly helpful. Iâ€™m calling it â€œBDDâ€ instead of â€œTDDâ€ just because the test class names describe a context and the method names in the test classes are behavior-oriented, inspired by the style found on the RSpec site.

I didnâ€™t write the tests/specs up front like youâ€™re â€œsupposedâ€ to. Instead, I thought about what I was trying to do, came up with an initial idea for the implementation, started writing code, and then added a test to make sure I was on the right track. This worked out pretty well. I also used the tests during refactoring. Writing the tests first might have worked out better, but I canâ€™t say for sure.

Now, for some real fun, implement a linked list based on only the test code below.

Update 6/30/07: Fixed links to glineenc.py so that it’s actually accessible.Update 7/15/08: Fixed links to glineenc.py again because of change to HTTPS on Trac site.

Here’s some Python code (complete with unit and doc tests) for converting a series of latitude/longitude points (i.e., a polyline) to the Base64 encoding that Google Maps understands. It’s particularly useful for long and/or complicated lines.

It’s based on the algorithm listed here and the JavaScript code here [page disappeared].

This site gives some more insight into it and has a pretty cool example of a fractal line here.

Here’s the code:

Previously, I had pasted the Python code right into this article, but I recently made a bunch of revisions and it was way too long. Here’s a link to the GitHub project instead:

Please note that this code is still in somewhat of a rough state. I have plans to polish and package it up, but for now, I’m using it as is and it’s working quite well (you’ll have to be patient to click that link as it takes ~20-30 seconds to generate the route, even though the line drawing itself is almost instantaneous).

JavaScript that uses results from `glineenc` looks something like this (assuming you’ve returned some JSON, say, with `encoded_points` and `encoded_lines` keys):

Restler is a base controller for Pylons projects that provides a set of default RESTful(ish) actions that can be overridden as needed. It also handles database connectivity as long as a few simple rules are followed.

It adds a bit of convention-over-configuration to Pylons and takes some inspiration from Rails’ scaffold_resource generator.

Restler’s aim is twofold: 1) make it easier to get started with Pylons and 2) encourage a RESTful project architecture.

It attempts to remove some of the pain of database setup in a Pylons project by providing a default configuration. All users need to do is specify their connection settings with one line in a config file.

PS If anyone happened to come across Restler on PyPI previously, some of the wonkiness has been removed (e.g., use of execfile to include restler) and most of the actions have been filled out such that they actually do (something closer to) the right thing.

PPS Sorry about the comment spam earlier on Planet Python. I’m not really sure why comments are coming through, and I don’t see any settings in Mephisto that would allow me to change that. I disabled comments for the offending post and emailed the Planet Python moderator to see if there’s anything that can be changed on that end.

I’ve been watching ToscaWidgets (TW) for a while now and keep thinking that widgets could be very useful. I’ve been perusing the TW site and the new TW mailing list, and there doesn’t seem to be much documentation on how to create new widgets.

Today, I got the idea to create a Google Maps widget. It started with someone asking in the #pylons IRC channel about getting access to a domain-specific Google Maps API key from the configuration settings during a request.

In the byCycle trip planner, our API keys are buried in a JavaScript file, but the config file would be a much better place for them. But then there’s the question of how to stuff the right API key into the JavaScript at the right point. Widgets seemed like they might be the answer, and since a map widget is something that could be useful in the trip planner, I decided to dive in.

I also created a simple Pylons project to test the widget. This entry on the new Python Web Documentation Project site was very helpful for showing how to integrate ToscaWidgets into a Pylons application (I assume there’ll soon be a magic incantation you can use instead of copying a bunch of stuff into your middleware settings).

What I’m going to focus on in this post is creating a new widget and packaging it up for use by others, using a recipe-style approach with commentary at each step. In a future post, I’ll show how to use the twMaps Google Maps widget in a Pylons project. For now, you can browse a sample Pylons app here.

Install ToscaWidgets

easy_install -U ToscaWidgets

Create a Package Layout

ToscaWidgets includes a paster template for creating a package layout:

paster create --template=toscawidgets

This will prompt for a bunch of info, then create a directory structure similar to this:

This layout allows for multiple related widgets in a package. You could put multiple widgets in widgets.py and import them all into ‘maps.init.py’, but I prefer to put each widget into a separate module, then import each one into ‘maps.init.py’.

There are only a few files you need to create for each widget. It’s possible to write your HTML template, CSS, and Javascript inline in the widget module, but usually that’s a bad idea. In my case, the HTML template was so simple, that I wrote it inline anyway (which is why the templates directory is empty).

Create Your Widget

Conceptually, creating a widget is quite simple. You create your static resources (stylesheets and javascripts) and template and tie everything together in a widget class. In your Web app, you just import your widget, pass it some parameters, and tell it to display itself. (You have to add a few lines to your templates alsoâ€”see below.)

There’s nothing particularly special about this; it’s just JavaScript.

(Aside: The only thing that may be interesting is the style, if you haven’t seen it before. The anonymous function that everything is wrapped in creates a private namespace. We can declare functions and vars in there and they’re not accessible from the outside. We return an anonymous literal object that exposes twGMap’s public interface. The private vars are accessible within this object.)

The stylesheet

/* gmap.css */
#twgmap {
width: 400px;
height: 400px;
}

Again, nothing interesting; just default dimensions to make sure the map will show up. A user could include a different stylesheet that overrides this.

We could probably also parameterize the dimension settings, send them to the “create_map“ JavaScript function, and set the map container dimensions using DOM methods (a good idea actually, now that I think of it). It would also be cool if it was possible to create CSS templates that can be filled in similarly to HTML templates. (It’s likely that this already possible and I just don’t know how to do it.)

The HTML template

Normally, you’d probably write your HTML template in a separate file under the templates directory. As you can see in the “GMap“ class below, though, the template for the the Google Maps widget is very simple, so I just wrote it inline.

To point to a template in a separate file, use Buffet syntax, which looks something like this:

template = 'template-package:path.to.template'

The widget class

Here’s where things start to get interesting. The HTML template, JS and CSS above are just standard stuff, and we could use them by copying them into our project and using script and link tags to include the JS and CSS in the template.

In other words, we don’t need to create a widget, but there are some compelling reasons for why we might want to.

For one thing, we normally have separate directories for different types of files (css, js, etc). We don’t have to do thisâ€”we could create a widgets directory with a layout similar to the one above. This would get us part way there in that it would at least be a little easier to detach a particular widget from a project and use it somewhere else.

Using Widgets, though, allows complete separation; the files don’t need to be copied into your project at all. You just import the widget and it sets things up so your framework knows how to access the widget’s resources. This makes it really easy for you to reuse your widgets and share them. In turn, it makes it easy for you to use other widgets (the twForms package has a bunch, for example).

Another thing that’s nice with Widgets is that we don’t have to manually include widget templates or script and link tags ourselves. The widget system knows how to do that for us, so that all that’s required to display a widget in a page is telling it to display itself in a particular place in the page.

So, there are some reasons to use widgets; now, let’s look at the code.

“CSSLink“, in essence, creates a stylesheet link tag. The link is relative to the “modname“ package so that if “modname“ is twmaps.widgets.maps.gmap and filename is static/my_widget/my_widget.css, the full path to the CSS file will be /twmaps.widgets.maps.gmap/static/my_widget/my_widget.css. “JSLink“ does the same thing.

Basically, CSSLink and JSLink are just fancy ways to create CSS & JS tags that can be associated with a widget in Python. Using “twgmap_css“ from the example above, twgmap_css.display() outputs `<link rel=”stylesheet” type=”text/css” href=”/twmaps.widgets.maps.gmap/static/gmap/gmap.css” media=”screen” />`.

Inside the “GMap“ class:

“params“ are the params that can be set when creating a widget; they get added to the “params“ list of any base classes. We can set a default value for a param, as I did with both “css_class“ and “map_opts“. (I think params can be set to be required too.)

The first argument passed to the constructor becomes the “id“ param. (One issue I had was that I was unable to set a default “id“ and then override it.)

“template“ is the HTML template for the widget. It can either be specified directly as a string or as pointer to an external template file. The names in “params“ are available in the template (e.g., ${css_class}).

“css“ is a list of CSS resources; note that it must be a list, even if it just contains one item

“javascript“ is just like “css“ (at least in basic usage)

“include_dynamic_js_calls“ tells the widget to include JavaScript function calls during page load; you specify these functions using “self.add_call“. The argument to “add_call“ is JavaScript code in the form of a string. Those JS calls are added to the bottom of the HTML body.

In the code above, I used “js_function“ to generate a Python function that when called, creates a string of JS code. The arguments to that function are converted to JSON, which become the args to the JS function call.

So there you have it: set up some CSS and JS links, point to your HTML template, define parameters for the widget that you can pass through to your template and/or JS, add some callbacks to initialize your JS, and that’s about it.

Test

Tests would be a good thing to add, yes. Using `paster create` to generate the package layout creates a tests package to use as a starting point.

Package It Up

Packaging is a matter of writing a setup.py file that contains various metadata (name, license, and so forth) and then running a build command. Using the `paster create` command generates a setup.pyâ€”it prompts for the metadata and fills in setup.py with that metadata and the necessary commands. It may be necessary to tweak setup.py, but then again, it may not.

Once setup.py is in order, we can run python setup.py bdist_egg. The resulting egg can be found in the dist directory. You could post this somewhere and people could download it and install it using python setup.py install. It’s much cooler though to put the package up on PyPI (AKA cheeseshop). Here’s one way to do that: