Buttons in a Silverlight DataGrid Header

Today I was hoping to complete adding an excel-like auto-filter to the Silverlight DataGrid which is used for the main view throughout a big chunk of our application. Instead I spent almost the entire day just getting the first step working - putting a button in the grid header and wiring up the click event.

Generally using Silverlight from IronPython is a pleasant experience, but there are one or two things that can't be done through code and have to be done in XAML. XAML is the markup language used to describe Silverlight user interfaces; the Silverlight UI is a cutdown mostly-subset of Windows Presentation Foundation (WPF), the hardware accelerated Windows [1] user interface framework that is part of .NET 3 and above.

With WPF you can not only define the visual elements (including animations) of a UI, but also setup data-binding and bind handlers to events. Event handlers are statically bound and this is a problem for IronPython. Even though we can now use clrtype to create true .NET classes from IronPython, the XAML loader uses Type.GetType() to find the types - and this fails with generated classes.

For almost everything this isn't a problem as we can just hook up events from code, but creating custom DataGrid headers and hooking up events to controls in them is one of the places where we can't.

So the first part of the puzzle is creating a custom style in UserControl.Resources and setup the ContentTemplate:

See how the DataTemplate contains a Grid with sub-ui-elements like a TextBlock and the oh-so-important Button. The {Binding} in the textblock means that the standard header text is displayed alongside our button. The XAML for the DataGrid specifies the style we created:

Ok, so far so good. When the columns are created for our grid they will use the ContentTemplate to create the column headers:

If we'd been using C# we could have hooked up the button click event in the style XAML. So how do we do this from IronPython? This is what took most of the day to work out. The column headers aren't exposed in any straightforward way, but once the columns have been created we can 'walk the visual tree' to find the column header object, and from there we can find the button. Walking the visual tree is done with VisualTreeHelper, and a recursive helper function:

fromSystem.WindowsimportVisibilityfromSystem.Windows.ControlsimportButton,DataGridTextColumnforentryinfind_children(datagrid,DataGridColumnHeader):forbuttoninfind_children(entry,Button):ifnotbutton.IsEnabled:button.Visibility=Visibility.Collapsedelse:button.Click+=handler_function# just one button per columnbreak

We make disabled button invisible because otherwise the grid will leave a disabled button visible in the header to the right of the populated columns.

So that was the hard part. The next problem wasn't quite so difficult; inside the event handler how do we know which column the button click is for? It turns out that getting a reference to the column inside the click event handler is easy:

The final version will use an image for the button content, but the hard part is done.

UPDATE

My colleague Stephan Mitt tells me that the only reason it was so easy to get the column from the button event is that he spent three hours previously discovering this API. Props to him for working this out and making my life easier. Stephan also wanted me to show you the end result with his nice filter images instead of my proof-of-concept 'Foo' buttons:

The code to set the image on the button looks like this:

fromSystemimportUri,UriKindfromSystem.Windows.ControlsimportImagefromSystem.Windows.Media.ImagingimportBitmapImagedefclick_handler(sender,event):uri=Uri('images/someimage.jpg',UriKind.RelativeOrAbsolute)bitmap=BitmapImage(uri)image=Image()image.Source=bitmap# The sender is the buttonsender.Content=image

As I was integrating this with our production code I encountered another problem. Many of the grids we used are loaded into TabControl pages, and the headers aren't created until the grid is rendered (the tab page the grid is in is selected). This means it matters when you run the code that walks the visual tree to find the buttons and setup the click handlers. A good place to do it is in response to the Loaded event, which is fired after the headers have been created.

As I explained we're writing a Silverlight application that communicates with our Django server and exchanges a lot of json with it.

Unfortunately, due to what is apparently just an oversight, the codecs module is incomplete for IronPython on Silverlight. This means that recent versions of simplejson don't work.

What we've been using is an older version of simplejson, that pulls in the obsolete and huge sre module in as one of its dependencies. On top of bloating up our application and adding several seconds to the startup importing it all the performance is not exactly blazing.

Fortunately part of the Silverlight SDK is System.Json.dll. Using this API from IronPython is pretty simple. The following code defines a loads function (load string - the same API as simplejson) that takes a string and parses it.

importclrclr.AddReference('System.Json')fromSystemimportBooleanfromSystem.JsonimportJsonValue,JsonTypeString=clr.GetClrType(str)defloads(inString):thing=JsonValue.Parse(inString)returnhandle_value(thing)defhandle_value(inValue):ifinValueisNone:returnNoneelifinValue.JsonType==JsonType.String:returnclr.Convert(inValue,String)elifinValue.JsonType==JsonType.Boolean:returnBoolean.Parse(str(inValue))elifinValue.JsonType==JsonType.Number:returnget_number(inValue.ToString())elifinValue.JsonType==JsonType.Object:returndict((pair.Key,handle_value(pair.Value))forpairininValue)elifinValue.JsonType==JsonType.Array:return[handle_value(value)forvalueininValue]# Should be unreachable - but if it happens I want to know about it!raiseTypeError(inValue)defget_number(inString):try:returnint(inString)exceptValueError:returnfloat(inString)

If you want it to handle decmials or dates you'll have to add that yourself. How it works is mostly straightforward, but there is one little piece of 'magic' in there. When you call ToString() (or str() they do the same thing) you get the original json back. For numbers and booleans this is fine as we can easily turn them into the objects they represent. For strings this is a nuisance as we get double quoted, escaped strings. The correct way to get the value we want is to use implicit conversion. In C# this looks something like:

stringvalue=jsonArray["key"];

Declaring the result as a string calls the appropriate JsonValue implicit conversion operator for us. IronPython doesn't have implicit conversion operators as we don't declare types and with a dynamic type system you rarely need to cast. Up until version 2.6 it wasn't possible to call implicit operators, but in IronPython 2.6 we gained a new function in the clr module; clr.Convert:

>>> print clr.Convert.__doc__
object Convert(object o, Type toType)
Attempts to convert the provided
object to the specified type. Conversions that
will be attempted include standard
Python conversions as well as .NET implicit
and explicit conversions.
If the conversion cannot be performed a
TypeError will be raised.

I haven't yet written the code to do json encoding, but the JsonValue class (and friends) can be used for encoding as well as decoding, so I expect the code will pretty much be the opposite of the snippet shown above...

Resolver One 1.7, BioPython and 3D Graphics

Although I've left Resolver Systems I still follow closely what they're up to. Resolver One, the IronPython powered spreadsheet with the programming model right at its heart, is an innovative and unique product [1] that deserves to flourish. Despite the horrific loss Resolver One seems to still be moving forward at great pace without me.

Major features in this new release, the last one in which I have been involved in the development [2], include:

Button-click handlers are now executed without blocking the rest of Resolver One. This means that if you accidentally write a handler that never finishes, or just one that takes longer than you want, you can cancel it while it's running and fix the problem.

Added long-awaited dialog for setting wrapping and aligment for cells.

External imported modules now shared between documents and RunWorkbook (better performance for RunWorkbook in particular).

Faster install time.

Improved responsiveness when dragging tabs and switching between tabs.

[Shift] while clicking Recalc toolbar button now now forces reload of all imported modules (like Shift-F9 has always done).

A few minor bugfixes.

Giles Thomas and team have also produced screencasts of a couple of particularly interesting uses of Resolver One:

This one uses Yahoo! Finance to download the close prices over the last two years for every stock that's currently in the Dow Jones index, then charts them in a 3D window which you can pan and zoom using the mouse. Here's a video showing it in action...

If you're interested in OpenGL Giles has a blog exploring WebGL, OpenGL in the browser: Learning WebGL.

Resolver One includes built-in support for Python C extensions like Numpy through the Resolver Systems sponsored open-source project Ironclad. Ironclad overcomes IronPython's inability to use compiled C-extensions. This video demonstrates the use of one such powerful library, BioPython, from within a Resolver One spreadsheet, to compare the molecular shapes of proteins. (3m12s)

Possibly not true. I've done some work on the port to IronPython 2.6, which has mainly been spearheaded by Glenn Jones and will hopefully be 1.8. That's assuming that the IronPython team get 2.6 final released soon.

IronPython Training in New York, January 21st 2010

Michael Foord is an early adopter of IronPython, having used it for (among other things) the creation of the Resolver One Python-driven spreadsheet product. He is the author of Manning's IronPython in Action and a well-known speaker at Python conferences throughout the world.

IronPython is the implementation of Python for the .NET framework and Mono. IronPython combines the power of the .NET framework with the expressiveness and flexibility of the Python programming language. In this workshop you'll get hands on experience with using IronPython: from the basics of integrating with the framework assemblies and classes to embedding IronPython in .NET applications.

This workshop is aimed at .NET developers with an interest in IronPython, and Python developers interested in IronPython. We'll cover how IronPython integrates with the .NET framework, including some of the tricky details that previous experience of Python or C# alone won't have prepared you for. We'll also be covering the IronPython hosting APIs and how to embed IronPython.

Reasons you might want to consider IronPython include:

Using Python libraries from .NET

Using .NET libraries from Python

Writing multi-threaded Python code without a GIL

Embedding IronPython in .NET applications for user scripting

Exploring new assemblies and classes with the interactive interpreter

System administration and scripting

Developing for Silverlight (Python in the browser)

Previous programming experience of Python is assumed, but no specific experience with Python or .NET is required. Places are limited, so reserve yours straight away.

Course Outline

The Interactive Interpreter

Getting started with IronPython, basic .NET integration, executing IronPython scripts. Using the interactive interpreter as a tool to explore live assemblies and classes.

IronPython for Python Developers

Why should a Python developer be interested in IronPython? From multithreading without a GIL, to multiple and sandboxed Python engines, to native user interface libraries (plural!) on Windows or even cross platform development with Mono; we'll be looking at some of what the .NET framework and IronPython has to offer Python developers.

Python for .NET Developers

A whirlwind guide to Python syntax and the wonders of dynamic languages for .NET programmers.

Tools and IDEs

A rundown of the options for developing with IronPython and the different IDEs available, including their different capabilities.
Application Development with IronPython

Using Python and .NET libraries to build applications with IronPython. From simple scripting system administration tasks, to web applications to desktop applications. Using Ironclad for compatibility with Python C extensions, using the Python standard library and working with the .NET framework.

Advanced .NET Integration

Using a dynamic language on a statically typed framework. Many of the features of the .NET framework require specialised knowledge to use them from IronPython; including making use of new features in IronPython 2.6 like the __clrtype__ metaclass.
Embedding IronPython and the IronPython Compliler

Using the IronPython hosting API for embedding in .NET applications. This can be used for creating hybrid applications with parts in C# and user scripting. With Pyc the compiler tool it can also be used for deploying IronPython applications as binary form. We'll also see how .NET 4 makes interacting with IronPython and the Dynamic Language Runtime simpler from C#.
Python in the Browser with Silverlight

The basics of creating a Silverlight application with IronPython and the development process. Plus an overview of the APIs available in Silverlight; for building user interfaces, making web requests, and interacting with Javascript and the browser DOM.

New Job with Django and IronPython

Big personal news; I've changed jobs. After more than three years working with Resolver Systems I felt it was time to broaden my development experience. I greatly enjoyed working with Resolver Systems and learned an enormous amount; I'm sorry to leave them but I'm sure they'll manage to cope without me.

I'm now freelance, starting with a contract with a web development firm based in Germany called Comsulting. I first came into contact with Comsulting earlier this year through their lead developer Sven Passig. One of their big customers wanted a web application with the front-end written in IronPython and Silverlight. I did some consulting for them on this as it was their first Silverlight application.

You can hear a bit about what they've been up to with IronPython and Silverlight on the Python 411 podcast that Sven recorded with Ron Jeffries:

I've just been onsite in Germany with them for two weeks but will mainly be working from home. Comsulting's biggest customers are within the German and Swiss media industry and they have several applications for tracking and organising advertising in magazines and websites. It turns out that these media companies really like Silverlight...

We're developing web applications for these companies and after working on the tail end of one project for the first week I started a new project with Stepan Mitt, a designer / developer who also contracts for Comsulting and who happens to be a really cool guy. We're using Django on the server (with CPython 2.5 on Linux) and Silverlight on the client side. I'm converting the Comsulting guys to Test Driven Development, but we still need to investigate the best way to functionally test Silverlight applications.

It's great to be working with CPython again, and especially with Django, but also great to be able to use my IronPython experience. Our Silverlight application communicates with Django via json, so we're using the Django ORM and authentication but our views generate json. I'm sure there will be a blog entry or two out of this.

IronPython in Action Second Quarter Sales

The author of Beginning Ruby wanted to make the ebook free. Apress refused and the author, noting that he still owned the copyright (whilst Apress have sole right to publish) says he would have no problem with his readers distributing an 'unauthorised' free ebook version.

Interestingly the author says that his ebook sales are minimal; compare that with my figures below.

Our very own Mark Pilgrim published Dive into Python with Apress but with the content under the GNU Free Documentation License. Someone took him at his word and got an alternative version listed on Amazon.com, much to the consternation of Apress. Mark takes the opportunity to put his money where his license is.

The first quarter started in January (strange that) but IronPython in Action only went on sale in March, meaning that the first quarter only included one month's sales. That month included all the preorders from places like Amazon, plus all the people who paid for the book during the Early Access Program, so they're surprisingly high.

Thankfully the sales were enough to pay off the advance entirely and even yield an extra $2700 odd dollars, of which almost $2000 was held in reserve in case of returns. Sales in the first quarter were 2600 print books and 651 ebook sales.

Total figures for the second quarter (4 months sales):

First Q.

Amount

Second Q.

Amount

Domestic gross sales

1520

~$35k

1662

~$37k

Domestic returns

(3)

(67)

International sales

818

~$13k

20

~$250 (!!)

Print book web sales

329

~$16k

96

~$4k

Ebook sales

651

~$18k

163

~$4.5k

Subsidiary rights sales

$2,600

0

Total

~$70k

~$41k

Impressively, domestic sales this quarter were higher than the first quarter. As the first quarter web sales included all the Early Access sales it isn't surprising that they're lower this quarter; international sales are a bit disappointing though.

Another interesting fact, out of total sales of 4305 print books plus 814 ebooks. That's almost 20%. As we're giving away ebooks with the print book this isn't duplicate sales and and is worth around $2000 in commission. With the $4200 commission cheque for this quarter I think I'm now close to having made minimum wage for the two years work that went into creating IronPython in Action.

Testable file Type and File I/O in Try Python

A few weeks ago I announced Try Python, a Python tutorial with interactive interpreter that runs in the browser using IronPython and Silverlight.

Because it runs in Silverlight, which is sandboxed from the local system, the parts of the tutorial that show how to do file I/O didn't work. I've finally fixed this with a pretty-much-complete implementation of the file type and open function that use local browser storage (IsolatedStorage) to store the files.

This sets the IsolatedStorage based backed in the storage module and then replaces the builtin file and open with the versions in storage. There is a corresponding restore_builtins function to put the original ones back. You could use this for convenient file usage inside Silverlight applications, but the code is neither efficient nor elegant so I wouldn't recommend it for production!

You can also call storage.file and storage.open directly without having to replace the builtin versions. If you use the browser backend in Silverlight then files will be persisted in browser storage and be available when an application is revisited later (unless the user clears the browser cache).

Note

The tests require Python 2.5 or more recent as they use the with statement. The actual implementation should be compatible with Python
2.4 or even earlier. (In fact because they use the storage_backend the tests will only run on Silverlight but it would be very easy to get them to run on CPython.)

You can see the tests run in the browser (slowly - as I run them in a background thread) at: storage module tests.

The storage module does have a default dictionary based backend for when used outside Silverlight. This simply stores files as strings in a dictionary using the full file path as the key (it has no concept of dictionaries). You could implement an alternative backend by implementing the four functions in the storage_backend module (or the method on the backend class in the storage module). This leads to an interesting potential use case.

Unit testing code that does file I/O is notoriously difficult. You can either let your unit tests do real file access or modify your production code to use something like dependency injection so that you can mock out the file access during the tests. Using this implementation you can swap out the builtin file type during the test, controlling how it behaves using the dictionary backend, without having to change the way you write your production code just to make it testable.

For this to be really useful it needs an implementation of functions in the os and os.path module (like os.listdir, os.remove, os.makedir(s), os.path.isfile, os.path.isdir and so on). This should be easy to do and I will get round to it as they would be nice things to cover in the Try Python tutorial.

There are several (mostly minor) differences between this and the standard file type, plus a few things still on the TODO list:

Differences from standard file type:

Attempting to set the read-only attributes (like mode, name etc) raises an AttributeError
rather than a TypeError

The exception messages are not all identical (some are better!)

Strict about modes. Unrecognised modes always raise an exception

The deprecated readinto is not implemented

Still todo:

The buffering argument to the constructor is not implemented

The IOError exceptions raised don't have an associated errno

encoding, errors and newlines do nothing

Behavior of tell() and seek() for text mode files may be incorrect (it
should treat '\n' as '\r\n' in text mode)

Behaves like Windows, writes '\n' as '\r\n' unless in binary mode. A
flag to control this?

Universal modes not supported

Missing __format__ method needed when we move to 2.6

Implementations of os and os.path that work with storage_backend

As an added bonus for Try Python the IronPython team have created a new .NET interop tutorial for IronPython and it is in written in ReStructured Text. It will be very easy for me to add this to Try Python as well as the Python tutorial; I'll wait a bit until it has stabilised though.

I've put up three new articles in the IronPython section of this website. The first two are aimed at developers new to IronPython, particularly .NET developers. The third article should be useful to anyone using or interested in IronPython:

Just getting started with IronPython or wondering why you should be interested? This article introduces IronPython, explains how it came into existence and some of its more common use cases. The article also shows the basic IronPython and .NET integration with the interactive interpreter. The IronPython interactive interpreter is a great way to learn plus a useful tool for working with the .NET framework. The article even shows how the Dynamic Language Runtime allows languages to interoperate by using a Ruby library (through IronRuby) from IronPython.

An introduction to the Python programming language for programmers interested in IronPython. It covers the basic syntax and semantics and touches on how dynamic languages are better (oops - I mean different) to statically typed languages.

The IronPython team have done a very good job of integrating Python and the .NET framework without having to change Python or introduce new syntax. Despite this there are various areas where you need to know some IronPython specific information; previous experience of Python or C# alone will not be enough.

This article contains a lot of the nitty gritty details of how to use Python the language to integrate with the underlying .NET framework.

A fourth article of mine has also gone online, although without any action - or even knowledge - on my part. An extract from chapter 8 of IronPython in Action has been posted to Developer Zone. This is the section on metaprogramming, introducing and explaining metaclasses in Python:

Resolver One 1.6.5 Released

It isn't long since we released version 1.6 of Resolver One, but we're trying to increase the frequency of our releases and get features into the hands of our users faster.

Resolver One is a highly programmable spreadsheet for Windows. Resolver One is written in IronPython and is fully programmable in IronPython. Through the integration of Ironclad you can use Python C extensions like Numpy in the spreadsheet grid and user code.

We've pushed out a new release of Resolver One, and as it is a medium-sized release we've called it version 1.6.5.

We already have more features completed and on the way, so hopefully we'll be able to do a 1.7 release soon...

In his email to the IronPython mailing list announcing the releas Resolver Systems boss Giles Thomas included the following information about Resolver One:

We are proud to announce the release of Resolver One, version 1.6.5. Resolver One is a Windows-based spreadsheet that integrates IronPython deeply into its recalculation loop, making the models you build more reliable and more maintainable.

It's also still (we think) the largest IronPython application in the world, with 56,000 lines of code backed up by 176,000 lines of unit and functional tests.

For versions 1.6 and 1.6.5, we've made it easier for people to share their spreadsheets. A new free "player" version means you can pass your work on to other people, and they can use it without having to buy anything, while a new Resolverlib makes calling your spreadsheets from IronPython programs as easy as calling a function.

Try Python: Python Tutorial and Interpreter in the Browser

Ever since Silverlight, with the capability of running Python code in the browser, I've wanted to create something to help people learn Python.

Try Python is the Python tutorial, with an interactive Python interpreter, running in the browser. It runs on Safari, Firefox and IE on Windows and the Mac and needs Silverlight 2 or 3 installed. At the moment it doesn't work with Moonlight 2 due to a bug in the beta, but hopefully it will work with the final release meaning you will be able to use Try Python with Linux and Firefox.

Not much of the Python standard library is included. I intend to expand the tutorial adding new modules as they are needed (the whole standard library is about 5mb and would make Try Python take much longer to load).

There are various possible ways forward (there are some ideas on the issues list). I'm about one quarter of the way through an implementation of file and open that uses local browser storage - so users can follow the part of the tutorial that does file I/O. A simple text editor (and import hook) that lets you create and import modules and packages in local browser storage would also be good.

I think what I'd like to do is something more interactive though - that presents short snippets of code with explanation and lets you work through them one at a time. This would be more interactive than the page by page style at the moment. Ideas and contributions welcomed of course...

IronPython Tools and IDEs

A frequent question on the IronPython mailing list is "what IDE should I use with IronPython?". For many .NET developers the question is phrased slightly differently, "how do I use IronPython in Visual Studio?". There are now several different major IDEs with IronPython support, including a few different ways of using IronPython with Visual Studio.

I've written a roundup of the major editors and how they support IronPython. This includes a look at the standard features you would expect in a Python editor, like autocomplete, calltips, debugging and more - with honourable mentions for other Python editors like Vim, Emacs, Komodo, Davy's IronPython Editor and the DLR editor that comes with the Pyjamas project. The article also has a roundup of standard tools for Python development; the code quality tools (PyLint, PyChecker and PyFlakes), profilers, debuggers, coverage, refactoring and so on.