Primarily bug fixes on top of Beta 1. Also maybe some top small features as requested by users.

IronPython 2.6 RC 1 - ~1.5 months later

Build is truly a candidate for release. If no showstoppers are reported this build will be shipped as RTM

IronPython 2.6 Final ~1 month later.

Final build - same as RC1 if showstoppers are not found. Otherwise same as RC2 or if only one or two minor issues are found then RC1 + fixes.

We are also discussing on the IronPython mailing list which Codeplex issues will be looked at for an IronPython 2.0.2 release, which will hopefully be coming soon... If you have favourite bugs now is the time to speak up!

1) Time is critical so that fact I can learn more .Net and Python at the same time is just great.

2) The Framework is substantial, stable and well maintained. In CPython I am frustrated by non-core libraries getting out of sync (some 2.4, some 2.5 etc months apart).

3) Python is a good application language, a good scripting language and it is currently the best and most mature one for .Net. The IronPython team are helpful and responsive too. It is also maturing very well.

4) IronPython makes it easier to get the best out of Windows - it is a more natural fit than CPython.

5) Winforms is nicer than WxPython :-) wxPython itself is great but can’t disguise the C style API enough for me. Nightmares of MFC development return too quickly!!

6) Silverlight is a great solution for Python in a browser. Nothing comes close in the CPython world.

Don't get me wrong - I have 3 versions of CPython installed and use Google App Engine too !!

Monday, April 27, 2009

Last week I was very fortunate to record a podcast on IronPython and dynamic languages on .NET with Scott Hanselman.

Scott is a well known developer who works for Microsoft, and one of the authors of Professional ASP.NET MVC 1.0. In this 46 minutes podcast we discuss why dynamic languages are interesting and gaining in popularity, and how IronPython fits into the .NET world - touching on why dynamic languages are easier to test and using IronPython to allow user scripting of .NET applications. Naturally both IronPython in Action and Resolver One get mentions along the way.

Michael Foord makes his living as a Python programmer. More specifically as an IronPython programmer. He chats with me about his company's use of IronPython, the DLR and why they picked Python over C# or VB.

I've also recorded an episode with a new .NET podcast called Sod This, which describes itself as "brain burps for the tech savvy". This is the creation of Oliver Sturm and Gary Short, two UK MVPs and devexpress evangelists. The interview took place during the BASTA conference in Germany; in a bar, so the audio starts of a bit rough but improves as the interview progresses. I even reveal my mystery past and what I did before programming in Python.

A few weeks ago I blogged about the podcast I'd done with the .NET Rocks guys. I've just discovered that they have a full transcript of the recording online, so if you prefer reading to listening it may be your cup of tea:

And finally, videos rather than podcasts. The videos from PyCon 2009 started to show up online almost as soon as the conference was finished. The tutorials took rather longer as they are three hours long each. The videos of the IronPython tutorial that Jonathan Hartley and I presented have gone up. You can read about the tutorial, with links to the three videos, here:

A new release of Resolver One. The major new features in this release are that we now only have one installer (instead of six different ones) with features activated by license. There are also 54 new financial functions, the ability to set alignment of text in cells, and you can now create new worksheets (and change the selected worksheet) inside button handlers. There are also a host of minor bugfixes as usual, including the ability to embed the Resolver One spreadsheet calculation engine in IronPython, without the need to use the workarounds that were necessary in previous versions. You can read the full changelog here.

Switching to a single installer was a lot of work for us, and although it makes things simpler for users (you hopefully!) it is hardly a major feature. We're well into the work on version 1.5, which does have some exciting new features and will hopefully surface soon...

Robert Anderson is the CTO of Digipede and he has been experimenting on distributing IronPython jobs across a Digipede network. Based on his experimentation they've built features into Digipede 2.2 to make this possible, and he's produced an IronPython Worker Sample demonstrating how the DLR integration works.

I have been playing with IronPython a little. With the release of Digipede Network 2.2, I am now able to post the sample I wrote.

By the way, one thing I like about this sample is that it shows how to keep user code completely de-coupled from the Digipede Network while still taking advantage of our deployment and payload distribution model. This has always been supported by the Digipede Network, and this makes a good example.

The primary goal of this sample is to demonstrate the standard Digipede Worker pattern using native IronPython objects. In addition, this sample provides• an example of using a Digipede Executive to host dynamic languages supported by the Microsoft Dynamic Language Runtime (DLR);• an example of a re-usable shim that de-couples user code (in this case, the Worker.py) from any dependency on the Digipede Network.

With the news that Digipede supported IronPython we were interested in seeing if it could be used to distribute spreadsheet calculation tasks from Resolver One across a grid. We produced a demo, which required slight modifications to Resolver One which will be included in Resolver One 1.5, and demoed it to the Digipede guys:

We had a call this morning with Giles Thomas and Glenn Jones of Resolver Systems. They demonstrated Resolver One running on the Digipede Network.

They used my IronPython Worker sample and customized the front-end Python code, leaving the C# adapter as-is. With very little coding they had an elegant grid-enabled spreadsheet. Try doing that with a spreadsheet or grid that isn’t based on .NET . . .

Giles said they will have support for this in Resolver One 1.5, coming out in the next couple of weeks.

Very cool.

I’ve just installed Resolver One to take a closer look. Already I’m impressed, but I’ll leave that for a future post.

Sunday, April 26, 2009

A C# project which, as far as I can tell, (the project page is in Japanese) sets up a local IRC server that acts as a gatway to twitter. It is scriptable in both IronPython and IronRuby and includes some example scripts to get you started.

"LitS3 is a library written in C# that provides comprehensive and straightforward access to Amazon S3 for .NET developers". s3cmd.py, by Atif Aziz is a command line interface to LitS3 written in IronPython:

Here's some code that I've done while learning the language. In this python code I explored the Google Calendar through its API. Creating an instance of the service,ClientLogin and retrieving entries were pretty much straightforward. But creating,updating and deleting calendar's event(or entries) really took me sometime to make it work. On creating entries, I had a hard time coding the this

Insert(TEntry)(Uri, TEntry)

in python. It took a help from IPY mailing list to get on with my programming. On updating and deleting entries, I thought it would be straightforward but it's not. But, anyway I figured it out how to delete or update entries.

DiscoveryScript is an Excel add-in that makes it very easy to run Python scripts from Excel ("Excel macros in Python") using IronPython. DiscoveryScript uses IronPython version 1.1.2.DiscoveryScript includes a Python console that you can use for testing commands. The 'excel' variable in the python script refers to an Excel Application object. See the Microsoft documentation for the COM object model for Excel.

Another blog entry from Dave Fugate, this one explaining recent changes to the way the IronPython source code is layed out on Codeplex:

Anyone who has seen the IronRuby source repository directory structure on Git knows that both IronPython and DLR's public repositories on CodePlex are heavily modified from the original version we use internally at Microsoft. Well it's still nothing like IronRuby's layout, but we have taken a few steps to get IronPython's CodePlex sources more in synch with our internal repository structure. In a nutshell, this means that all DLR C# projects have moved into a new "Runtime" directory and Silverlight projects can now be found under "Hosts\Silverlight".

One of the important additions to IronPython 2.6 is something that enables several important new ways of integrating with the .NET framework. One the use cases is for .NET attribute support, which has long been missing from IronPython.

The new feature is __clrtype__. This allows you to customize the CLR type used for Python classes. By default a Python class in IronPython is not a true .NET class. This is because Python classes are much more flexible than .NET classes (and can be garbage collected) and are better represented by a .NET object than a .NET type. This unfortunately means that as well as not supporting attributes, Python classes don't play well with any .NET feature that uses reflection over the CLR type to introspect the features of an object. This particularly includes data-binding.

__clrtype__ works by extending the standard Python metaclass mechanism that allows you to customize class creation. To customize the CLR type that forms the underlying type for your class, your metaclass should implement the __clrtype__ method to return a System.Type.

The new __clrtype__ feature is pretty raw, and it likely that layers of functionality can be built on top of it to provide easier access to the things it makes possible.

In the meantime DevHawk (Harry Pierson the IronPython PM) has written a series of blog posts showing how to use the new magic.

In IronPython 2.6, we’re adding the ability to customize the CLR type of Python classes. This means you can add custom attributes, emit properties, whatever you want. For those of you who’ve been dreaming of implementing WCF services or databinding in Silverlight purely in IronPython, then this is the feature for you.

In a nutshell, IronPython 2.6 extends Python’s metaclass feature that lets you to customize the creation of classes. In the metaclass, you can implement an IronPython-specific method __clrtype__ which returns a custom System.Type of your own creation that IronPython will then use as the underlying CLR type of the Python class. Implementing __clrtype__ gives you the chance to implement whatever reflectable metadata you need: constructors, fields, properties, methods, events, custom attributes, nested classes, whatever.

Before we start using __clrtype__ metaclasses, we need to understand a bit about how IronPython maps between CLR types and Python classes. IronPython doesn’t support Reflection based APIs or custom attributes today because IronPython doesn’t emit a custom CLR types for every Python class. Instead, it typically shares a single CLR type across many Python classes. For example, all three of these Python classes share a single underlying CLR type.

Technically, you could emit whatever custom CLR Type you want to in the __clrtype__, but typically you’ll want to emit a class that both implements whatever static CLR metadata you need as well as the dynamic binding infrastructure that IronPython expects. The easiest way to do this is to ask IronPython emit a type that handles all the dynamic typing and then inherit from that type to add the custom CLR metadata you want.

Let’s start simple and hello-worldly by just customizing the name of the generated CLR type that’s associated with the Python class. There’s a fair amount of boilerplate code that is needed even for this simple scenario, and I can build on that as we add features that actually do stuff.

Now that we have the basic __clrtype__ metaclass infrastructure in place, let’s enhance it to add support for CLR fields. To do this, we’re going to need to add two things to our custom CLR type. First, we need to define the fields themselves. Second, we need to make sure that Python code will read and writes to the statically typed fields for the specified names rather than the storing them in the object dictionary as usual. Here’s the updated version of ClrTypeMetaclass.

When I was first experimenting with __clrtype__, I got to the point of making CLR fields work and then immediately tried to do some data binding with Silverlight. Didn’t work. Turns out Silverlight can only data bind against properties – fields aren’t supported. So now let’s add basic property support to ClrTypeMetaclass. Python has a rich mechanism for defining properties, but hooking that up requires DLR binders so for now I’m going to generate properties that are simple wrappers around the associated fields.

I’ve gotten to the point where I can actually demo something interesting with __clrtype__ metaclasses: Silverlight Databinding. This is a trivial sample, data binding a list of Products (aka the sample class I’ve been using all week) to a list box. But according to Jimmy, this is something he gets asked about on a regular basis and there’s a AgDLR bug open for this. The __clrtype__ feature is specific to IronPython but I bet the IronRuby guys could implement something similar if they wanted to.

Saturday, April 25, 2009

MSDN Flash is a UK developer magazine distributed monthly by Microsoft to ten of thousands of developers with news and upcoming events. Every issue also includes a 400-500 word technical article and two of the articles in 2008 were on the subject of IronPython and the Dynamic Language Runtime.

The best articles of 2008 have been collected into a free ebook (pdf), and both the articles on dynamic languages are included:

Every two weeks we send an email out to tens of thousands of UK based developers. This email is called the MSDN Flash. The Flash contains many useful sections including a 400 to 500 word technical article on a developer related topic either written by a member of the Microsoft UK technical team or a member of the broader UK developer community. We have had some great articles over the years which to some extent end up “hidden away” in the archives of the MSDN Flash. This is a shame as the authors have worked hard to condense complex topics into short articles which are informative and take only a few minutes to read.

I decided it was time to surface the best of the articles on a regular basis and provide them in an easy to download and read eBook. This initial edition covers the period January 2008 to January 2009 but we aim to create a new eBook every six months from here on in. The next edition should therefore be out around August 2009.

One of the classic use cases of IronPython for .NET developers is embedding it into applications written in C#. The DLR is designed to be used in this way and has a nice API for embedding. This makes it easy to add user scripting to applications.

It also allows for a more general extensible architecture, where part of the application is written in IronPython and it can also be extended with IronPython.

MetaNote is a text editor. Ultimately, MetaNote intends to be the most versatile editor imaginable. See that button in the toolbar? Right click on it, and edit the code behind it. Don't like the way 'Find' works? -- right click on it, and edit the code. Need a new button in the toolbar? So add it already, with a single click. Share packs of extensions and macros with other users. Everything in MetaNote is under your control, effortlessly, at runtime.

MetaNote is open source, with a Google Code Project Page where you can checkout the sources or download a binary release.

Friday, April 24, 2009

This Japanese blog entry shows two code examples for working with environment variables. The example for Python (CPython - the 'standard' implementation of Python) uses the os.environ, although the code shown should work fine with IronPython as well. The IronPython example uses the .NET class System.Environment to access environment variables and System.IO.StreamWriter to save data to a file.

This Japanese blog entry starts with a brief example of Python syntax and then a fairly complete example of embedding IronPython 2 in C#. It includes setting up the DLR classes (engine and execution scope etc) and then accessing Python variables by type, pulling a Python list out as an instance of IronPython.Runtime.List.

Extension methods are a statically typed version of monkey-patching for .NET languages; allowing you to add methods to types by defining static methods. They don't (currently) play well with IronPython, although there are workarounds explored by Saveen Reddy in these blog entries. This Chinese blog entry illustrates the differences with a nice big diagram.

A very interesting entry using IronPython to programmatically post to a Wordpress blog using the XML-RPC API. The code shown is very simple and uses two assemblies: SharpLab.XML-RPCLib2.dll and SharpLab.WP-XML-RPCLib2.dll. The core class seems to be WordPressClient. As far as I can tell this is code that accompanies what was actually the first book on IronPython (albeit in Japanese): IronPython - Windows Script Programming. (Although that assumption is likely to be wrong given that the blog this post is on is called SharpLab. I couldn't find a download link for those assemblies those.)

Ioke is a dynamic language created by Ola Bini, one of the core developers of JRuby (Ruby on the JVM). Ioke aims to be a highly expressive language, influenced by Io, Smalltalk, Lisp and Ruby. The primary implementation of Ioke runs on the JVM; but Ola recently unveiled the .NET (and Mono) implementation: Ioke for the CLR:

Ioke E ikc is the first release of the ikc Ioke machine. The ikc machine is implemented in C# and F# and run’s on Mono and .NET. It includes all the features of Ioke E ikj, except for Java integration. Integration with .NET types will come in another release.

There are several interesting pieces in ikc. Among them I can mention a new regular expression engine (called NRegex), a port of many parts of gnu.math, providing arbitrary precision math, and also an implementation of BigDecimal in C#.

Cobra is a .NET programming language, with Python inspired syntax, dynamic and static binding and first class support for unit tests and contracts. Several updates since I last reported on Cobra. These updates include many new language featuress and documentation improvements.

Ben Hall is a UK developer working for Red Gate. In this article Ben demonstrates creating a new generator for SQL Data Generator which allows the user to define the data to be generated based on IronRuby code. The article includes the source code and binaries for the SQL Data Generator example generator which you can start using within your project:

You can embed DLR-based languages such as IronPython and IronRuby into applications. This gives you instant scripting, and with it, the potential for greatly simplifying the development process. You can even take advantage of existing applications that have a plug-in architecture, such as SQL Data Generator and .NET Reflector. Ben Hall takes SQL Data Generator and shows how this technique allows you to create simple generators specifically for your needs.

ASP.NET MVC is .NET's belated answer to web application frameworks like Django and Rails. Phil Haack is one of the core developers, and this entry he explores one of the ways that dynamic languages based on the DLR can be used with ASP.NET MVC - specifically IronRuby.

What if you could write all your business logic and controller logic in your language of choice, but have your views written in a light weight scripting language. If my web application were to host a scripting engine, I could actually store the code in any medium I want, such as the database. Having them in the database makes it very easy for end users to modify it since it wouldn’t require file upload permissions into the web root.

This is where hosting the DLR is a nice fit. I put together a proof of concept for these ideas. This is just a prototype intended to show how such a workflow might work.

Where you can download the source code that accompanies the book and also download two chapters for free. These are chapter 1 (introduction to IronPython) and chapter 7 (on testing with unittest). There is also a discount code for purchasing the ebook or print book direct from Manning.

The blog where I post news about IronPython in Action, especially reviews and updates to the sources / errata.

Below is the preface to IronPython in Action. It tells the story of how Christian and I came to be involved in IronPython and write the book.

A programming language is a medium of expression. —Paul Graham

Neither of us intended to develop with IronPython, least of all write a book on it. It sort of happened by accident. In 2005 a startup called Resolver Systems set up shop in London. They were creating a spreadsheet platform to tackle the myriad problems caused in business by the phenomenal success of spreadsheets. The goal was to bring the proven programming principles of modularity, testability, and maintainability to the spreadsheet world—and having an interpreted language embedded in Resolver One was a core part of this. As Resolver One was to be a desktop application used by financial organizations, it needed to be built on established and accepted technologies, which for the desktop meant .NET.

At the time the choice of interpreted language engines for .NET was limited; even IronPython was only at version 0.7. The two developers who comprised Resolver Systems (1) evaluated IronPython and discovered three important facts:

Although neither of them was familiar with Python, it was an elegant and expressive language that was easy to learn.

The .NET integration of IronPython was superb. In fact it seemed that everything they needed to develop Resolver One was accessible from IronPython.

As a dynamic language, Python was orders of magnitude easier to test than languages they had worked with previously. This particularly suited the test-driven approach they were using.

They decided to prototype Resolver One with IronPython, expecting to have to rewrite at least portions of the application in C# at some point in the future. Three years later, Resolver One is in use in financial institutions in London, New York, and Paris; and consists of 40,000 lines of IronPython code (2) with a further 150,000 in the test framework. Resolver One has been optimized for performance several times, and this has always meant fine tuning our algorithms in Python. It hasn’t (yet) required even parts of Resolver One to be rewritten in C#.

We are experienced Python developers but neither of us had used IronPython before. We joined Resolver Systems in 2006 and 2007, and were both immediately impressed by the combination of the elegance of Python with the power and breadth of the .NET framework.

Programming is a creative art. Above all Python strives to empower the programmer. It emphasizes programmer productivity and readability, instead of optimizing the language for the compiler. We’re passionate about programming, and about Python.

In 2007 one of us (Michael) set up the IronPython Cookbook to provide concrete examples for the newly converging IronPython community. Shortly afterwards the two of us decided to write a book that would help both Python and .NET programmers take advantage of all that IronPython has to offer.

1 Giles Thomas, who is CEO and CTO, and William Reade, a hacker with a great mind for complex systems.2 With perhaps as many as three hundred lines of C# in total.

Like most UI toolkits WPF requires that interaction with components of the GUI be done on the same thread that they were created on. The WPF technique for communicating with the user interface from another thread involves using a dispatcher.

Jose Ramon Calzada discovers that Python decorators provide an ideal way of ensuring that functions or methods are invoked onto the UI thread.

Curt Hagenlocher (core IronPython developer blogging as IronCurt) had been working on a profiler for IronPython; one of the standard programming tools that up until now has been difficult or unavailable for IronPython. This is his second post on the profiler, which is being built into IronPython itself. He explains how it works, how to use it, and how it can also be used as a code coverage tool.

Harry Pierson (IronPython PM and blogging as DevHawk) has been working on an IronPython debugger. In this blog entry he explains how he has extended the breakpoint handling of his debugger beyond the simple breakpoint setting that he implemented very early on:

"Setting a breakpoint was the second feature I implemented in ipydbg. While setting a breakpoint on the first line of the Python file being run is convenient, it was obviously necessary to provide the user a mechanism to create their own breakpoints, as well as enable and disable existing breakpoints."

I very recently blogged about the interesting Entity Framework Profiler, with scripting support through the DLR, that Simon Segal was working on. He has posted a new entry, which also makes the version of the profiler with scripting support available for download:

"Because I would like to write my Repositories and monitor their behaviour in the Profiler I decided that I could kill two birds with one console (so to speak). The first bird to kill involved my spending as much time as possible in the pursuit of both learning IronRuby and gaining better understanding of the more interesting problems that can be solved with dynamic languages implemented on top of the DLR. I can see the possibilities of hosting the DLR to provide scripting support to applications and after having implemented my own hosted IronRuby consoles I eventually decided to reuse one that was more fully featured, saving me quite a bit of time by best fitting the Profiler’s needs. The console I chose in the end was IronEditor which is available on codeplex."

I had a requirement to create a state machine for a product at work. The product is a project management tool and therefore has the concept of a job which based on the users actions moves from one state to another. There was also the additional requirement that the state machine should be configurable by/for different customers. Esentially, the state machine is a large flow diagram but the implementation needed to allow for completely ripping up the first customers flow diagram and replacing it with a completely different one for the next customer.

From the C# prototype I created I had a number of helper methods for doing things to manipulate my job business entity and even create a project in project server! I didn't really want to throw this away, so I didn't - I created an abstract class and then implemented the subclass in IronPython.

To do this I needed to import the classes from my C# code which I will be using (such as Job) and then subclass my StateMachine abstract class and implement the Transition abstract method. You will notice the reference to self in the Transition methods parameter list which basically means that it is an instance method.

In order to be able to use the IronPython implementation of the state machine, I needed to read in the code, essentially compile it and then keep a handle to the state machine so that I could use it at some point in the future.

Daniel Paull has an implementation of John Conway's Game of Life (cellular automata) written with XAML/WPF (Windows Presentation Foundation - a Microsoft UI library built on top of DirectX) and using embedded Python scripts.

Following on from my series on embedding DLR scripts in XAML, I present an implementation of John Conway's Game of Life in XAML/WPF using embedded Python scripts. The game is loaded completely from loose XAML. Even the initial game state is defined in dynamically loaded XAML files!

The game is hosted in the very dynamic application described in a previous post.

The Game of Life is played by calculating successive generations of the board. We add a button to calculate the next generation. A simple implementation of the Game of Life is written in the Python script. The button's DataContext is bound to the board, allowing the script to retrieve the board from the button.

Sunday, April 19, 2009

Benjamin Nitschke's has written a couple of posts on writing your own programming language with the Dynamic Language Runtime.

The first is a long list of links and resources on the DLR and the tools (like the ANTLR parser) used in creating a programming language. The second is a very long entry comparing the performance of code written in various ways - including on top of the DLR.

I will also try to post some useful links about my recent DLR (Dynamic Language Runtime for .NET) research.

I have been working a bit on the DLR before, mostly together with Silverlight, which was cool, but Silverlight was way too hard to work with and I still think it is not distributed enough. And even before that quite a bit with the Visual Studio SDK, early IronPython versions and other language implementations in .NET and even with native c code (but usually I just modified existing samples). I have also modified Lua for my own needs recently and made it run on the Xbox 360 and PS3 and modified some behaviour for my projects, since we use it for our upcoming game and I use it quite a lot together with IronPython on the tools I write at work.

The last few days I was working a little bit with the DLR and got to a point today where I was wondering about the performance of a very simple while loop in the DLR, specifically in the ToyScript DLR sample project and of course IronPython. I am totally aware that the following comparison DOES NOT really provide accurate information about the actual performance of the languages and techniques used. Please keep this in mind when reading this article, it only covers a very certain aspect of all used languages. More specially how they perform when adding numbers in a while loop a lot of times! Some of it reflects that compiled languages are faster than dynamic languages, but you should not compare the absolute values, Python or Lua is not really 100 times slower, in many cases you can reach almost similar performance as long as you implement or call performance-critical code in a clever way (e.g. calling C++ code, using frameworks, etc.).

I was only interested about very rough comparisons to figure out which ways are fast and which languages or techniques are kinda slow. Some code written in Assembler or direct IL code execution was really fast (duh, no wonder), but all execution times of C++, C#, Assembler or IL are way faster than all other dynamic languages or DLR approaches. The important thing for me was to figure out why dynamic languages like IronPython or ToyScript on top of the DLR were like 100 times slower for this specific code. At the end of writing this article I was able to provide pretty good performance with the DLR, which is just 2-3 times slower than the fastest execution time. That's pretty good and could probably be even improved more.

Basically I used the following code sample, which adds 1.0 to counter 10 million times while decreasing the loops variable until it reaches 0. In languages that use types, counter would be a double (64bit floating point number).

A very interesting entry that compares the performance of a simple loop in:

C# 3.0

Native C++

Assembler

IronPython 2.6

ToyScript

Python 2.6

Python 3.1

Lua 5.1.4

Script.NET

IL Emit

Irony + DLR (dummy)

DLR AST with objects

DLR AST with doubles

His charts not only compare execution times but also parse and build times.

Based on the work done by Ben Hall in his IDE for DLR languages, IronEditor, Simon Segal has added "scripting into the profiler so users could compose scripts and execute Entity Framework code directly from the Profiler itself".

Pygments is an extremely versatile Python library that does syntax highlighting. It not only supports many input languages, but can also output in many common markup formats (for wikis and so on).

Windows Live Writer is a WYSIWYG Windows blogging tool by Microsoft. It allows you to write blog entries on your desktop (shock news - not everything needs to happen online), including adding images and videos. It integrates with blog services like Windows Live, Wordpress, Blogger, Live Journal, TypePad, and many more. I've never used it, but hear good things about it from developers I respect.

Devhawk (Harry Pierson the Microsoft IronPython PM) uses live writer, and has written a plugin that uses IronPython and Pygments with a Windows Forms multi-line textbox as the code editor window.

Pygments for WL Writer is a smart content source. In WL Writer’s terminology, that means when you click inserted text in the editor window, it is treated as an atomic entity which you can then edit by using the Edit Code button in the Pygments for WL Writer sidebar editor. I I often found that I would edit my code multiple times – usually to shorten lines so they’d fit on my blog without wrapping. CodeHTMLer for WL Writer is a standard content source, so it just spews the formatted code as HTML onto the page.

From an IronPython perspective, there’s some interesting stuff there. I decided to compile the pygments library into a DLL for easier distribution. If you look in the source, there’s a folder for the Pygments source as well as the parts of the standard Python library that Pygments depends on and my custom HTML formatter. Those all get compiled via a custom script which can be called by the build.bat file in the project root.

As usual this project is available for download from Harry's Skydrive. The source is up on Github. There are plenty of features he is thinking of adding, so if you use it don't forget to give him feedback.

In a separate post he notes an update he has made (download URL is unchanged).

The only change is that I now override OnSelectedContentChanged in the sidebar control. That way, if I have multiple blocks of pygmented code in a given post, the sidebar UI updates with the correct language and color scheme of the currently selected code block.

Quite a few of Harry's projects are up on GitHub. Harry has enabled issue tracking so that you can now submit issues, bugs and feature requests:

Jim Hugunin's talk on the status and future of IronPython, including how IronPython has affected the Common Language Runtime and C#.

Come learn about our secret plans to use IronPython to take over the world. The emphasis will be on demos showing the seductive possibilities that IronPython enables for Python developers. These include taking advantage of the newest features in Windows 7, running your Python code in the browser with Silverlight, and many more.

IronPython is an implementation of Python running on .NET. This talk will provide an overview of the IronPython internals. The talk will start with a high-level walk through of the IronPython architecture and source layout. From there we’ll drill into details such as method dispatch, how the .NET and Python type systems interrelate, multi-runtime support, and how IronPython uses the DLR. Finally we’ll bring it all together and show you how you can do Python aware interop between .NET and IronPython. Whether you’re just curious about language implementations, planning on hosting IronPython in your own app, or just want to write a new built-in module for IronPython you’ll find something interesting in this talk. Knowledge of C sharp would be helpful but is not required.

Sarah Dutkiewicz's talk on the community distribution of IronPython: FePy. Notably the demos were done from a Linux VM hosted in Windows.

As Python grows in popularity, IronPython has started making more waves. What is IronPython and why should regular Python programmers be familiar with it? This session will introduce the open source .NET implementation of Python known as IronPython without using Windows. Come see FePy (IronPython community edition) in action via Mono -- a cross-platform open source implementation of the .NET framework.

Microsoft .NET DLR: Rolling along nicely; they have fixed a number of performance problems since last year, and have matured their dynamic language support, as showcased in IronPython. Call-site caching is the name of the game this year, on DLR, the JVM (that's what invokedynamic does), and the rush of new high-performance JavaScript implementations. DLR folks also enjoy working with .NET expression trees, very rich run-time type information, reified generics, iterators (generator-like things). They aren't fooling with continuations, at least this year. They have apparently sped up tail-call, which used to be much slower than regular calls.

The language summit addressed several issues, including how the main Python project can make life easier for the other implementations.

As for the discussion around alternative implementations; the most concrete decision was to give checkin privileges to key contributors to the major implementations so that the Python tests can be improved and made more useful. For example, both IronPython and Jython have identical modifications to several Python tests that currently depend on reference counting and deterministic finalization. The goal is to get to the point where IronPython, Jython, and PyPy (etc) don't have to maintain their own (different) versions of the Python tests.

As part of this the Unladen Swallow guys are intending to contribute their benchmarks to core Python, so that we can have a standard set of benchmarks for measuring the performance of Python implementations. The discussion around this is happening on the previously unused stdlib-sig mailing list.

Curt Hagenlocher (core IronPython developer) has a blog entry outlining a new profiling tool for IronPython which is included in IronPython 2.6 Alpha 1. With Harry Pierson's debugger we're almost spoiled for tools now!

I was recently working on improving the performance of a Python-based application and thought it would be useful to profile my code. My intuition was telling me that the problem was in a certain area of the code, but I wanted hard numbers to back up this intuition before I went to the trouble of doing a rewrite. Unfortunately, neither the CLR Profiler nor the profiling support in Visual Studio are very effective when used with IronPython, as they weren’t really created for environments where code is dynamically compiled at runtime.

But Tomáš had written a profiler for IronRuby nearly half a year ago, so I thought I’d port this to IronPython, where it’s now experimentally available in IronPython 2.6 Alpha 1.

The way Tomáš’ profiler works is pretty straightforward. At the start of every method we compile, a snapshot is taken of the current time. Another is taken at the end of the method and the difference is stored for later retrieval. The only tricky thing about it is trying to update the statistics in a manner that’s both thread-safe and has minimal effect on performance.

I’ve modified it a bit from the original, though, and added some fun features. In particular, it now tracks the number of calls made to each method and also keeps separate counters for the amount of time spent exclusively inside the method versus the duration spent inclusively in both this method and in any methods it calls.

Friday, April 10, 2009

Dave Fugate is an IronPython tester, and often responsible for releases. He's posted a few blog entries on recent changes in IronPython 2.6 (alpha 1 was just released) and building IronPython from source for Silverlight.

In Python 2.6 the standard library collections module uses sys._getframe, which doesn't work for frame depths greater than 0 in the current IronPython. A whole host of other standard library modules depend on collections, meaning that they are all currently broken for IronPython 2.6! Dave explains how to modify collections.py to get round the problem. The good news is that in the final version of IronPython 2.6 getframe will be available from a command line switch (there is a performance hit for having it enabled). Without the switch it will be undefined (and collections.py already has a workaround for this), so that in either case you will be able to use an unmodified standard library.

Not as easy as you might expect it turns out... Dave tells us the magic incantations required to build source releases of IronPython and IronRuby for use with the Silverlight browser plugin. Writing Python code that runs in the browser is great fun by the way.

One of the big changes in IronPython 2.6 is the use of adaptive compilation for performance reasons. Compiling Python source code to IL bytecode is expensive (but has performance benefits itself) and if you only execute code once then it isn't worth the overhead. With adaptive compilation, running Python code from source with IronPython doesn't compile by default, but compiles code in the background that you execute more than once. This improves both startup and execution time for scripts or infrequently executed code. If you are compiling your code and doing a binary distribution this change doesn't affect you.

As adaptive compilation is a work in progress you may currently see a performance regression with IronPython 2.6. Using the -O command line option switches IronPython back to compiling everything (the 2.0 behaviour).

After a general introduction to dynamic languages in the DLR, I’ll talk about what you have to download and install in order to use dynamic languages with .NET. Then, we’ll look at two of the new .NET Dynamic Languages, IronPython and IronRuby and we’ll work with some of the basic syntax using the command line interface. Then we’ll talk about the role of dynamic languages in Silverlight and we’ll use three tools for creating, editing and testing web applications that use dynamic languages: Chiron, the DLR console and Visual Studio 2008.

This is the Dynamic Languages SDK for Silverlight that recently had a refresh for Silverlight 3. In this post Harry discusses XapHttpHandler, an ASP.NET handler for IIS that can dynamically build dynamic applications using Chiron. He particularly likes this feature as he wrote it!

XapHttpHandler does the same exact on-demand XAP packaging for dynamic language Silverlight applications that Chiron does, but it’s implemented as an IHttpHandler so it plugs into the standard ASP.NET pipeline. All you have to do is put the Chiron.exe in your web application’s bin directory and add XapHttpHandler to your web.config

Something else that got releases recently was the first alpha of IronPython 2.6. This version of IronPython will target Python 2.6, and performance improvements (particularly startup and import time) have been the major focus of this version. In this entry Harry talks about one of the new techniques IronPython uses to improve performance: Adaptive Compilation. Something else that will be in IronPython 2.6 is support for Python stack frames.

Kean Walmsley, of the "Through the Interface" blog, is an AutoCAD programmer who has been meaning to try out Python for some time. He is somewhat sceptical of dynamic typing, but has heard many good things about Python:

Python is also of interest because of its cross-platform availability: it’s an open source language with its roots in the UNIX/Linux world, but is now gaining popularity across a variety of OS platforms (one of the reasons it’s the scripting language chosen for at least one of our cross-platform products, Autodesk Maya).

He has a (short) series of blog entries on getting IronPython (and IronRuby) working with AutoCAD.

I had originally hoped to build a .NET assembly directly using IronPython – something that appears to have been enabled with the 2.0 release of IronPython - which could then be loaded into AutoCAD. Unfortunately this was an exercise in frustration: AutoCAD makes heavy use of custom attributes for identifying commands etc., but IronPython doesn’t currently support the use of attributes.

Then, thankfully, Tim Riley came to the rescue: we’ve been in touch on and off over the years since he started the PyAcad.NET project to run IronPython code inside AutoCAD, and Tim was able to put together some working code which actually registered commands (after I’d pointed him at a function he could use from AutoCAD 2009’s acmgdinternal.dll – an unsupported assembly that exposes some otherwise quite helpful functions). He ended up choosing an implementation that had also been suggested to me by Albert Szilvasy: to implement a PYLOAD command using C# which allows selection and loading of a Python script (because Python is, ultimately, all about scripting rather than building static, compiled assemblies).

After getting my feet wet in the last post with my first IronPython application running inside AutoCAD, I decided it was time to attack a slightly more challenging problem: jigging a database-resident Solid3d object. The idea had come after I’d received a question by email from David Wolfe, who wanted to have a fully rendered 3D view of a cylinder he was jigging.

From what I can tell – and I’m really a newbie in both these languages – there is relatively little to separate the two: both Ruby and Python have their devotees but they ultimately to belong to different sects of the same faith (it’s hard to escape religious analogies when talking about programming languages, for some reason :-). That said, the fiercest arguments often seem to occur between people who very nearly agree. This is not something I know enough about to get involved in – even if I wished to – so I’m going to steer well clear of it, and simply show some simple Ruby code that does the same as in the equivalent post for IronPython. If you Google “python ruby comparison” you should find plenty of opinions out there.

Tuesday, April 07, 2009

I was recently at the QCon conference in London, speaking on IronPython in the "Emerging Languages" track organised by Ola Bini. After the conference the guardian newspaper published an article on the diverse languages that are becoming mainstream. At first I was surprised that a British paper could publish a sensible article on programming languages, but then I noticed it was by Tim Anderson - a noted and long time tech blogger and writer. A couple of extracts:

The recent trend is towards dynamically typed languages, which use inference to reduce the amount of code to be written. Java, C# and C++ are "static typed" languages, whereas JavaScript, Ruby, Python and PHP (used by Facebook) use "dynamic typing". In static languages, the programmer must declare each variable's class (such as string, integer, or list) before the program runs; in dynamic typing, it is assessed as the program runs. Fowler says Ruby is between two and five times more productive than Java. "In static languages, there are all these speed bumps. You can just express yourself more rapidly in languages like Ruby and Python, and that gives you an edge. That edge does make a significant productivity difference."

Despite growing interest in emerging languages, the dominance of Java and C# is unlikely to be shaken soon. One reason is that many of the alternatives compile to a format that executes on the Java or .NET runtimes, such as Groovy and JRuby, which are dynamic languages for Java, and Iron Ruby, Iron Python and F#, which run on .NET. This enables easy integration and access to rich runtime libraries. "People are not learning new languages to escape from platforms, rather they are trying to find new ways of doing things better on the existing platforms," says Bini.

The Microsoft Live Framework is "the uniform way for programming Live Services from a variety of platforms, programming languages, applications and devices".

These samples are in addition to the ones that ship with the SDK and include ProfileInfo which "Demonstrates how retrieve a user’s Windows Live Profile information using IronPython and the Live Framework .NET Toolkit".

Part 1 of this post gave a quick overview of writing a custom “parser generator” for Excel formulas. Here we’ll take a look at a way to bind that to the DLR. You can try out a sample Silverlight 2.0 application here. (Code available for download from Codeplex.)

The goal here is to create a calculation engine that understands Excel-like formulas, allowing for cell dependencies, simple ranges and custom functions.

I thought it would be kind of fun to create an IronPython script that created a Windows Form which could execute a script. This means I could run the script and get window which I could then use to run the script again and get anther window, then..

A video from Mix 2009 on using dynamic languages to write and test Silverlight applications. Unfortunately it demos IronRuby, but John Lam is the presenter so it should be entertaining (and the core Silverlight development techniques for IronPython and IronRuby are very similar). The tag line: how programming with dynamic languages helps improve the dev experience.

A blog entry by Darren Hawley. It is an IronPython recipe showing a WCF (Windows Communications Foundation - the web services library included in .NET 3.0) host that consumes a library written in C#. It shows how to use ServiceHost, BasicHttpBinding and other core WCF classes.

It has become de-rigueur in these link collections to include an entry not in the English language. This article is an interview with William Rank, who is with the R&D company label.newtest-ip, which after using Python for years now uses IronPython in its applications. From google translate:

This language [Python] allows us to develop sophisticated scripts to reproduce complex transactions such as controlling the browser, simulating a mouse, keyboard ...

In fact we are not developing directly "by hand" the scripts, but we have automatic software generation to create programs from actions by a user (think of what the generator Office with macros and vba ). We may supplement the generated scripts, but most of the generation is automatic.

Then in 2005 we chose to develop our robots to measure technology. Net. The choice of. Net has been driven by gains in productivity, simplicity and power. So naturally we turned to for the IronPython scripting. This allowed us to fully exploit the environment. Net. On the other hand as we use third party technologies. Net choosing Ironpython vs Python was imposed on us.

Harry Pierson (Microsoft IronPython PM) has been experimenting with writing an IronPython debugger (ipydbg) on top of the .NET MDbg APIs.

When we last left him, he had implemented Just My Code (JMC) support allowing you to step into IronPython code. Since then he has made a lot of progress including a colourful REPL (interactive console) and solving some difficult problems around getting to real values behind boxed IronPython variables (needed for showing local variables). He has blogged extensively on what he has been up to (more recent posts last):

A lot of functionality is now present. In the command routing entry Harry outlines the seven commands supported by the debugger so far: Continue, Quit, Show Stack Trace, Show Locals, Step Over, Step In, and Step Out. The next step:

While having a debugger REPL is really convenient for prototyping new ipydbg commands, it’ll really shine once I get function evaluation working. Then I’ll be able to open a REPL console where the commands are executed in the target process instead of the debugger process as they are now. That will be very cool. Until then, the latest code is – as always – up on GitHub.

On the subject of downloading the source code for the ipydbg project, Harry has another blog entry outlining all of his tinkerings and code that he has made available. This includes codedom.py, ipypulldom.py, and xml2py.py - all under the MS-PL open source license:

Sunday, April 05, 2009

I'm late with the news, but there is a new release of IronRuby. There has also been a fresh release of the Silverlight dynamic languages to work with the beta version of Silverlight 3. Jimmy has an unhealthy preference for IronRuby over IronPython, but he's posted a few blog entries on both subjects.

Staying true to our “Conference-driven development schedule,” this release is in conjunction with MIX ‘09, where Silverlight 3 Beta and ASP.NET MVC 1.0 were announced, to name a few. John Lam spoke at MIX today at 12:30pm in Las Vegas about using dynamic languages in Silverlight for testing, scripting, and application building. And some ASP.NET MVC with IronRuby is thrown in there too.

IronRuby passes approximately 80% of RubySpec, the best test suite Ruby has today. IronRuby is best at language compatibility, passing 95% of those tests, but worse on the standard libraries with a pass-rate of 77%. Overall the pass-rate as gone up approximately 10% since the last release, and there hasn’t been a major library push since getting Rails running. For IronRuby 1.0, that number is going to be as high as possible, and we’ll ensure that by continuing to work with the RubySpec project to improve the test suite.

Since this release is around MIX, and John’s talk is going to be focused around Silverlight and ASP.NET MVC, it seems like a great time to polish IronRuby’s .NET interop. For example, the release enables calling generic methods, implementing interfaces with IronRuby, and better conversions between .NET and Ruby types. We’ve also started to build a .NET interop test suite, using the same infrastructure RubySpec uses (MSpec), to provide a executable specification of how IronRuby and .NET play together. There will also be a written specification showing what these .NET interop features are good for, but not for this release.

The Silverlight Dynamic Languages SDK (the snappily named SDLSDK) is the home for the tools to build Silverlight applications with DLR based languages. This release includes support for Silverlight 3, console (REPL) in Silverlight applications and testing support.

While many developers love to program in dynamic languages like Python or Ruby, they may not have that freedom in their day-to-day jobs, for a variety of tecnical and business reasons. However, dynamic languages are as useful when embedded in existing software, possibly making the design better than a non-scripting-enabled version. Languages based on the dynamic language runtime are really easy to embed in .NET applications, so hopefully this post inspires you to add scripting support to your applications.

Testing is a great use for dynamic languages, especially if the application being tested is written in a compiled language like C# or VB. Ruby is an especially interesting language for writing tests in due to the popularity of RSpec, a behavior-driven-development testing framework. IronRuby makes it really easy to test .NET applications because of it's .NET integration and ability to run real Ruby programs, like RSpec. However, this becomes much more difficult to do in Silverlight for a variety of reasons. And Silverlight doesn't have good options for testing to begin with, except for the Silverlight Unit Test Framework, but I yearn for something simpler. This post will show you how to test a C# Silverlight application with IronRuby, using a testing framework called Bacon, a lightweight RSpec clone.

So next someone should adapt the testing work to use unittest and IronPython...

In this podcast I have a fifty minute conversation with the .NET Rocks guys on IronPython and dynamic languages in general. I'm even nice about Ruby! Apparently the .NET Rocks podcast has around 300 000 (!) regular listeners, so it was great to be able to promote Python and IronPython in Action to the .NET community.

IronPython PM Harry Pierson on the CodeCast podcast. The actual interview starts a little way into the podcast:

In this episode of CodeCast, Ken Levy and Markus Egger discuss upcoming developer event news and dynamic languages. This show’s interview is with Ken discussing IronPython with Harry Pierson, a program manager on Microsoft’s dynamic languages team.

Thursday, April 02, 2009

IPY4SPD is a new project on Codeplex by Nathan Freeze - the iLoveSharePoint guy. It integrates IronPython scripting capabilities with SharePoint and made a release on March 31st. The tag line for the project is fun. The project page has some screenshots demonstrating how to use it.

IPY4SPD - When the need to get it done overwhelms the need to do it right!This project adds IronPython script activities to Sharepoint Designer allowing busy developers and power users to scratch out workflows quickly.

Now you can manipulate your SharePoint objects at will using the pure clean power of Python!

Script errors are formatted and reported in the workflow history.

Notes:Returning a value from the script variable is optional.References to System and Microsoft.SharePoint are passed to your scripts by default (just import them)Four contextual workflow variables are available to your script activity:

We've had a great time at PyCon and I'm still here in Chicago on the last day of the sprints. The whole event, from language summit through conference through sprints, was pure awesomeness and IronPython was right at the centre. There is also exciting news about IronPython in Action, but all of this will have to wait.

In the meantime there are a couple of IronPython events happening soon:

Aroj will take you through some cool ways you can use the power of Python in the .NET world. This talk includes a demo of embedding an IronPython engine in a .NET application to enable interactive exploration and dynamic behaviour.

On the 8th April Darrell Hawley is speaking to the Ann Arbor .NET developer's group:

You've heard about IronPython and maybe have even taken the time to write the obligatory "Hello, World" application. Now what? The buzz around IronPython continues but there's still little guidance on tools and development processes. This presentation focuses on the basics of interacting with C# and VB.NET libraries using IronPython while introducing useful tools and techniques to get you started. Though experience with .NET is necessary, the samples and discussions are understandable to even beginning Python developers.