EuroPython 2011 schedule

Monday 20th June 2011

Designing interfaces so that other code can interact with ours (whether our code is a library, framework, application, website...) is a very common and clearly crucial activity, but it's fraught with dangers -- there's a lot stuff we all keep doing wrong time after time. This talk shows some common categories of API design errors encountered in the wild, with tips on how to avoid them when you design your next API.

Goals: to show you how to focus resources and attention to help avoid mistakes in API design. Prereqs: experience designing and developing software, esp. software which needs to interact w/other existing software.

This is a story about how Spotify started, how we evolved, where we are today and how Python has always been a big part of our success. Where do we use it? Where do we not use it? Learn about the Unix way and the Debian way of maintaining a healthy love for Python, while two-timing with other languages such as Java and C++.

Twisted is one of the best asynchronous network programming frameworks out there, and can help you build cool stuff very easily, once you understand the core design.

Unfortunately, Twisted is also a huge framework and can be very daunting for beginners to approach. This training session aims to help people with little or no asynchronous programming experience understand both Twisted and asynchronous programming. The plan is to cover core asynchronous topics, the Twisted way, and as many practical applications including web services, IPC, GUI applications etc.

The audience should be competent in core Python constructs. No previous asynchronous or threading programming experience required.

UPDATE: Everyone that plans to attend the talk, please install Twisted (http://twistedmatrix.com). Make sure everything is OK by doing:

python
>>>from twisted.internet import reactor
>>>reactor.run()

The above should block until you hit Ctrl-C and then exit the Python interpreter.

Stackless Python and the greenlet package for CPython are two different implementations of coroutine support for Python. (Coroutines are fundamental building blocks of I/O frameworks like gevent, Eventlet, Concurrence and Syncless to conveniently handle thousands of socket connections at a time without creating threads.) Stackless and greenlet implement a different interface. However, each is powerful enough so that it can be used to emulate the other one. In this talk we explore the differences and discuss design decisions and testing strategies of the emulations we have implemented.

In Electron Paramagnetic Resonance Imaging, we are faced with a deconvolution problem that has a strong impact on the image actually reconstructed. Faced with the need of mapping the distribution of organic matter in Terrestrial and Martian rock samples for applications in exobiology, we needed to see how to extract a maximum amount of information from our data : our approach uses reservoir computing artificial neural networks coupled to a particle swarm algorithm that evolves the reservoirs’ weights.

The code runs on the Hybrid Processing Units for Science (HPU4Science) cluster located at the Laboratoire de Chimie de la Matière Condensée de Paris (LCMCP). The cluster is composed of a central data storage machine and a heterogeneous ensemble of 6 decentralized nodes. Each node is equipped with a Core2 Quad or i7 CPU and 3-7 NVIDIA Graphical Processing Units (GPUs) including the GF110 series. Each of the 28 GPUs independently explores a different parameter space sphere of the same problem. Our application shows a sustained real performance of 15.6 TFLOPS. The HPU4Science cluster cost $36,090 resulting in a 432.3 MFLOPS/$ cost performance.

That talk is meant to demonstrate on a practical case how consumer grade computer hardware coupled to a very popular computer language can be used to tackle a difficult yet very elementary scientific problem : how do you go from formulating the problem, to choosing the right hardware and software, and all the way to programming the algorithms using the appropriate development tools and methodologies (notably Literate Programming). On the math side, the talk requires a basic understanding of matrix algebra and of the discretization process involved when computing integrals.

Gevent (http://gevent.org) is a coroutine-based library that enables you to write highly-concurrent network and web applications. Learn in detail what Gevent is, what it does and how it does it.

I will introduce a coroutine-based approach to network programming, explain it benefits and pitfalls.We will walk through the Gevent API, including the most recent additions, and understand how it is implemented. I will make a quick overview of the most interesting 3rd party add-on and applications after that.

Django is an excellent web development platform with an extensive developers community. It may be the most popular Python web framework nowadays. However, the exisiting django based CMS products are not as feature complete as other comparable solutions (Drupal, Plone, Alfresco, ModX, etc.).

The Merengue project tries to fill this gap. It is financed by the University of Málaga (Spain) and takes advantage of existing code and knowledge from other projects (Plone, Drupal, Pinax, django-cms, feincms, LFC, etc.)

This training session will introduce the Python scientific stack to Engineers who use matlab in their day-to-day job and want to switch to an open solution or explore other alternatives. The basics of Python will first be presented: syntax, variable types and data structures, functions and flow control, exceptions. Python modules and tools required for matlab-like programmation in Python will be presented: ipython, numpy and matplotlib. Several Python applications typical of engineering problems will be presented and compared with their matlab version, as time will allow: plotting (time series, histograms, pseudocolor plots, etc.), basic I/O (e.g. ASCII, CSV, matlab MAT files), signal processing, mapping, etc. The creation of user interfaces with PyQt will be briefly introduced. Differences between Interactive and non-interactive programming will be described. Along the session, key differences with matlab will be underlined and discussed. Sources of information and documentation, online and offline, will be presented.

These concepts will be introduced as coding exercises using the Python programming environment provided by the Python(x,y) distribution, which is freely downloadable and includes recent versions of Python, numpy and matplotlib. This session will also focus on using Python(x,y) efficiently for Python programmation. Attendees should bring their own laptop running Windows. It is also recommended that they download and install the pythonxy distribution from http://www.pythonxy.com/.

Although no knowledge of Python is required to attend this session, a basic knowledge of matlab and of its typical programming usage is needed.

What was my motivation for working with Python after two years of PHP? There is so much you can do. Functions are objects you can pass around, you can decompile functions to see how they work internally, you can alias a bound method to a global function and it will just work. You can change import semantics, override the method resolution order, you can write metaclasses, you can hook into the CPython interpreter API and much more.

Five years later I came to the conclusion that this is really not what Python is about and that a lot of what I did was interesting but not necessarily a good idea. This talk is a 45 minute recompilation of things I really shouldn't have done in the first place. Expect a bunch of neat and interesting hacks that showcase the possibilities of the language.

At ep.io we built a Python hosting platform from the ground up, designed to run large numbers of web applications on a small number of physical machines both securely and in a reasonably scalable way. This talk will show you how we built our infrastructure - using Redis, eventlet, PostgreSQL and more - and what lessons we learnt from our first few thousand deploys.

See how we split services into multiple processes and greenthreads; the pains of building a cooperatively-multitasking PTY module; how Redis isn't the answer to everything, but is still very useful; how to persuade third-party software to work securely in a shared environment; and how important it is to have good logging, especially when you have more than five servers.

We will show the tools and the infrastructure that makes easy creating own python project in Ubuntu and distributing it to millions of users. It will be shown several tools: Launchpad, Quickly and and the Ubuntu's PPA (personal package archiving).

All of us are familiar with using open source tools to develop software applications with, but instead of writing code, it is also possible to create the manuscript of a book in very much the same way. These days, authors have many choices in terms of the tools which are available to aid them in writing a book. Gone are the times when the only option is using a proprietary word processor, so why not try it using open source tools? It's especially attractive to those who develop software using open source tools as well those who are writing books about open source tools. Both code and the written word are created in plain text files. These files are easily archived using some version control system. They can be shared easily via e-mail and can also be backed up in the same manner. In the end, the final product is shipped to the customer: in the case of software, it's distribution via CD or DVD, or perhaps a release for installation on a server, and for a book, it's delivering a manuscript to the publisher, perhaps in a format such as HTML or PDF®.

Upon some reflection, writing a book is, in many ways, similar to writing an application: you create text or code which is saved to a file (or files--typically one but could be more) file in some format, proprietary or otherwise. Additions, changes, or deletions are made to the file(s), which is then updated when the file is (re)saved. Backups need to be made, some versioning may be required to keep track of multiple files during the course of development, and finally, the resulting file(s) have to be delivered, as described above. Because of all the similarities described, we assert that it is possible to use a development environment utilizing Python and other open source tools to not only write application with, but also be able to create the entire manuscript for a book as well! While this entire concept should not really be mind-blowing, it does represent a trend seemingly growing in popularity in the open source world.

In particular, we highlight the following requirements that are necessary when creating a manuscript, especially with co-authors:

Text Formatting

Editing

Versioning and Backups

Automated Testing of Code

HTML/PDF® Generation

Author Communication

In this talk, we highlight one particular case study, the development of Addison Wesley's 2009 publication, "Python Web Development with Django", as well as provide some insight into other projects created using similar tools. Open source software has radically affected how applications are built today, and this phenomenon has the ability to affect the book publishing world in the same way.

In this talk I'm going to explain the main features and possibility that Panda3D gives us when we want to create a game from scratch. Then, I'll explain how the engine is structured and how it works conceptually going also through code examples and some practical operations.

It's meant for beginners who have also never worked with realtime graphics and the only prerequisite is a bit of python knoweledge including how to write a simple program and how to run it. The goal is to create a simple realtime application that take care of user input and reacts depending on it.

PostgreSQL (or “Postgres”) is an immensely powerful, incredibly extensible relational database, released under a permissive open source licence that is similar to that of CPython. PL/Python is one of PostgreSQL's standard server-side procedural languages, through which Python stored functions can be defined that can be called directly from SQL, quite seamlessly.

Play to the strengths of Postgres, by writing application business logic within the database in either Python 2 or Python 3. Enforce advanced business rules using Python, including constraints on both tables and database level datatypes (“domains”). By pushing the logic tier into the database, you can potentially greatly increasing your application's scalability by minimising the number of database roundtrips.

Includes case studies and topical coverage of PL/Python related enhancements in upcoming 9.1 release, and recent 9.0 release. The talk only assumes a very basic understanding of databases.

With the growth of AJAX and other client-side technologies many Python programmers, web-applications increasingly involve large amounts of JavaScript. Many of us find that, just to keep doing our job, we have understand JavaScript better.

This tutorial, which was also given at EuroPython 2009 and 2010, provides an introduction to JavaScript that emphasises the similarities and differences between Python and JavaScript. It has a focus on the concepts underlying JavaScript, and in particular the object model and the lack of built-in classes. It also covers quite a few JavaScript gotchas and other traps for the unwary.

The PSF recently adopted a diversity statement. While some argue that we should do this just because it's "the right thing to do", others question why we, as a technical community, should even worry about diversity. This talk addresses the diversity question, not in moralistic or ethical terms, but in pragmatic ones. Studies on creativity and productivity demonstrate the potential benefits of diversity for the Python community. Making Python the best language it can be is not merely helped by increasing diversity, but may be dependent on diversity.

Tuesday 21st June 2011

Share the lessons learned from a decade of core Python development, what worked and what didn't. Look at the development process and thinking behind some of Python's successful APIs and ones that leave something to be desired. Learn general principles for designing a good API for public consumption.

The target audience are pythoneers, pythonistas and enthusiasts who want to learn how to put together a web project in a professional way

The training aims at exploring all the necessary elements for successfully putting together a web project. It provides knowledge and best practices about a range of aspects which must be taken into account when developing for the web. At the end, the participants are expected to have a fully-working, deployed and running web project with all the basics covered where they can borrow from in future projects.

List of contents:

Basics about web servers and differences between a client program and a web service.
Architecture of a web project.
Overview of web security concepts and best practices.
Deployment techniques and tools
Development/Staging/Production environments and how to manage them successfully in a Python web project. Use of VCS and Virtualenv.
Comparison of Python web frameworks.
Testing with Python and Selenium.
Best practices for logging and backups.
Statistics and how to take advantage of them.
Benchmarking and why it is a key factor for success.
Scalability: layered architecture and Cloud Computing.
Caching structures and tools. Cache-breaking techniques.

Additionally the basics of Django are presented as it will be used as a base for the project. It is also included how to deploy in AWS using boto.

The talk will give an introduction to the concepts used in the Python Database API and relational databases in general. Connection, cursors and transactions are discussed, and their use in a typical Python database application are demonstrated. The talk will also touch upon some advanced database programming techniques and discuss best practices.

There is a huge number of messaging patterns identified in the industry [7] and a small selection of these will be presented to whet your appetite :-)

AMQP [1] is a fairly new and open messaging standard with a number of freely available open source message brokers (RabbitMQ [2], ZeroMQ [3], qpid [4] etc.) with different features and performance trade-offs.

In the course of the presentation you will be
* introduced to AMQP concepts and jargon
* introduced to available Python AMQP bindings (focussing on an asynchronous (txAMQP [5]) and a synchronous variant (most likely kombu [6]))
* shown code examples demonstrating how to use these Python bindings

The session is divided into two parts, of roughly 2 hours each. People who
are interested only in the first part, can leave the session after it.
However, the first part is a prerequisite for the second one, thus people are
not advised to join in the middle of the session

The session is meant to be highly interactive. People are invited to bring
their own laptop and try things by themselves.

Part 1: Run your application under PyPy
--------------------------------------------------

This tutorial is targeted to Python users who want to run their favorite
Python application under PyPy, and exploit the most of it. The following
topics will be covered:

- how to fix/avoid CPython implementation details (e.g., refcounting)

- general overview of how the PyPy JIT works

- how to optimize your program for the PyPy JIT

- how to view and interpret the traces produced by the JIT

- how to tweak the parameters of the JIT and the GC

- how to use existing CPython C extensions on PyPy, and fix them if
necessary

Part 2: Write your own interpreter with PyPy
-------------------------------------------------------

PyPy is not only a Python interpreter, but also a toolchain to implement
dynamic languages. This tutorial is targeted to people who want to implement
their own programming languages, or who simply want to know more about how the
PyPy JIT works internally.

The students will be given the source code for a toy language implemented in
RPython. They will learn:

- how to translate it to C using the PyPy translation toolchain

- what are the "hints" needed by the JIT generator, and how to place them

Then, they will be challenged to add the proper hints to the toy interpreter,
to get the best result with the JIT.

Python is currently at a crossroads: Python 2 has taken it from a quiet word-of-mouth language to primetime, with many companies around the world using it and an ever-increasing global marketshare of the programming world. But now comes Python 3, the first version of the language that is not backwards compatible with previous releases.

What does this mean? Are all my Python programs going to break? Will I have to rewrite everything? How much time do I have? When is Python 2 going to be EOL'd? Is the language undergoing a complete rewrite and will I even recognize it? What are the changes between Python 2 and 3 anyway? Also, the next generation is already here, as Python 3 is over two years old now. What has been ported so far, and what is its current status? Are migration plans or transition tools available? If I want to start learning Python, should I do Python 2 or Python 3? Are all Python 2 books obsolete?

With the growth of the web, how we publish is changing. Paper alone is not enough, web pages are also required. And for many web pages it is not enough to rely on the browser's print capabilities. Sphinx, used since 2008 for Python's documentation, is a creative response to this tension.

This talk will take a broader view of the problem. For example, many browsers support web fonts for display but not for printing. Mathematics on web pages is still difficult, but SVG holds out much promise not only for mathematics but for tables and other complex typeset material. Achieving accessibility remains difficult.

Based on over 15 year's experience in print and electronic publishing, this talk will survey problems and emerging solutions. Software and sites covered include LaTeX, PDF, SVG, MathJax, web-fonts, PediaPress, Sphinx and arXiv.

rdflib is a python library implementing a database with various triples back-end, parser, data serializers, SPARQL is a Python interface to extract/insert triples. We integrated it in Django reusing the database connection and exposing an ORM interface, along with full-text search on literals. This presentation shows a django-rdflib case study with a PostgreSQL backend in Brain Architecture Management System (http://brancusi1.usc.edu) - a neuroscientific project for the University of Southern California. Benefits of the flexible RDF structure will be shown, allowing researchers to insert free format data, making data public with a customizable serialization and use the powerful full-text search integrated in PostgreSQL.