Spring Python

Spring Python 1.1

Back in 2004, a new revolution was brewing in the Java industry: the Spring Framework. It challenged the concept of monolithic, one-size-fits-all application servers by offering pragmatic solutions without heavy handed requirements. Contrary to previous frameworks, Spring did not require all-or-nothing adoption. Instead, it offered a practical approach by providing convenient abstractions over commonly used patterns. It also embraced the concept of dependency injection, allowing non-intrusive adoption of powerful options. The Spring Framework doesn’t hinge on the technology that it is coded in. The concepts and methods, known as the Spring way, work anywhere. In 2006, I created the Spring Python project. This was to combine the non-intrusive, powerful concepts of Spring with the nimble, dynamic flexibility of Python.
This book will guide you through the building blocks of Spring Python with the aim of giving you the key to build better Python applications. The first seven chapters show you how to download and get started with Spring Python. Each of these chapters introduces a different module you can use to build applications with. The eighth chapter shows how to use the modules in concert to build a more powerful yet easier to maintain application. The ninth chapter introduces a command-line utility used to rapidly create applications. The final chapter shows how to integrate Python and Java together, quickly and easily. I have written this book with the intention that developers can discover the scope and beauty of the Spring way and how it can enrich development, without complicating it. I often look at software development as more craftsmanship than science, and believe Spring Python can make for a valuable tool in any developer’s tool box.

What This Book Covers

Chapter 1: Getting Started with Spring Python gives you a quick glance at various features of Spring Python, followed by information on how to download and install it.

Chapter 2: The Heart of Spring Python—Inversion of Control introduces you to Spring Python’s core container which is reused through the rest of the book to empower testing and non-intrusive features.

Chapter 4: Easily Writing SQL Queries with Spring Python shows you how to rapidly write pure SQL queries without dealing with mind-numbing boilerplate code. It also shows how Spring Python works nicely with ORM-based persistence.

Chapter 5: Adding Integrity to your Data Access with Transactions shows how to seamlessly wrap business code with SQL transactions.

Chapter 6: Securing your Application with Spring Python shows how to wrap web applications with flexible authentication and authorization services, while easily supporting policy changes. It also shows how to code custom extensions to integrate with security systems not currently supported.Chapter 7: Scaling your Application Across Nodes with Spring Python’s Remoting shows how Spring Python nicely integrates with Pyro, the RPC library for Python. This powerful combination will allow you to non-intrusively develop simple apps on your desktop and retool them for multi-node production systemsChapter 8: Case Study I—Integrating Spring Python with your Web Application invites you to use all of Spring Python’s components to build a strong, secured banking application. This shows how the various parts of Spring Python work together in concert to empower you the developer.Chapter 9: Creating Skeleton Apps with Coily introduces Spring Python’s plugin-based Coily tool to rapidly create web applications.Chapter 10: Case Study II – Integration Spring Python with your Java Application ends the book by showing how easy it is to link Java and Python components together using Jython. This allows polyglot programmers to mix together their favorite services.

Spring Framework Articles

The Heart of Spring Python—Inversion of Control

Many software teams seek ways to improve productivity without sacrificing quality. This has led to an increase in automated testing and, in turn, sparked an ugh a company representative or frominterest in making applications more testable. Manugh a company representative or fromy automated test frameworks promote isolating objects by swapping collaborating objects with test doubles to provide canned responses. This generates a need for developers to plugin test doubles with as little change as possible. On some occasions, managers have trimmed budgets by cutting back on development and integration hardware. This leaves developers with the quandary of having to develop on a smaller footprint than their production environment. Software developers need the ability to switch between different test configurations and deployment scenarios. Software products sold to customers often need the ability to make flexible adjustments on site, whether its through a company representative or from the customer. In this chapter, we will explore how Spring Python’s Inversion of Control (IoC) container meets all these needs by making it possible to move the creation of critical objects into a container definition, allowing fl exible adjustment without impacting the core business code.
This chapter will cover:

How IoC containers make it easy to isolate objects under test in order to work with automated testing.

A detailed explanation of IoC with high level diagrams. We will learn how Dependency Injection is a type of IoC and uses the Hollywood principle of Don’t call us, we’ll call you!

Spring Python’s answer to the community debate of IoC in dynamic languages.

How mixing Python and Java components together is easy, can be done many ways, and provides a good way for developers to choose what best fits their needs.

Swapping production code with test doubles

As developers, we need the ability to exchange production objects with mocks and stubs.

Mocks and stubs are two ways of generating canned responses used
to test our code. Mocks are used to generate canned method calls.Stubs are used to generate canned data. Both of these are useful
tools to simulate external components our code must work with.
For more details, see http://martinfowler.com/articles/
mocksArentStubs.html

In this example, we are going to explore how an IoC container makes it easy to exchange production objects with mock instances. We start by developing a simple service along with some automated testing for a wikiengine.

First of all, let’s define a simple service for our wikiengine. The following WikiService has a function that looks into a MySQL database and returns the number of hits for a page as well as the ratio of reads per edit.

In this situation, WikiService directly defines data_access to use MySqlDataAccess. The statistics method calls into MySqlDataAccess to fetch some information, and returns a tuple containing the number of hits against the page as well as the ratio of reads per edit.

It is important to point out that our current version of WikiService has a strong dependency on MySQL as implied by directly creating an instance of MySqlDataAccess.

The startup code creates an instance of WikiService, which in turn creates an instance of MySqlDataAccess. We create an instance of our main web application component, WikiWebApp, and start it up, giving it a handle on our service.

We have not defined WikiWebApp or MySqlDataAccess.
Instead, we will be focusing on the functionality WikiService
provides, and on how to isolate and test it.

Let’s look more closely at testing the code we’ve written. A good test case would involve exercising statistics. Considering that it uses hits and edits from MySqlDataAccess, it is directly dependent on connecting to a live MySQL database. One option for testing would be pre-loading the database with fixed content. However, the cost of set up and tear down can quickly scale out of control. If you have multiple developers on your team, you also don’t want the contention where one developer is setting up while another is tearing down the same tables.

Since we do not want to continuously setup and tear down live database tables, we are going to code a stubbed out version of the data access object instead.

This solution nicely removes the need to deal with the MySQL database by plugging in a stubbed out version of our data access layer. These dependencies are depicted in the following diagram.
WikiServiceTestCase depends on WikiService and StubDataAccess, as well as an inherited dependency to MySqlDataAccess. Any change to any of these dependencies could impact our test code.
If we build a huge test suite involving hundreds of test methods, all using this pattern of instantiating WikiService and then overriding data_access with a test double, we have set ourselves up with a big risk. For example, WikiService or StubDataAccess could have some new initializing arguments added. If we later need to change something about this pattern of creating our testable WikiService, we may have to update every test method! We would need to make every change right. This is where Spring Python’s Inversion of Control can help.

More about Inversion of Control

Before diving into our solution, let’s look a little deeper into the meaning of Inversion of Control. Inversion of Control is a paradigm where we alter the way we create objects. In simple object creation scenarios, if we have modules X and Y, and X needs an instance of Y, we would let X directly create it. This introduces a direct dependency between X and Y, as shown in the following diagram.
It ‘s dependent because any changes to Y may impact X and incur required updates. This dependency isn’t just between X and Y. X is now dependent on Y and all of Y’s dependencies (as shown below). Any updates to Y or any of its dependencies run the risk of impacting X.
With Inversion of Control, we break this potential risk of impacts from Y and its dependencies to X by delegating creation of instances of Y to a separate container, as shown below.
This shifts the dependency between X and Y over to the container, reducing the coupling.

It’s important to note that the dependency hasn’t been entirely eliminated.
Because X is still given an instance of Y, there is still some dependency
on its API, but the fact that Y and none of its dependencies have to be
imported into X makes X a smaller, more manageable block of code.

We saw in our wikiengine code how WikiService was dependent on MySqlDataAccess. MySqlDataAccess is dependent on MySQLdb, a python library for communicating with MySQL. Using the container and coding against a well defined interface opens up the opportunity to change what version of data access is being injected into WikiService.
As we continue our example in the next section, we’ll see how IoC can help us make management of test code easier, reducing long term maintenance costs.

Adding Inversion of Control to our application

We a lready took the first step in reducing maintenance costs by eliminating our dependency on MySQL by using a test stub. However the mechanism we used incurred a great risk due to violation of the DRY (Don’t Repeat Yourself) principle. For our current problem, we want Spring Python to manage the creation of WikiService in a way that allows us to make changes in one place, so we don’t have to edit every test method. To do this, we will define an IoC container and let it handle creating the objects for us. The following code shows a simple container definition.

First of all, let’s create an IoC container using Spring Python and have it create our instance of WikiService.

In this version of our wikiweb application, the service object was obtained by asking the container for wiki_service. Spring Python ispatches this request to WikiProductionAppConfig where it invokes the wiki_service() method. It is now up to the container to create an instance of WikiService and return it back to us. The dependencies are shown in the following diagram:
At this intermediate stage, please note that WikiService is still dependent on MySqlDataAccess. We have only modified how our application gets a copy of WikiService. In later steps, we will completely remove this dependency to MySqlDataAccess.

Let’s see what our test case looks like when we use the IoC container to retrieve WikiService.

With this change, we are now getting WikiService from the container, and then overriding it with the StubDataAccess. You may wonder “what was the point of that?” The key point is that we shifted creation of our testable wiki_service object to the container. To complete the transition, we need to remove all dependency of MySqlDataAccess from WikiService. Before we do that, let’s discuss Dependency Injection.

Dependency Injection a.k.a. the Hollywood principle

So fa r, we have managed to delegate creation of our WikiService object to Spring Python’s Inversion of Control container. However, WikiService still has a hard-coded dependency to MySqlDataAccess. The nature of IoC is to push object creation into a 3rd party location. Up to this point, we have been using the term Inversion of Control. The way that Spring Python implements IoC, is through the mechanism of Dependency Injection. Dependency Injection, or DI, is where dependencies are pushed into objects through either initialization code or by letting the container directly assign attributes. This is sometimes described by the Hollywood cliché of Don’t call us, we’ll call you. It means that the object which needs a certain dependency shouldn’t make it directly, but instead wait on the external container to provide it when needed.

Because Spring Python only using Dependency Injection, the terms,
Inversion of Control, IoC, Dependency Injection, and DI may be used
interchangeably throughout this book to communicate the same idea.

The following version of WikiService shows a complete removal of dependence on MySqlDataAccess.

We have added a definition for MySqlDataAccess so that we can inject this into the instance of WikiService when it’s created. Our dependency diagram now shows a complete break of dependency between WikiService and MySqlDataAccess.
As described earlier, this is akin to the Hollywood mantra Don’t call us, we’ll call you. It opens up our code to the possibility of having any variation injected, giving us greater fl exibility, without having to rewrite other parts of the system.

Adding Inversion of Control to our test

With t his adjustment to WikiService and WikiProductionAppConfig, we can now code an alternative way of setting up our test case.

First, we subclass the production container configuration to create a test version. Then we override the data_access object so that it returns our stub alternative.

Using this technique, we inherit all the production definitions. Then we simply override the parts needed for our test situation. In this case, we return a slightly modified version of WikiService that is suitable for our testing needs.

Now, we can alter our test suite to fetch WikiService just like the production main code, except using our alternative container.

The dependencies (or lack of!) between WikiService and StubDataAccess are shown in following diagram:
This makes it easy to plug in our alternative StubDataAccess. An important thing to note is how our test code is no longer overriding the data_access attribute, or doing any other special steps when creating WikiService. Instead, this creation logic has been totally turned over to the IoC container. We can now easily write as many tests as we want with no risk of changes to WikiService or data_access, provided we continue to rely on the IoC container to handle object creation. We just visit the blueprints of WikiTestAppConfig to make any future alterations.
In conclusion of this example, the following diagram shows our current objects as well as potential enhancement to our system. In this possible situation, we have defined multiple data access components. To find out which one is being injected into WikiService, we simply look at the relevant container’s definition, whether its production or a particular test scenario. Our current system may use MySQL, but if a new customer wanted to use PostGreSQL, we simply create another variation of our container, and inject the alternate data access object. The same can be said for Sqlite. And all of this can be done with no impact to WikiService.

Spring Framework Articles

Container versus Context

You may have noticed in this chapter’s example how we keep referring to the code that contains our object definitions as a container, and yet, the class used to create it is called ApplicationContext. This chapter will clarify the differences between a context and a container. Spring Python has a base class called ObjectContainer, which is responsible for managing the object definitions, creating instances of objects based on these definitions, and is largely responsible for the functionality that we have looked at so far. Any instance of this class would be a container in object oriented terms. In fact, we could substitute that class in our previous example everywhere we see ApplicationContext, and it would act mostly the same.
ObjectContainer has a subclass called ApplicationContext. An instance of ApplicationContext is a context. However, from an OOP perspective, a context is a containe r. So generally referring to instances of either one as an IoC container is common practice. An ApplicationContext has some differences in behavior from an ObjectContainer as well as extra features involving life cycle management and object management. These differences will first be shown in the following table of features and will be covered in more detail later.

Lazy objects

Earlier I said that most of the behavior in this chapter’s example would be the same if we had replaced ApplicationContext with ObjectContainer. One difference is that ObjectContainer doesn’t instantiate any objects when the container is first created. Instead, it waits until the object is requested to actually create it. ApplicationContext automatically instantiates all objects immediately when the container is started.

In the examples so far, this difference would have little effect, because
right after creating a container, we request the principle object. This won’t
be the case with some of the examples later in this book.

It is possible to override this by setting lazy_init to True in the object’s definition. This will delay object creation until first request.

One reason for using this is if we had some object that was only needed on certain occasions and could incur a lot of overhead when created. Setting lazy_init to True would defer its creation, making it behave as an on-demand service. This override works for both ObjectContainer and ApplicationContext.

Scoped objects

Another key duty of the container is to also manage the scope of objects. This means at what time that objects are created, where the instances are stored, and how long before they are destroyed. Spring Python currently supports two scopes: singleton and prototype. A singleton-scoped object is cached in the container until shutdown. A prototype-scoped object is never stored, thus requiring the object factory to create a new instance every time the object is requested from the container.

When a singleton object includes a prototype object as one of its
initializing properties, it is important to realize that the prototype
isn’t recreated every time the cached singleton is used.

The default policy for the container is to make everything singleton. The scope for each object can be individually overridden as shown below.

Each request for data_access will yield a different instance. This happens whether the request is from outside the container, or from another container object injecting data_access. This override works for both ObjectContainer and ApplicationContext.

Property driven objects

ApplicationContext will invoke after_properties_set() on any object that has this method, after it has been created and all container-defined properties have been set. Here are some examples of how this can be useful:

Including validation logic in class definitions. If some properties are optional and others are not (or certain combinations of properties need to be set), this is our opportunity to define it, and let the container validate that an object was defined correctly.

Starting up background services. For example, PyroServiceExporter launches a daemon thread in the background after all properties are set.

Post processor objects

ApplicationContext will search for post processors. These are classes that manipulate other objects in the container. They extend Spring Python’s ObjectPostProcessor class.

When the context starts up, all non-post processors are fed one-at-a-time to each post processor twice: before the initialization and after. By default, the object is passed through with no change. By coding a custom post processor, we can override either stage, and apply any changes we wish. This can include altering the object itself, or substituting a different object.

This post processor looks at an object’s name, and if its name ends with Service, returns a proxy that exports the object as a Pyro service, using the object’s name as part of the URL. Otherwise, it simply returns the object. This example shows how we can easily develop a convention-over-configuration client-server mechanism, simply by using Spring Python’s IoC container and some custom post processors.

Context aware objects

We can define classes that get a copy of the ApplicationContext injected into the attribute app_context by extending ApplicationContextAware.

This is only recommended for objects that need to manipulate the context
in some fashion. It is not a good practice to use this hook to simply fetch
objects, because it ties our code directly to the container.

Instead, use the principles of Dependency Injection as shown earlier in this chapter.

Spring Framework Articles

Debate about IoC in dynamic languages

There is a powerful debate in the technical community about whether or not IoC is of any value in dynamic languages such as Python, Ruby, and others. This section addresses several of the arguments, and shows how IoC definitely has value in any language. If you are already sold on the idea of IoC, you can skip this section and learn about more options to configure IoC definitions provided by Spring Python. One argument is that IoC was invented as a paradigm to facilitate static languages, particularly Java. In the dynamic world, where passing around objects with handles is much more fl uid, it is understandable that IoC may not solve as many problems as it does for static languages such as Java. While it is true that many Java developers that engaged in using IoC frameworks saw considerable leverage in productivity, it is also credible to not leave dynamic languages out of this. The strongest principle that transcends the dynamic/static divide and is handled by IoC is DRY (Don’t Repeat Yourself). By defining the creation of an object in one place, and being able to use that definition repeatedly is very useful, and makes it easy to adjust how the object is created.
One keen Ruby developer went through several phases of developing a Ruby IoC solution. The first round involved static configuration files very similar to the XML option. Later on, he realized that coding a Ruby DSL was simpler, easier to read, and more of the Ruby way. Finally, he realized that using Ruby directly in his code without the framework was actually easier and more salient. In finishing his blog article, he alludes to the fact that he uses DI every day. He just doesn’t need a framework to do it.
In response, I am the first person to say that not every object created in a system needs to be served up by an IoC container; just the critical ones that would benefit from being swapped with test doubles, environmental changes, and wiring up different configurations for different customers. For example, coding small scripts and simple apps may actually not require any IoC at all. However, as our apps grow, the need to manage dependencies and object creation grows, and in those situations, Spring Python’s IoC container can meet those needs. IoC also provides easy solutions for situations that need remoting, transactions, security, and other. Opponents to dynamic IoC solutions have suggested that these solutions are simple port jobs and are really more for people not familiar with the dynamic language environment. In essence, people that like IoC when coding statically with C# or Java, want the same warm comforts in Python. While Spring Python has utilized some of the same class names as Spring Java, it is by no means a simple port job. Spring Python has ported many of the concepts but not the code. One of the best examples of this is Spring Python’s security module. The architecture of Spring Security (originally known as AcegiSecurity) has become a de facto security standard in the Java industry. Their architectural concepts involve using a common set of interfaces to link into countless security systems such as database, X.509, LDAP, CAS, kerberos, OpenID, OpenSSO, and more, without having to get tied down to a specific API. Instead, the concepts of authentication and authorization have been abstracted to the right level, such that it is easy to add security after a system has been developed without major rework. The credit for this strongly resides in the power of IoC and its ability to wrap this critical service around already written business logic. It would be foolish to pass up this powerful security architecture on the basis of simple static versus. dynamic arguments. Another argument tends to be based on Spring Java and its historical use of XML for configuration files. While XML is commonly used for many Java-based frameworks, the Python community tends to avoid XML unless customer requirements dictate otherwise. Some developers have criticized Spring Python for being an XML-based container. However, this book has already shown pure Python container definitions. Developers don’t have to use XML at all to harness the power of Inversion of
Control. By using Python directly for container definitions, the following powerful options are available:

We can code a switchable context where the container reads the hostname and deduces whether it is running in development, test, or production environment

We can code a clustered context where we vary object configurations based on which side of a cluster we are running

We can write code that reads extra properties from an external source, like an Apache web server’s httpd.conf file, allowing us to stay nicely integrated

We can look up object properties in a database

Frankly, the options are limitless, given the power of Python and the added utilities Spring Python provides

Spring Framework Articles

Migrating a Spring Java application to Python

In the previous section, I made the point that Spring Python doesn’t require us to use XML. However, there is support for XML along with other formats we haven’t looked at yet. Other formats are provided to make it easier for Spring Java developers to migrate to Python using what they already know.
Spring Python has two file parsers that read XML files containing object definitions. SpringJavaConfig reads Spring Java XML configuration files, and XMLConfig reads a format similar to that, but uniquely defined to offer Python-oriented options, such as support for dictionaries, tuples, sets, and frozen sets.

SpringJavaConfig does not support all the features found
in Spring Java’s XML specifications, including the specialized
namespaces, Spring Expression Language, nor properties. At this
point in time, it is meant to be convenient tool to demonstrate simple
integration between Python and Java.

Let’s explore the steps taken if our wikiengine software had originally been written in Java, and we are exploring the option to migrate to Python. To make this as seamless as possible, we will run the Python bits in Jython.

Jython is a Python compiler written in Java and designed to run
on the JVM. Jython scripts can access both Python and Java-based
libraries. However, Jython can not access Python extensions coded
in C. Spring Python 1.1 runs on Jython 2.5.1. Information about
download, installation, and other documentation of Jython can be
found at http://jython.org.

Let’s assume that the Java code has a well defined interface for data access.

Thanks to Jython’s ability to transparently create both Python and Java objects, our Spring Python container can parse the object definitions in javaBeans.xml.

As our development cycle continues, we find time to port MySqlDataAccess to Python.

After doing so, we decide that we want to take advantage of more of Python’s features, like tuples, sets, and frozen sets. The Spring Java format doesn’t support these. Let’s convert javaBeans.xml to Spring Python’s XML format and save it in production.xml.

Having migrated this much, we decide to start coding some tests. We want to use the style shown at the beginning of this chapter where we swap out MySqlDataAccess with StubDataAccess using an alternative configuration. To avoid changing production.xml, we create an alternate configuration file test.xml.

The order of filenames is important. First, a list of definitions is read from production.xml. Next, more definitions are read from test.xml. Any new definitions with the same name as previous ones will overwrite the previous definitions. In our case, we overwrite data_access with a reference to our stub object.

After all the success we have had with the Python platform, we decide to migrate the container definitions to pure Python. First, let’s convert production.xml to WikiProductionAppConfig.

We simply replace XMLConfig(“production.xml”) with WikiProductionAppConfig(). ApplicationContext comfortably accepts any combination of definitions stored in different formats. Spring Python’s fl exible container gives us a fine grained control over how we store all our definitions. We could continue and easily replace test.xml with something like WikiTestAppConfig written in pure Python.
This demonstrates how easy it is to convert things a piece at a time. We could have stopped earlier based on what we wanted to keep in Java and what we wanted to move to Python. If we move the rest of the Java parts to Python solutions, then we may opt to use either Python or Jython. The point is that XML support makes it easy to move from Java to Python with minimal impact.

It is important to realize that just because we were starting with
a Spring Java application doesn’t mean this technique is only for
Spring-based Java projects. In the previous example, we quickly
moved past javaBeans.xml and wired the Java components
using Spring Python’s XML format.

Summary

In this chapter we explored swapping test doubles using Inversion of Control (IoC). This is not the only use case for IoC. Being able to easily reconfigure things with no impact to the core business logic is of incredible value. IoC allows us to move object creation to a centralized location. This empowers our applications to be easier to inject alternatives. This makes things such as testing, remoting, transactions, and security much easier. Spring Python supports many formats including pure Python, Spring Python XML, and Spring Java XML. This gives users a wide range of choices based on their needs. By working with Jython, it is much easier to mix together Python and Java code together.
Exchanging one object definition with another constitutes a useful service. Throughout this book, other services will be explored. Just remember, at the heart of those services is Spring Python’s IoC container. In this chapter, we learned:

IoC containers make it easy to isolate objects under test by swapping production collaborators with test doubles

Detailed definitions of IoC and Dependency Injection

IoC containers positively impact our application by separating creation of objects from their usage, and this helps modularize our applications

The Hollywood principle of Dependency Injection means that our objects don’t create their collaborators, but instead wait for the container to create them and inject them into our object

Spring Python has an ObjectContainer with the basic parts of an IoC container. This container handles dependency resolution, setting properties, and lazy initialiation

Spring Python has an ApplicationContext, that subclasses the container, and provides extension points for developers to manipulate creation of objects

While there may be debate about using IoC in dynamic languages, the benefits of delegating object creation to a separate container is not confined to just static languages

By using Spring Python’s container with Jython, it is easy to mix Python and Java components together. This tactic is even more productive for applications that were already built with Spring Java

In the next chapter, we will look at adding services to existing code using Aspect Oriented Programming (AOP).

About Krishna Srinivasan

He is Founder and Chief Editor of JavaBeat. He has more than 8+ years of experience on developing Web applications. He writes about Spring, DOJO, JSF, Hibernate and many other emerging technologies in this blog.