Friday, July 17, 2009

In the first half of this series, I described our experience with the latest Spring-LDAP module, concluding that an alternative mechanism would be needed due to deployment issues. I promised an alternative iBatis-over-LDAP solution, and in this article I provide demonstration code samples. With this approach in place, we can view an LDAP repository as an ordinary SQL-based database, albeit with its own variant of SQL. Other advantages include the standard iBatis benefits, in particular the ability to declaratively specify SQL queries and as such manage them from a single point of change.

What follows is a description of how I put a quick-and-dirty prototype together, in particular using Spring 3.0.

Dependencies

The key to this solution is a JDBC driver that wraps LDAP repositories, created by Octet String (and now owned by Oracle). Once we have a JDBC driver, in theory we should be able to layer iBatis over it.

The last jar listed above is the Spring 3.0 version of transactions. The 2.5.6 version is required for Spring-LDAP, and since the classes in both jars are largely (if not completely) the same, this raises a warning flag about classloading conflicts. I didn't get far enough into a test execution to hit any problems; instead I found that deploying Spring-LDAP with Spring 3.0 AOP causes a runtime error out of the gate (during Spring initialization), and at that point I bailed out on use of Spring LDAP (please refer to my first article for the details).

Finally, though it's not necessary for this example, I include the Spring-AOP libraries, if for nothing else to demonstrate that we won't see the same runtime error as happened using Spring-LDAP with Spring 3.0. You can safely exclude these for your own prototyping of this solution (but if so, just be sure to remove the AOP references in the Spring configuration file described below):

Construct the DAO for the domain object. I recommend using an interface and its implementation - because (1) using interfaces is in general a good idea for testability, polymorphism, and etc., and (2) in particular with AOP-proxied classes in Spring, I'd favor this approach because it facilitates use of JDK dynamic proxies instead of CGLIB proxies. Use of CGLIB involves these things: (1) more library dependencies, (2) final methods cannot participate in AOP, and (3) constructors are called twice due to CGLIB implementation details. Again, use of an interface is an optional step; if you exclude it, you'll need the CGLIB jars (which I haven't used - and plan to avoid using - so I can't point you to the exact set needed).

The database properties are described in a standalone file, to facilitate easy modifications to suit different environments (test, development, production). The key here is to specify the ignore_transactions parameter in the URL; adding the search_scope as a default scope is an optional convenience so that this won't be needed in every one of your SQL queries. Let's name the file database.properties in the com/mybiz/model directory; you'll of course need to adjust the parameters here to match your environment:

Following through with my convention from the first half of this series, I've bold-faced the relevant configuration pieces so you can compare how things are specified. Above, we see that the database connection parameters (LDAP URL, base node specification, principal and password) have moved from the Spring configuration (when using Spring-LDAP, as per my previous article) and into this file.

Next, layer iBatis over the database connection. We need first the SQL Map Configuration file; it doesn't need to do anything but point to your collection of SQL Maps. Let's call it SqlMapConfig.xml in the com/mybiz/model directory:

Here, we have bold-faced the LDAP attribute to be fetched, the node from which the search begins, and the search qualifier. While these query parameters plus the database connection parameters were all in one place using Spring-LDAP, here they've been split up into separate files. It's your judgment as to whether this is a good thing or not; personally, I consider these things as separate concerns, so I prefer it. However, you might prefer single point of change for all things database-related and/or all things configuration-related. More importantly, we can now use iBatis' parameterized substitution to declare a query, keeping it readably intact while varying its parameter values dynamically. With Spring-LDAP, we can likewise achieve dynamic query construction, but the model is more procedural.

In the above SQL Map, you'll notice the interesting use of an LDAP base specification instead of a table name (and again, you'll need to adjust this query to match up with your LDAP repository structure). The docs for the JDBC-LDAP bridge do mention that one can map things to an aliased table name, but for our purposes here, transparency and simplicity are more important.

The next step is to configure iBatis with Spring. I included transactional and AOP support, though these are optional for this exercise. Some explanations:

Though LDAP itself is not transactional (hence the ignore_transactions parameter above), you might be in an environment where you'll also be accessing normal relational databases, for which you'll want Spring's transactional support. Since we're now solely Spring 3.0, there is in either event no fear of classloading conflicts. However, to make this a bare-bones proof-of-concept, you can exclude the tx namespace declaration in the beans tag, the transaction manager bean and the tx:annotation-driven tag.

This Spring config file also configures in AOP solely to demonstrate that there is no longer any deployment conflict as was seen with combining Spring-LDAP and Spring-3.0-AOP; it's not essential for the iBatis-over-LDAP solution per se. As I mentioned above, if you specify AOP in this Spring configuration, you'll need the Spring-AOP runtime jars. Again, to make this a bare-bones proof-of-concept, you can exclude the aop namespace declaration in the beans tag and the aop:aspectj-autoproxy tag.

For that matter, Spring itself is not essential to layering iBatis over LDAP - I was able to remove Spring from the stack entirely and get the same successful result. However, Spring provides various benefits to an iBatis application, so I include it here for completeness (and, frankly, because I rely on Spring to help me). If you exclude it, you'll need to configure iBatis manually; I do not include those details here.

You'll notice there are no bold-faced sections in the Spring configuration file (i.e., no application-specific configurations are specified). The database connection and database query configuration information are now in separate files dedicated to these responsibilities alone, vs being embedded in what's potentially a rather large Spring configuration. Again, your take on this may vary from mine: I consider this a good thing, but you might prefer all configuration information to be in a single file.

Executing this program, with database connection information, queries, and etc. tailored to your LDAP environment, should yield a list of attribute values as expected.ConclusionUsing iBatis-over-LDAP instead of JNDI-LDAP, you can:

configure, connect to and query both RDBMS and LDAP databases in the same manner

be unencumbered by JNDI-LDAP programming models

gain all other benefits that iBatis has to offer, since the LDAP-ness has been (mostly) abstracted away. For example, fine-tuning queries for performance can be done without changing Java code.

Using iBatis-over-LDAP instead of Spring-LDAP, you gain all of the above advantages, plus these:

all SQL is established declaratively (vs being constructed programmatically), facilitating maintenance and debugging

as a result, the DAO needs fewer properties - i.e., with Spring LDAP, the DAO needs things like the LDAP base, LDAP attribute(s) to be returned and a search-filter qualifier to configure the search method. However, using iBatis, all of these things are simply part of the SQL statement specified in the SQL Map file.

Spring-LDAP has a lot to offer, including numerous features that this solution lacks, and for many applications will be very suitable. In my opinion, its strongest value-add is the compensating transactional control in an LDAP environment. If our application was more demanding than it is, I'd be motivated to dig further into the Spring-LDAP feature set and would likely be all the more impressed - in my opinion, what the Spring team has been doing is the best thing to ever happen to J2EE, and their LDAP work is likely no exception. But for our application needs - very simple CRUD + Spring 3.0 AOP - the Spring LDAP module is not the right choice. The iBatis-over-LDAP solution is clean and simple, meeting our requirements with a minimum of fuss.

Thursday, July 16, 2009

As the Spring team points out, the standard JNDI-LDAP programming model is cumbersome, at best - and there's no question that the Spring-LDAP module simplifies that problem considerably, though not as completely as some would prefer. In a 2008 article, Colin Lu describes a homegrown facade using an iBatis-style framework to address his concerns with Spring-LDAP. Our own experience with Spring-LDAP has been a mixed bag, and in this 2-part series, I'll discuss another approach that leverages iBatis-over-LDAP directly. This first article starts with the JNDI-LDAP approach, looks at the improvements available with Spring-LDAP, and describes our attempt to leverage Spring-LDAP in a Spring 3.0 environment. Since that attempt was not successful, my followup article will present an alternate solution that layers Spring-iBatis over LDAP.

JNDI-LDAP Example

Ill start with the plain-vanilla JNDI-LDAP model, and evolve it in two steps - first to Spring-LDAP, and secondly using Spring-iBatis. As presented by Sunil D. Patil in his 2007 article, an example class might look something like this (key pieces are bold-faced so you can keep track of them as we evolve the program):

As you can see, the bold-faced portions comprise the application-specific configuration, and are a small part of the overall program - most of the rest is boilerplate that can and should be factored out. That's what the Spring team has done for us.

Spring-LDAP Example

Mr. Patil does an excellent job of moving you from this verbose beginning to a lean Spring-LDAP treatment of the same program, so I won't repeat those details here. The transformation I came up with includes a test driver program, a DAO and a Spring configuration file. Here is my test driver:

Again, much cleaner than the original - perhaps even reusable. All of the application-specific configuration and most of the cruft has been factored out, thanks to Spring. Here's the Spring configuration that encapsulates application-specific details:

Note that I'm working with the latest Spring-LDAP release, 1.3.0, and as such my packages and property names are slightly different from the 2.0.1-based example in Mr. Patil's article - in particular, please note the bold-italic sections above.

The Problem

Now, I'm also working with Spring 3.0 to gain the latest-and-greatest dependency injection, AOP and iBatis integration. Since the Spring LDAP Reference document claims that "...Spring LDAP 1.3 is supported on Spring 2.0 and later", I initially assumed I'd be OK. However, when my program is deployed as a standalone, it actually fails with this:

I used an Ivy configuration to fetch dependencies for org.springframework.ldap, and in checking back with the artifacts from that retrieval, I found the missing class is available only in the 2.5.6 distribution (in particular, I needed org.springframework.transaction-2.5.6-SEC01.jar), so I would need to do some mixing-and-matching of 3.0 and 2.5.6 jars in my deployment to get this to work.

Yes, I had misgivings about this; the last thing I want is to experiment with combinations of libraries that might cause classloading or other runtime mismatches later on - a costly misadventure in a production environment. So my next step was to test drive the Spring-LDAP standalone with the 3.0 core and the 2.5.6 transaction libraries; this worked out just fine - the missing class is of course available and my simple test program succeeded without a problem. Next, I combined this program with another test driver that applies Spring AOP - but now this message appears:

Removing Spring-LDAP from the mix of course remedies my AOP program - this is no surprise since the LdapTemplate is implicated in the error message above. Checking a little deeper, I find that the 2.5.6 transaction jar has many (if not all) of the same classes as the 3.0 transaction jar that I need to support my AOP configurations. So there's a growing doubt that I will succeed in a full-blown deployment, given the obvious concern around classloading problems, let alone how quickly I encountered the error as seen above.

Analysis of the Problem

Now, I could certainly take steps at this point to Google the error message (which, in fairness, I did - but I found no solution), experiment further, query the forums, and etcetera - but the cost/benefit of further effort here is a question mark. My mission is not to leverage Spring-LDAP; my mission is to establish a clean alternative to some existing JNDI-LDAP code. If Spring-LDAP worked without a hitch in my 3.0 deployment, I would choose that direction and move on. In fairness, and impressively enough, "it just works" has been the case for everything else I've tried so far with Spring 3.0 - injection, iBatis integration, AOP and in particular AOP-based transaction management. But here are some points about our application requirements, rationalizing why I'm reluctant to put much more effort into Spring-LDAP:

Our LDAP-based use cases are dirt simple; this part of the application is not enterprise-scale. We need simple CRUD operations, nothing more. The path of least resistance here is to just wrap that CRUD in our own DAO facade and be done with it.

The application itself, however, is enterprise-scale, and we're prototyping modern technologies like Spring to facilitate a rewrite of the web tier. The legacy web tier is a mix-and-match of more than a few technologies - Struts 1.x, Dojo, (massive amounts of) JavaScript, iFrames, JSTL, JSP, etc. - and is a poster child for a "how many different ways can we do the same thing?" approach. We are motivated this time to keep the technology selection down to a bare minimum, establish a minimal set of reusable patterns, choose frameworks that "just work" and think twice before going with things that have problems out of the gate. The resistance we're meeting here with Spring-LDAP in a 3.0-AOP context gives us cause for pause.

While we could get around the deployment problems with Spring 3.0 AOP and 2.5.6 LDAP modules by isolating the LDAP component as a web service (e.g.), this begins to feel like overkill. This would add an extra network hop to retrieve information, and gives us one more point of failure (the new service) - this is probably not worth it, given #1 above, and especially since we'd be doing this only to enable use of Spring-LDAP. That's the tail wagging the dog.

Consider an Alternative Solution?

As such, I am now motivated to explore a second alternative to JNDI-LDAP, that of wrapping Spring-iBatis around a JDBC-LDAP bridge driver. But first, here are some points further validating my conclusion on #3 above, i.e. reluctance to use Spring LDAP in a separate service for our application:

Clearly Spring-LDAP is a vast improvement over the JNDI-LDAP programming model. But, while the code is cleaner and simpler, it'sjust not that much code to worry about for our application (see #1 above).

Spring-LDAP provides the ability to execute some business logic either before or after a search (but, we can also do this using regular Spring AOP, if I understand this feature correctly).

We can optionally implement a method in the Spring-LDAP inner class that converts LDAP object into custom Java objects (but, we can also do this using an iBatis approach).

Spring-LDAP provides the ability to create dynamic filters (but, this obscures the query being constructed since it's done in Java code. An iBatis approach simply externalizes the SQL in one place, supporting dynamic queries using parameterized templating).

Likewise, basic CRUD idioms in Spring-LDAP are available; but these are also constructed procedurally in java code. Again, an iBatis approach is our preference.

Spring-LDAP provides simplified "dynamic authentication" (i.e. an arbitrary user logging in to the LDAP system, as opposed to an implicitly declared principal as done in the example). However, this is not one of our requirements - the use of a single declaratively specified principal (via Spring configuration) is all we need.

Spring-LDAP provides "compensating transaction support" - which is a solid improvement over the transaction-impaired LDAP environment - but we do not need this. From the Spring-LDAP docs: "The client side transaction support will add some overhead in addition to the work required by the original operations. While this overhead should not be something to worry about in most cases, if your application will not perform several LDAP operations within the same transaction (e.g. a modifyAttributes followed by a rebind), or if transaction synchronization with a JDBC data source is not required (see below) there will be nothing to gain by using the LDAP transaction support."

The list of Spring-LDAP features goes on, but for our application needs, none are compelling enough to warrant deploying it as a separate service (which for now is the only solution we have to get around the deployment issue with 3.0 AOP). In my next post, I'll present our solution that layers Spring-iBatis over the JDBC-LDAP bridge.

Thursday, July 9, 2009

This is the next in a series of tips, insights and recommendations around Spring 3.0. This time, I offer our experiences to-date around use of Spring's ORM facility, or more accurately its integration with iBatis (which is not, strictly speaking, an OR mapper).

iBatis - General

Use iBatis "implicit result mapping", i.e. adding an alias to each column in the select field list, to match up with Java object properties so iBatis will automatically map them for you (and as such, there's no need for resultMap constructs). For example, the following maps database column names (that might need to follow database naming conventions) to the property names of the corresponding Java object:

Factor connection details out of the iBatis SQL map configuration and into Spring beans configuration instead, to simplify the iBatis configuration (since these details often vary between environments (eg development, test, production)) and to consolidate application-wide configuration information into a single place. Put these connection details into e.g. an Apache DBCP DataSource bean. Use the Spring PropertyPlaceholderConfigurer bean to support the externalization.

The SQL map client will automagically read in the SQL map configuration information. Note that with the Spring XML beans file under ./WEB-INF, you must use the "classpath:" prefix to reference the iBatis SQL map configuration file.

Alternately, one can use the Spring SQL map client class instead of the iBatis version (see section 14.5 in the the Spring Reference Doc). The primary advantage here, as far as I can tell, is that formerly checked SQLExceptions are now wrapped in unchecked exceptions. However, I'd consider this optional, and for that matter undesirable in certain situations - e.g., in a RESTful web service where specific checked exceptions are caught so they can be mapped to corresponding HTTP response codes (and in just such a situation, I've chosen the iBatis SQL map client instead of the Spring version). The debate around documenting checked (and unchecked) exceptions as an important part of specifying program behavior is beyond the scope of this post; bottom line, sometimes wrapping exceptions that you can't do anything about with unchecked ones is useful, but sometimes your methods will intentionally throw their own checked exceptions that clients are meant to handle explicitly.

iBatis-Spring: Transactions

Use the Spring TransactionManager and annotation-driven transaction management to establish simple, declarative transactional behavior. Use the JDBC data source transaction manager unless you expect to be managing multiple resources in "global" transactions or want the application server to manager transactions (e.g. to take advantage of advanced features like transaction suspension, etc.), in which case use the JTA transaction manager. Downside of JTA transactions is lack of ability to test outside the container (since it requires JNDI and possibly other container functionality).You'll need to provide the AOP autoproxy tag and associated namespace attributes to get declarative transactions to work.

Annotate DAO methods as appropriate (i.e. non-read-only unless you care about non-repeatable reads, etc.) with @Transactional. This tells Spring that a transaction is required and to start one if it's not already executing within a transactional context (because the default propagation is REQUIRED). That method body will execute as one transactional unit and will auto-commit if it completes successfully. If a runtime exception is thrown, the transaction will be rolled back. This becomes increasingly important as your java method executes more than one operation (multiple updates, delete + insert, etc.)

Use of Spring AOP means use of proxies; so, if you want to use JDK dynamic proxying instead of CGLIB proxying, your DAOs should implement an interface that specifies its public methods (as implied above with MyDao implements DaoBase). Dynamic proxying is recommended for these reasons: with CGLIB,

While using interfaces supports annotating methods in the interface only - i.e. you don't need to maintain these in the implementation - this is considered risky by the Spring team; so, at a minimum annotate concrete classes, and optionally annotate interfaces also (the former for the functionality, optionally the latter for documentation).

How do you know you're getting transactional behavior? You can monitor transaction activity by turning on a log4j appender to debug for TransactionInterceptor; then, add and remove @Transactional to various methods in your interface to observe the logging statements. In a log4j.properties file:

While transaction behaviors can be configured programmatically, this couples the code to Spring Framework and as such is not preferred.

Beware of self-invocation where you expect transactional behavior - unless there's already a transactional context, a new one will not be created. Consider the use of so-called AspectJ mode if you expect self-invocations to be wrapped with transactions as well (set tx:annotation-driven mode attribute to "proxy" - but this will require use of CGLIB libraries).

So we need a factory mechanism to decide at runtime which way to go - assuming of course we want to test our web classes, which of course we do. After all, one of Spring's advantages is loose coupling to facilitate testability.

Now, while it's true that I could probably figure out a way (aka a hack) to convince a test class to use a web-aware Spring context, and not have to bother with this factory nonsense, I'd rather spend my time figuring out a way to do things that have good design sensibilities.

First, use a ServletContextListener to cache a web-aware implementation:

After presenting myself as someone who was versed in both database and server-side Java, my interviewer asked a great question: what are some similarities in the design considerations with both data modeling and software design? Though I stumbled a bit on my answer, two good things came of it: (1) I did get the job and (2) I've thought about the answers to that question ever since.

That's actually one of the better parts of stumbling over interview questions - it motivates you to circle back to the topic and nail it for future reference. In the bigger picture, that's one of the better things about interviewing frequently (whether you need a job or not) - it gives you reminders around your knowledge gaps. But, I digress.

Today I came across two posts that, taken together, reminded me of one of the first database-software-similarity answers I came up with. The first is around primary keys, and the second is about responsibility-driven design. The primary-key post advocates maintaining separate tables for separate concerns, and the responsibility-focus post advocates maximizing cohesion in your classes with the Single Responsibility Principle. I'd suggest that, in principle, we're talking about the same thing in both cases.

The primary-key post concludes with this: "Database design skills begin with identifying the kinds of things that must be tracked, putting each into a table, and assigning the primary keys to those tables." I'd suggest an analogous takeaway for the software developer: add equals and hashCode implementations to those application classes that call for unique identification at runtime - in particular for use in Java collections. Given this, perhaps the well-worn analogy between database tables and Java classes is not quite right - maybe it's table == collection, where the objects in the collection implement hashCode. And, if so, then hashCode == primary key.

Note that I'm not suggesting that an object's hashCode value should be used as the primary key value in any table that represents it, nor am I suggesting there's necessarily a one-to-one correspondence between classes and tables. Those kinds of discussions are moving more towards implementation concerns; I'm speaking strictly from a design perspective.

I'll follow up with more thoughts about the similarities between database and software design, plus some articles that clarify some misconceptions I've seen around normalization and other database topics.

Welcome to the Perimeter Sweep Blog

My blog is largely intended to be a placeholder for topics involving software development - architecture, technology drill-downs, best practices, various solutions, workarounds, gotchas and the like - things that will remind me what I've learned over time. If it helps you out also - all the better.

Subscribe To This Blog

About Me

I'm a Senior Software Engineer, an avid runner, and formerly a professional musician...currently the proud father of a super-tyke, raising two Siberian Huskies and married to my best friend. Life is good.