Archive for June, 2011

Grandpa: Nothing gave Buttercup as much pleasure as ordering Westley around.
Buttercup: Farm boy, polish my horse’s saddle. I want to see my face shining in it by morning.
Westley: As you wish.
Grandpa: “As you wish” was all he ever said to her.
Buttercup: Farm boy, fill these with water – please.
Westley: As you wish.
-The Princess Bride

Here to serve.

That sounds fantastic when somebody says that to me. It’s a bit harder for me to say that even though I know full well that when I serve others I usually benefit greatly if not more than the one I’m serving. My focus is not about you or me but about software. Software that can be a service to others.

Wow, really!? …and that’s just scratching the surface. This is not a new idea by any means. There seems to be an ebb and flow to centralizing and decentralizing the elements that make up a given application. What I have found is that in and of themselves they are neither right nor wrong. They are what they are. It’s for a given context that a level of appropriateness comes to play. Hardware capabilities, intranet and internet speeds and accessibility, user expectations and usage patterns. These all start telling a story and create a setting in which to place an application.

Distributed applications are cheered and jeered at the same time. Sometimes by the same people. They represent an incarnation of DRY (Don’t Repeat Yourself). As such they can be awesome. “You need widgets? Call the Widget Service!” The ease in which you can get data and functionality can be very impressive. Pulling an application into a hundred remote calls can also be something else. Slow.

If you have a single application it may not make sense to go the SOA route. You can still design and build this way to think separate but deploy combined smartly making everything a local call without the over head of serialization. This pattern really works best to serve multiple applications that have shared needs or need shared data.

If the functionality is not intended to be shared with multiple applications do not make it a service. Don’t pull in consultants and fire up a major ESB project with crazy orchestration to negotiate and transform data between disparate systems when there’s only one consumer. We’ll talk more about application design another time. Right now I’m focused on services.

What is a service?

To me it’s just an application. Instead of the consumer being a user it is another application. It’s also data with behavior. Hey… that sounds like encapsulation from the OOD school. Go figure. It is. At it’s core SOA is a very Object Oriented approach to designing an application. This is why you can design this way regardless if you’re deploying as a separate service or as an integrated business unit within your application. You should be thinking this way already. We’re just gonna pull part of the application out so others can call it without:

This service should offer value. It should be strongly cohesive and fully encapsulated. The service should only do what it is named to do. A ‘beach finding’ service should not know anything about gas stations. It knows beaches. You should not have any beach logic in your application, it’s in the beach service. If you want to know where it is ask the service. If you want to know how popular it is ask the service. If you want to know how to get there… ask the mapping service, the beach service doesn’t understand directions.

I like to think of services as Legos. Very simple to use and combine. By connecting a few together I have a working application. That’s when the magic happens. The real benefit is that the service is independent of your application and has it’s own delivery and maintenance schedule. If it’s tested and solid that part of your app is already done. Don’t do it again. It’s free for everybody after the first consumer. When your library is big enough you can build a robust new application quickly and primary focus on the business needs at hand. That’s adding value!

From a deployment standpoint I like my service layer to be out of the DMZ and locked up tighter. It’s an extension of your data and is equally sensitive. Lock it up and protect it as best you can. Sure this is a hardware issue but at some point we’re going to actually deploy something on a real box and it’ll be subject to real users in the real world. I recommend doing a security audit periodically so you know where you stand.

If you’ve followed our RESTful talks you should already be thinking of making RESTful services. Keeping them stateless will allow you to load balance and scale this tier as needed with minimal effort. Be careful as you may be stateless but if you’re caching data (yeah, yet another topic to dig into) make sure you’re either caching that externally or have a way to synchronize.

I’d be remiss not to state that your RESTful service should probably return plain text JSON constructs. They’re so easy to use these days. Most languages have utilities available to generate and consume JSON so you don’t even have to sweat this one. So you’re service can now be consumed by anyone, anywhere, and… wait. No, I don’t really like that all that much. I’m going to go back to locking up my service tier. If I really want this exposed externally I recommend creating a gateway application to handle those chores. Keep your tiers intact, and in ship shape. You’ll do just fine.

Previously I’ve mentioned that we’re on a mission (part 1, part 2). We’re instituting a new platform. A central part of this initiative is a remote service layer available to all of our applications. There is a lot of talk out there about services, orchestration, and RESTfulness, but who’s talking about what is IN that service?

Me, that’s who. Buckle up as we’re about to begin part 3 of our journey!

I was originally going to discuss the dangerous phenomenon of scope creep. This is the result of the scope of a project being expanded over its lifecycle. While this is indeed something to watch for, and can lead to disaster, I’m going to go in a different direction for this entry. Recent conversations within our team brought to light a more poignant topic. One that still gets to the heart of the matter I wanted to address!

Top Down means to start with the business use case and break it down into parts and implement those parts. From a services standpoint this would mean once the application has defined what it needs appropriate services are created to satisfy those needs.

This is where you’ll see odd service names, or service names that mirror business units. You’ll also see something else if you look closer. Methods that don’t quite fit together. If you take the business use case out do the calls seem to be a jumbled mess? Do they cross specific areas or domains?

Let’s define some more terms:

Separation of Concerns
This is the practice of ensuring there is minimal to no overlap in functionality between any two systems – in our case applications and services.

Areas of Responsibility (AOR)
This is a military term that defines a geographic area that a commander has authority over. For our purposes that refers to logical concepts and models and which application or service is the commanding officer.

If the only things that makes the individual method calls make sense is the common consumption by a single application you have a warning sign. Break the key ideas into groupings. These groupings can lead you to see the real AOR’s.

Something that may not be visible, and it one of the harder ideas to grasp is what you’re missing with Top Down design. You can meet the needs as requested and be successful. You had a request and you responded. Huzzah. You consolidated logic and optimized performance. You’ve applied caching and load balancing and have a robust system. These are all incredibly valuable and worthy in their own right… but would you believe that there’s more?

This is where Bottom Up design comes in.

Step away from the originating application and become the service. You really need to step into these shoes and now look at the world from this point of view. Ask the hard questions! Should my Beach service respond with ratings? Yeah, that makes sense. It’s information that is very specific to the beach. When I want the rating I’ll most naturally want to start there to see if I can find this data.

Should it respond with directions? Should it contain it’s own address? Oh, now we’re approaching the grey area.

How many other objects have an address? How many other objects will need to find directions? These concepts are related to the beach, but are not in the beach’s domain. I recommend keeping them out.

The beach can have a URI to an address object served by the address service. This is logical. The address service can return latitude and longitude coordinates. This is also logical. And the mapping service can give me directions, even if that service is really just Google Maps. Does Google know about my Beaches? Anything about my rating scheme? Nope, it doesn’t have to and is more powerful because it doesn’t.

Fiercely protect what is and is not in the domain of each service. When you do this something grand can now occur. Emergent applications.

When your services are designed to meet the applications needs, and yet remain flexible and locked into a core definition of what that service represents. It can be used by others. Used in ways that you didn’t design. This is the critical part. If you create a Bucket service intended to be used at the beach you could have success defining how much sand it can hold. But what a missed opportunity. The service that simply defines the bucket and rules for the bucket is more powerful. If that Bucket can be used to hold water, oil, rocks… whoa. How about turning it upside down to be stacked! You didn’t think of that one, but the service should. Not. Care. Define the entity and let the use case and applications define the rest.

A system of services that can be leveraged for uses beyond their design. New concepts and ways of assembling applications that were previously impossible. This is what we’re doing. We’re starting to see the potential. We’re fighting for the services because they can’t fight for themselves.

Elwood: It’s 106 miles to Chicago, we got a full tank of gas, half a pack of cigarettes, it’s dark… and we’re wearing sunglasses.
Jake: Hit it.
-The Blues Brothers

I’m on a mission. I’ve been tasked with stewarding a new paradigm in how we build our software here at Lampo. There are many facets to this and being inspired by Kevin’s recent posts I, too, am going to try exposing the process as I go through it. I welcome you to join me on my journey and hopefully we’ll learn something along the way.

A new paradigm.

Have you introduced something new to your organization? It can be met with criticism or fanfare… or both. I’m fortunate to work at a company that believes in the book Who Moved My Cheese. This is required reading for our employees. The essence is that change is inevitable and there are ways to enjoy this process. I recommend reading this lil’ classic if you haven’t already. Some change ends up moving a lot of cheese. Changing fundamental architecture of all applications falls into this bucket.

What is this change?

Simply put we are moving to a service oriented architecture. That’s overly simplified and doesn’t encompass the whole strategy. “One Platform” is more like it. Our existing software is not bad by any means. There’s nothing fundamentally wrong with it. It’s exactly what has made us successful. If you’ve ever had the thought of changing how things are take a moment to remind yourself that your company’s success is based on the state of things today. It’s successful. The new change is untested, and high risk. Proceed with caution!

Why change?

Yes we have been successful. Our current apps have brought us to where we are. Will they get us to where we want to go? No. We’re growing and are experiencing growing pains. This is where change comes in!

We have a fair number of applications. We also have a fair number of capabilities that are similar across many of these applications and are not truly specific to any one app. This includes, but is not limited to concepts such as handling content, managing users, security, user interface and user experience. Kevin’s recent posts on Implementing Modular Web Design is talking about one of these new foundational concepts. We are very decentralized and desire to move to a centralized system.

We’re weaving SOA into our fabric because it fits in with our objectives and our end goal to give hope. Always, always, always know why you’re in business and what you are trying to achieve. This change isn’t something we’re doing because it’s popular, or a great way to impress peers. We chose a direction that fit our team and fit our mission.

Fundamental changes are risky ventures. I’ve heard them called “science projects” and anyone bringing one up was asking for trouble. I’ve known some companies that have almost sunk their business with a “major re-write”. Embracing change also means to understand what’s at risk – both good and bad.

This type of open discussion has helped form our strategies around introducing these changes. This level of change is deep. It effects our estimates, project management, developer communication, code management, testing, deployment process, and governance. Risk management needs to be ever present when every stage of the development life cycle is effected.

Be Confident!

We have an amazing team. I’m confident in my teammates, the mission, and myself. I have no doubt we will be successful. I’m not going to measure success on adhering to a spec or standard on how to implement an SOA architecture. It’s about abstracting out common elements into a reusable toolkit in order to deliver better products quickly.

Elwood: It’s got a cop motor, a 440 cubic inch plant, it’s got cop tires, cop suspensions, cop shocks. It’s a model made before catalytic converters so it’ll run good on regular gas. What do you say, is it the new Bluesmobile or what? – The Blues Brothers

We send millions of emails every year trying to help people. To help us have unique conversations with all those people simultaneously, we rely on a vendor named Exact Target. Around here we tend to push the envelope, and with very few exceptions they’ve been able to keep pace with us every step of the way.

Not long ago, we had a service interruption. We use their API to do a lot of different things, and several times over the period of a week, our API requests started failing. We quickly found and used a workaround to get by it the first few times, but when it kept happening we felt like we needed a deeper investigation. After some work, they found the issue on their end, they apologized profusely, and corrected it. Two days later, I arrive at my desk and find a package from them. It included a handwritten note, a $5 Starbucks gift card, and orange-flavored candy. (Everything they do is orange.)

I sent our contact an email thanking them for the gift and got this reply:

Tim, I’m glad you enjoyed the token of our consolation for the recent issues! Obviously candy oranges do not negate the product issues, but we hope it proves our appreciation for your understanding and solid partnership. We also appreciate the feedback and kind remarks – it always helps make a day! Please let us know how we can continue to help and support your team in the future. Best, Megan

Our team strives to super-serve people and we like to build relationships with vendors that do the same. The way Exact Target handled this situation impressed me. It’s awesome to see things done at such a high level of excellence. I’m so impressed with it that I’m showing this care package to people around our office hoping that we can learn from the example. How does your company super-serve your customers?

I’ve developed a new code-smell sensitivity to “if.” In fact, branching in general triggers an alarm as I read through code (especially if I just typed it).

Can something so fundamental to programming be a code smell? That gets me thinking that maybe it isn’t so fundamental to OOP. In high academia, I was told that “structured programming” was the greatest formalism since the slicing of bread. There are 3 required pillars to structured programming:

Sequence – A set of statements are executed in a certain order.

Selection – Statements are executed conditionally.

Iteration – A set of statements can be executed multiple times.

In light of declarative programming and OOP, all of these may be a code smell. For now, I’ll just tackle ‘selection’ and come back to the others as time permits.

Here we have a selection, a choice if you will. How much dough do we need and how long do we bake it? Well that depends on the thickness of the crust. Another way of saying that is that it depends on the type of pizza. That’s our first hint. First a detour through some smarter minds than mine.

I’m a big fan of the “Tell, don’t ask” principle and its close relative, “The Hollywood Principle.” Essentially, you don’t ask an object about its state and then act based on that state; you tell the object what you want it to do. In my example, the client code asks the pizza object about its state and makes a decision based on that state. That decision probably belongs near the state itself.

Recall the ‘type’ hint. If you have a choice based on some type of something and you’re using a language with a type system (if not, I’d really like to hear about it), use that! In my modifications I created an inner type ‘Crust’ for the ‘Pizza’ type. Now the ‘selections’ are tucked into the types.

In the client code, I just tell the object of a certain (and probably unknown-at-compile-time) type to give me the data I need.

Okay, okay, I’m actually asking for data again. With more refactoring, my client code should actually just look like this:

pizza.bake()

The behavior that uses the values for baking time and amount of dough would still be needed, but only inside the pizza object. The pizza object itself would still want to get these values from its crust inner type, perhaps composed through an inversion of control (IoC) container like Spring.

Is there a case where one needs to perform selection instead of deferring the decision to a type? That is, what are the exceptions to my argument above?