Aleksandar Seovic's Coherence Blog

April 07, 2010

Only a recent comment on one of my previous posts made me realize that it's been more than 6 months since I posted anything here. There are many reasons for that, but I guess they all boil down to one simple fact -- I just got too busy.

The good news is that the book is finally out, so I should have more spare time to devote to blogging.

Back in February I did a podcast about a book with Cameron Purdy for the Oracle Author Podcasts series. One question Cam asked during the interview was "why is Coherence such an important part of scalable architectures", and my answer was basically because "it is the right tool for the job". I'd like to expand a bit on that answer here.

The part of a typical application that is most difficult to scale is a data layer. Scaling web servers out is trivial -- you simply add more of them behind the load balancer. Because HTTP is naturally a stateless protocol, HTTP requests can be easily balanced across many machines.

The situation is similar with the application servers, but with an important difference -- in order to be able to scale application servers out, we need to make middle tier services stateless. This is not necessarily a bad thing, but it can lead to both performance and scalability problems if done incorrectly.

Unfortunately, very few services are truly stateless in real life. What is usually the case is that they need some data to process, so the common approach is for a service to load data from the data layer, process it, and persist the result back in the data layer. That means that in order to make application services stateless, we need to push all the state (even the transient state, such as HTTP sessions) to the data layer. This puts additional load on the data layer -- the more web and application servers we have, the harder our data layer will have to work to satisfy incoming requests.

The problem is that most applications use RDBMS as a data layer, and relational databases are both difficult and expensive to scale. When you use master-slave replication, clustering or sharding to scale your database, you significantly increase both the complexity and the cost of the system. Even if you use an open source database, such as MySql, you will have additional hardware and administration costs. In the case of sharding, you will also increase development cost quite a bit.

The bottom line is that relational databases were simply not designed for scale out. Even though we have come up with different solutions to scale them, all of them feel like a kludge to a certain extent, and have some fairly significant limitations.

On the other hand, Coherence and other in-memory data grids were designed with scalability as a primary goal. In the case of Coherence specifically, it is trivial to scale the cluster out by adding more machines. Coherence will dynamically repartition the cluster, which will automatically reduce both the data volume and the processing load across the cluster.

The net result is that in-memory data grids allow you to put state back into the application. This can significantly reduce the load on a database -- it is not uncommon to see database load drop 80% or more after Coherence is introduced into the architecture. They also allow you to keep transient data, such as HTTP sessions, only within the grid, reducing the load on a database even further.

I have covered scalability, as well as performance and availability, in a lot more detail in the first chapter of the book, which can be downloaded for free, but the bottom line is that in-memory data grids can help you achieve all three when used properly within the architecture, and will allow you to use relational database as it was meant to be used -- for data persistence and complex relational queries.

September 27, 2009

I'll be doing a presentation on scalability and how Coherence can be used to improve it for the Tampa JUG on Tuesday, Sep 29 (two days from now).

On Thursday, Oct 1 I will do the talk on Coherence Tools, an open source project seeded by the code I wrote for the book, at the NY Coherence SIG. I will do the same talk at the Bay Area Coherence SIG the following week (Thursday, Oct 8) and will be in SF through the 14th, attending Oracle OpenWorld.

The week after I'll be in New Orleans for SpringOne (Oct 19-22). It's been a while since I've done anything on Spring, but it will be a lot of fun to see everyone and learn what's new. Plus, they have finally ;) ported the SpEL expression language I created for Spring.NET back in 2005, and will release it as part of Spring 3.0. Looking forward both to Arjen's talk that covers new Spring 3.0 features and Craig Walls' talk on SpEL in particular to see what they've done with it (I'm sure Rob would say they improved it -- he always does ;)).

In any case, if you are at any of these events, please say hi. I love to meet Coherence users and learn how they are using Coherence in the wild.

September 19, 2009

I guess I should've covered why I named this blog the way I did in the intro post few months back, but better late than never ;)

One of the things I like most about Coherence is how simple it makes interop between various platforms. While the cluster-side code has to be written in Java, writing client applications is equally simple whether you write them in Java, C# or C++. There are no web services or other heavy-weight technologies to make your life miserable -- you simply code your application against the appropriate client library, and the client library itself takes care of data marshaling and the low-level communication details between the client and the cluster. Best of all, the API used is the same one you would use within the cluster, and apart from the minor platform idiosyncrasies, it is equivalent across the supported platforms.

There are two underlying technologies that make this possible. The first one is Coherence*Extend, a low-level, TCP/IP-based messaging protocol that is used for communication between the client and the cluster. While it is truly amazing piece of software in its own right, Coherence*Extend by itself wouldn't help much if it wasn't for the second piece of the puzzle -- Portable Object Format, or POF.

Portable Object Format

POF is a platform-independent serialization format that allows you to encode equivalent Java, .NET and C++ objects into the identical sequence of bytes. In other words, for any given class with the identical data members, the bytes on the wire will be exactly the same, regardless of the platform.

Implementing POF Serialization

In order to make your objects portable, you need to implement serialization code by hand. While this definitely isn't the most exciting code you'll ever write, it is quite simple and typically doesn't take more than a minute or two per class.

There are two ways to implement POF serialization. The first one is to implement PortableObject interface (IPortableObject in .NET):

As you can see, making class portable is not a rocket science -- you simply implement PortableObject interface and use the appropriate PofReader and PofWriter methods to read and write class members to a POF stream.

However, what if we don't have the source code for the class, or are not allowed to modify it? We can still make it portable using the second approach, which is to implement an external serializer. This is exactly what we will do for our .NET class:

As you can see, writing an external serializer is not much more complex either (ignore for now the calls to Read/WriteRemainder methods -- they have to do with object evolvability, which is a subject that warrants its own post).

POF Context

POF serializer does not encode class name into the POF stream -- after all, doing so would defeat its purpose, as class names are platform-specific. Instead, it encodes integer type identifier, and leaves it up to the user to ensure that type identifiers map to appropriate classes on each platform.

The mapping is achieved using one of PofContext (IPofContext in .NET) implementations. SimplePofContext allows you to map types programmatically, and is very useful for unit testing. However, in real applications you will likely want to externalize type mappings into a file, which is where ConfigurablePofContext comes in.

The ConfigurablePofContext allows you to specify mappings in an XML file. For example, configuration file for the Java POF serializer that can serialize our Person class would look like this:

On the .NET side the configuration is very similar. The only difference really is that the XML schema is used instead of DTD, and that because our class doesn't implement IPortableObject interface we also need to configure the external serializer explicitly:

Now that both Java and .NET serializer have been configured, you can serialize Person instance on one platform and deserialize it on the other.

Conclusion

In addition to portability and platform independence, there is much more to like about POF.

For one, it is extremely compact format. Instead of verbose class names, it uses integer identifiers to represent types, and you have probably already realized from code examples that the same is true for class attributes, where integer indexes are used instead of property names. It is quite common to achieve 3-5 times size reduction of serialized data when compared to standard Java or .NET binary serialization, which in the context of Coherence means 3-5x less network traffic and more importantly 3-5x less RAM needed in the cluster (or caching 3-5x the data in the same amount of RAM, depending on how you look at it).

The second benefit is the raw serialization speed. POF serialization is consistently 10-12 times faster than Java or .NET serialization. While this is pretty much irrelevant for individual cache puts and gets, due to network access overhead, it can significantly improve the performance of queries and aggregations against the cache that require objects to be deserialized.

Finally, as of Coherence 3.5, binary POF values can be manipulated directly using PofValue interface and related classes, providing you with one additional way to avoid excessive serialization and deserialization of objects in a partitioned cache. The widely discussed PofExtractor and PofUpdater are built on top of this functionality, but I personally find the former much more interesting as it opens up a world of possibilities when it comes to direct binary manipulation of cached objects.

In the next post, I will show one such example by implementing an equivalent of a VersionedPut entry processor that does not need to deserialize either the old or the new value.

June 14, 2009

I am happy to say that the book has been publicly announced. Now I really need to finish that last chapter ;-)

Joking aside, it has been quite an endeavor, and it took significantly more time than I originally expected. I started writing it back in June of last year while I was in Belgrade, continued after I moved back to the States, and am finishing it now in Santiago de Chile.

Overall, the experience has been largely positive, and I would recommend it to anyone who is passionate about a particular subject. It was also a huge learning experience, as it forced me to fill in a lot of gaps when it comes to my personal knowledge of Coherence and to really dig dip into the internals in some cases to find out how exactly things work.

Fortunately, I had a great group of people from the Coherence engineering team by my side, who were more than happy to answer my questions and to review the stuff I have written. That kind of help is invaluable, and I want to thank everyone for the help and suggestions they have provided. Thank you guys!

There is still a lot of work to do, but at least I can see the light at the end of a tunnel.

May 28, 2009

One of the things that often gets Coherence newbies is the fact that there is no built-in facility that can be used to generate object identifiers, something similar to Oracle sequences or SQL Server identity columns. Unlike databases, which store data as tuples within the database tables, Coherence stores objects as cache entries consisting of a key (identity) and associated value.

That means that in order to insert an object into a Coherence cache, you need to provide its identity as well, to serve as the key.

If the object you need to insert into the cache has a natural key, your job is easy. For example, if you need to cache Country instances, you can simply use their ISO codes as cache keys. Unfortunately, most objects don't have natural keys, or have a composite one that is too complex, so you need to come up with a way to generate synthetic keys up front.

One option is to use UUID as an identifier, and Coherence provides excellent UUID implementations in Java, .NET and C++, so if that alternative works for you, problem solved.

However, many people dislike using UUIDs as synthetic identifiers for their objects for several reasons:

They are big. Coherence UUID implementation is 256 bits, which means that each key will be 32 bytes, compared to 4 bytes for an integer or 8 for a long (actually, likely even less than that, as both integers and longs are packed when serialized using POF).

They are cumbersome to use. While for the most part you don't care about the type of identifier, UUIDs can get unwieldy when they leak to the surface, and more often they not they eventually will. For example, UUID within the URL doesn't look very friendly.

If you are in that group, and are looking for an easy way to assign unique integer-based identifiers to your objects, keep reading.

Sequence Generator Design

Conceptually, generating sequential numbers is quite simple -- all you need to do is to associate sequence name with the last assigned number, and provide a way to increment it. However, there are several issues to consider, especially in a highly concurrent and distributed system such as Coherence:

Accessibility -- it should be easy for a client application to obtain the next sequence number, which means that the sequences should be stored in a central location accessible to all cluster nodes, as well as remote clients.

Concurrency -- in order to ensure that no duplicate numbers are issued, you need to synchronize access to a sequence. If many clients try to obtain the next number from a sequence at the same time, this can lead to a bottleneck.

Performance -- obtaining the next number should be fast. Ideally, there should be no network or database calls involved, especially in a highly concurrent environment.

Reliability -- you must ensure that sequence numbers are persisted reliably as soon as they are incremented, or you might end up issuing duplicate numbers after system failure.

Coherence itself is an obvious central location, so we will create a sequences cache to store our sequences. The cache will be keyed by sequence name, and the values will be instances of a very simple Sequence class:

public class Sequence implements Serializable, PortableObject {

private long last; ...}

Basically, the only state our sequence has is a long field that is used to store the last assigned number.

In order to improve performance and reduce contention on individual sequences, we will allow clients to request a block of numbers at once. The size of the block should be configurable on a case by case basis -- for some high-contention sequences it might make sense to allocate few hundred numbers in a single call, while for others you might want to allocate individual numbers by setting block size to 1.

/** * Construct a new sequence block. * * @param first first number in a sequence * @param last last number in a sequence */ public SequenceBlock(long first, long last) { m_next = new AtomicLong(first); m_last = last; }

/** * Return the next number in the sequence. * * @return the next number in the sequence */ public synchronized Long generateIdentity() { if (allocatedSequences == null || !allocatedSequences.hasNext()) { allocatedSequences = allocateSequenceBlock(); } return allocatedSequences.next(); }

As you can see, generateIdentity method is synchronized, which makes the generator thread-safe within the single JVM. Cluster-wide concurrency is ensured using SequenceBlockAllocator entry processor, which is guaranteed to execute atomically:

/** * Allocates a block of sequences from a target entry. * <p/> * If the target entry for the given name does not already exist in a cache, * it will be created automatically. * * @param entry target entry to allocate sequence block from * * @return allocated sequence block */ public Object process(InvocableMap.Entry entry) { Sequence sequence = entry.isPresent() ? (Sequence) entry.getValue() : new Sequence();

In order to ensure reliability, we need to store the sequence numbers in a persistent store that is accessible from all Coherence nodes, such as relational database. This can be easily accomplished by configuring sequences cache to use read/write backing map and appropriate cache store implementation. Because of the differences in environments and persistence framework preferences, this is left as an exercise for the reader.

Using Sequence Generator

Once you have everything in place, actual usage of the sequence generator is quite simple -- all you need to do is create an instance of SequenceGenerator class and invoke generateIdentity method whenever you need a new id.

The most logical place to place this code in is the class for which you need to generate the identifier. For example, in order to use sequence generator to create identifiers for new Account instances, you would make a SequenceGenerator a static field of the Account class and use it within the create factory method: