Parallel Programming Paradigms in Clojure, Part II

This is the second part in the Parallel Programming in Clojure series that introduces some advanced topics and functionalities for concurrency in Clojure. In Part 1, we gave an overview of concurrency in Clojure and covered some basic topics including Agents, Concurrency Macros, Futures and Java Interoperatability. In this Part 2 blog post, we will discuss some mutable types such as Refs and Atom along with their use cases and examples. We will also shed some light on fork/join parallelism through Reducers that should further enhance your understanding about the possibilities for using Clojure.
Often, parallelized code can be further accelerated through fine tuning, which requires knowledge of the advanced constructs in Clojure that are available. You should, for instance, have an understanding of using atomic operations and shared memories to perform even the basic functionalities in serial codes, such as counting and data sharing. Knowledge of these advanced concepts will also help you avoid race conditions, deadlocks, and non-deterministic results. This post will introduce these topics to you, but be sure to look at the Clojure books mentioned later on to dive further into the details.

Refs and STM

Besides Agents that were discussed in Part 1, Clojure provides two other mutable types. One of them is a coordinated Reference type (refs) that can be very useful when multiple identities are participating in concurrent and overlapping operations. They are helpful in avoiding inconsistent states, deadlocks and race conditions, to name a few. These properties of refs are due to Clojure’s implementation of Software Transactional Memories (STM), which is a systemic implementation of error-prone memory management. In Clojure, each STM transaction ensures that the changes being made to the refs are atomic, consistent and isolated. The STM system implements multi-version concurrency control (MVCC) which is also supported by databases such as PostgreSQL and Oracle. The refs are usually mutated or changed using three functions: ref-set, alter, and commute, all of which are explained in detail below.

ref-set

This is the basic function to mutate a reference. The mutations to a ref, however, can only be done inside an STM transaction to ensure its properties as explained above. The simple example below shows the creation of ref and its mutation using ref-set inside a built-in transaction macro dosync.

1

2

3

4

(defall-users(ref{}))

(dosync

(ref-setall-users{}))

{}

Alter

The references can also be mutated using the alter function, which reads the current value of the ref, performs a function on it, and stores the new value in the macro. All of these operations are done as a single atomic operation. For instance, access to read and write to other threads is blocked until all the three operations have been performed. A general form of an alter function is given below:

1

(alter ref function&amp;args)

The first argument of the alter function is the reference to be mutated, followed by the function to be performed on the ref. The remaining arguments are the ones specified in the call to alter.

Commute

The commute function, as the name applies, is for operations that are mathematically commutative in nature. A function is commutative if it does not matter in which order the arguments are applied. For example, If y = 10 and z = 2, consider the expressions:

1

2

x *y=y *z;

x/y!=y/z;

Hence the multiplication function is commutative, whereas the division function is not.

In terms of Clojure programming, if two operations are launched to mutate a ref at once, one of them will proceed, whereas the other is blocked until the first operation has completed. The order of the execution is not known; hence, the operations must be commutative in nature and must not depend upon the order of the execution. This is ideal to implement counters where each operation should increment the current value by one without depending on the current value of the counter, hence not depending on the order of thread execution. The general form of commute functions is shown here:

1

(commute ref function&amp;args)

As you can see, the syntax is similar to alter and performs the operations in a similar atomic manner. The function passed in this case, however, should be commutative.

Atoms

The atoms are constructs in Clojure which allow independent and synchronous changes to mutable data. They are different to refs since the changes are independent and non-coordinated — either all changes happen or none does. In comparison to agents that make asynchronous changes, the atoms work synchronously or immediately. Hence, each construct has unique properties making them suitable for different applications.

The atoms are usually mutated using either of these three functions: Reset!, Swap! and Compare-and-Set!. Since the changes made to an atom are independent of other atoms, there is no need to use transactions during mutations. The following code shows the creation of an atom followed by the explanation of the above mentioned mutation functions for atoms.

1

(def new-num(atom0))

This is the general form of creating an atom with an initialization to zero.

Reset!

The Reset! function for atoms works similar to ref-set for references. It discards the current value of the atom and sets it to the value provided in the arguments.

1

(reset!new-num new-value)

Here, the reset! function is setting the new-num to new-value in a single atomic operation.

Swap!

The Swap! function has the following general form:

1

(swap!new-num function&amp;args)

The swap! function reads the current value of the new-num, performs the function on it using the arguments and stores the result in new-num, all in a single synchronous atomic operation.

For example:

1

(swap!new-num+5)

Here, the value ‘5’ will be added to current value of the new-num ‘0’ and the updated value of new-num will become ‘5’.

Compare-and-Set!

The Compare-and-Set! reads the current value of the atom, compares it with the supplied old value and if the match is true, it replaces the current value with the supplied new-value. It takes the following general form:

1

(compare-and-set!new-num old-value new-value)

Hence, it does two operations internally, a comparison and a mutation. However, it is a low-level function and is used internally by the swap! function. Besides swap!, it can be used to make several different types operations such as comparators.

Reducers

Reduction is a special approach available with many programming languages that uses fork and join concepts to achieve data level parallelization. In Clojure, the reducers are introduced in the clojure.core.reducers library and one their most useful features is that they can automatically partition the operations on tree based data structures leading to parallelization. The included fold function takes a reducing function, a combining function and a collection, and returns the reduced results by computing sub-segments of the collection in parallel. This can lead to better resource utilization, since an otherwise serial operation is being divided into multiple sub-operations that can perform in parallel. In some parallel languages, such as OpenCL, such functionalities can be implemented in kernels to achieve large performance increases. However, not all operations can be optimized in this way, especially operations with propagating dependencies.

Safari Books Online has the content you need

Check out these Clojure books available from Safari Books Online:

The Joy of Clojure goes beyond the syntax, and shows how to write fluent, idiomatic Clojure code. Readers will learn to approach programming challenges from a Functional perspective and master the Lisp techniques that make Clojure so elegant and efficient. This book will help readers think about problems the “Clojure way,” and recognize when they simply need to change the way they program. Not just another book about programming philosophy, this book tackles hard software areas like concurrency, interoperability, performance, and more.

Clojure Data Analysis Cookbook presents recipes for every stage of the data analysis process. Whether scraping data off a web page, performing data mining, or creating graphs for the web, this book has something for the task at hand. You’ll learn how to acquire data, clean it up, and transform it into useful graphs that can then be analyzed and published to the Internet. Coverage includes advanced topics like processing data concurrently, applying powerful statistical techniques like Bayesian modelling, and even data mining algorithms such as K-means clustering, neural networks, and association rules.

Clojure Programming, helps you learn the fundamentals of Clojure with examples relating it to the languages you know already—whether you’re focused on data modeling, concurrency and parallelism, web programming, statistics and data analysis, and more.

About the author

Usman Aziz is a technical lead at TunaCode, Inc., a startup that delivers GPU-accelerated computing solutions to time-critical application domains. He holds a degree in Computer Systems Engineering. His current focus is on protecting bulk data. He can be reached at usman@tunacode.com.