This post reprises the examples in the reducers post but now includes
equivalent contrived transducers examples as well.

I’ve includes some material from the reducers post for completeness, but without
any explanation, so you don’t have to flip between the two. But you may
want to refer to the reducers post for reducers background.

The Code

The Code - repo is on Github

The repo with the example code can be found on
Github. Its a
Leiningen project.

The Code - misc

The Examples Collection

Many of the following examples will use the same collection
holding the population of a small village. The village
has four families, some families have two parents, others one; and
each family between one and three children. One family lives in the North, and one each in the South, East and West.

This is stylised, artificial and contrived data designed to be easily understandable to most and in no way intended to suggest any family structures, conventions or arrangements as socially preferable. Just saying.

Example 1b - using a transducer to count how many children in the village

The transducers way looks very similar. Literally the only difference is to use core map
rather than the reducers map.

First the transducer
function ex1b-map-children-to-value-1 is created using the new
arity for map that takes just the mapping function, no
collection.

Then the transducer is used with the new transduce function to
reduce the collection and return the answer. transduce take the transducer
function as its first argument, then the “usual” reducer arguments
of reducing function, initial value and collection.

;; Example 1b - using a transducer to add up all the mapped values;; create the transducers using the new arity for map that;; takes just the function, no collection
(defex1b-map-children-to-value-1 (map #(if (=:child (:role %)) 10)))
;; now use transduce (c.f r/reduce) with the transducer to get the answer
(transduce ex1b-map-children-to-value-1 +0 village)
;;=>8

Example 2 - how many children in the Brown family?

An obvious way to find how many children just in the Brown family
would be to select ( filter) the members of the Brown family, and
use the same map function - e.g. ex1a-map-children-to-value-1 - from
Example 1 to count the children.

Example 2a - using a reducer to count the children in the Brown family

Along with map, reducers have a filter function that returns another function that can be used with reduce:

;; Example 2a - using a reducer to count the children in the Brown family;; create the reducer to select members of the Brown family
(defex2a-select-brown-family (r/filter #(="brown" (string/lower-case (:family %)))))
;; compose a composite function to select the Brown family and map children to 1
(defex2a-count-brown-family-children (comp ex1a-map-children-to-value-1 ex2a-select-brown-family))
;; reduce to add up all the Brown children
(r/reduce +0 (ex2a-count-brown-family-children village))
;;=>2

Its worth observing reducersreduce does not need to create
any intermediate collections.

Example 2b - using a transducer to count the children in the Brown family

The transducer-aware corefilter function can be used to select
the Brown family members, while the transducerex1b-map-children-to-value-1 can be used to map children to 1, else 0.

As with reducers, the two functions ex2b-select-brown-family
and ex1b-map-children-to-value-1 can be composed together.

And as before, transduce is used to count (reduce) the number of children.

;; Example 2b - using a transducer to count the children in the Brown family;; create the transducer filter to select members of the Brown family
(defex2b-select-brown-family (filter #(="brown" (string/lower-case (:family %)))))
;; compose a composite function to select the Brown family and map children to 1;; NOTE: transducer comp functions are applied left-to-right
(defex2b-count-brown-family-children (comp ex2b-select-brown-family ex1b-map-children-to-value-1))
;; transduce to add up all the Brown children
(transduce ex2b-count-brown-family-children +0 village)
;;=>2

Note there is a gotcha here. The functions in the composed
transducer ex2b-count-brown-family-children are
applied left-to-right not right-to-left as is usual with comp.

Although not explicitly stated in Rich’s
post I guess transducers
do not create intermediate collections either.

Example 3 - how many children’s names start with J?

We already know the answer: just the 3 children in the Jones family.

Algorithmically, this is a three step pipeline: filter on children, a filter on
names beginning with “J” (or “j”) and finally count of how many (children) in the result.

Example 3a - using a reducer to count children with names beginning with J

;; Example 3a - using a reducer to count children with names beginning with J;; select (filter) just the children
(defex3a-select-children (r/filter #(=:child (:role %))))
;; select names beginning with "j"
(defex3a-select-names-beginning-with-j (r/filter #(="j" (string/lower-case (first (:name %))))))
;; In Example 1 we created the _map_ function;; ex1a-map-children-to-value-1 to enable reduce to count the number;; of children.;; But the need to count the number of entries in a collection using;; reduce, after a pipeline possibly involving many filters and;; mappers, is a common one. This is straightforward to do, in the final;; stage of the pipeline use a map function to transform each entry to;; value 1.;; map entries in a collection to 1
(defex0a-map-to-value-1 (r/map (fn [v] 1)))
;; create the three step count-children-with-names-beginning-j function
(defex3a-count-children-with-names-beginning-j (comp ex0a-map-to-value-1
ex3a-select-names-beginning-with-j
ex3a-select-children))
;; reduce the village with the ex32-count-children-with-names-beginning-j function
(r/reduce +0 (ex3a-count-children-with-names-beginning-j village))
;; =>3

Its worth labouring the point that composing the custom reducer count-children-with-names-beginning-js from
individual filters and mappers is a very powerful technique.

Example 3b - using a transducer to count children with names beginning with J

As with transducers map, filter has a new arity, taking just the
filtering function, no collection.

Example 4 - creating a collection of children whose names start with J?

Sometimes you will want the resulting collection itself, post map,
filter, etc, and not reduced any further.

Example 4a - using a reducer to create a collection of children whose names start with J?

Since we want the actual entries, rather than count them, we need a
pure filter pipeline, similar to Example 3 but one that doesn’t use
the ex0a-map-to-value-1 mapper.

Creating a vector of the J children can be done simply by using into.

Under the covers into uses reduce so creating the vector is just a
matter of applying the
ex4a-select-children-with-names-beginning-with-j reducers to the
village and then using the resulting collection with into to create the vector.

Example 5 - calculate the average age of children on or below the equator

A more involved, but still straightforward, example to finish this
section: what is the average age of the children who live on or below
the equator? By equator I mean where home is East, South or West.

To do this, the value of home will be mapped to a latitude and
longitude. For example West will be :lat 0 :lng -180 and South is
:lat -90 :lng 0.

Example 5a - using a reducer to calculate the average age of children on or below the equator

;; Example 5a - using a reducer to calculate the average age of children on or below the equator;; map :home to latitude and longitude
(defex5a-map-home-to-latitude-and-longitude
(r/map
(fn [v]
(condp= (:home v)
:north (assoc v :lat90:lng0)
:south (assoc v :lat-90:lng0)
:west (assoc v :lat0:lng-180)
:east (assoc v :lat0:lng180)))))
;; select people on or below the equator i.e. latitude <= 0
(defex5a-select-people-on-or-below-equator (r/filter #(>=0 (:lat %))))
;; To find the average age, we need to add up all the children's ages and;; divide by how many children.;; Note, rather than creating a composite pipeline function, in this example;; the individual stages of the pipeline are used explicitly.;; count the number of children on or below the equator
(defex5a-no-children-on-or-below-the-equator
(r/reduce +0
(ex0a-map-to-value-1
(ex5a-select-people-on-or-below-equator
(ex5a-map-home-to-latitude-and-longitude
(ex3a-select-children village))))))
;; sum the ages of children
(defex5a-select-age (r/map #(:age %)))
(defex5a-sum-of-ages-of-children-on-or-below-the-equator
(r/reduce +0
(ex5a-select-age
(ex5a-select-people-on-or-below-equator
(ex5a-map-home-to-latitude-and-longitude
(ex3a-select-children village))))))
;; calculate the average age of children on or below the equator
(defex5a-averge-age-of-children-on-or-below-the-equator
(float (/ ex5a-sum-of-ages-of-children-on-or-below-the-equator ex5a-no-children-on-or-below-the-equator )))
;; =>17.3

Example 5b - using a transducer to calculate the average age of children on or below the equator

;; Example 5b - using a transducer to calculate the average age of children on or below the equator;; map :home to latitude and longitude
(defex5b-map-home-to-latitude-and-longitude
(map
(fn [v]
(condp= (:home v)
:north (assoc v :lat90:lng0)
:south (assoc v :lat-90:lng0)
:west (assoc v :lat0:lng-180)
:east (assoc v :lat0:lng180)))))
;; select people on or below the equator i.e. latitude <= 0
(defex5b-select-people-on-or-below-equator (filter #(>=0 (:lat %))))
;; create a "utility" transducer to select children on or below the equator
(defex5b-select-children-on-or-below-the-equator
(comp ex3b-select-children
ex5b-map-home-to-latitude-and-longitude
ex5b-select-people-on-or-below-equator))
;; create a transducer to count the number of children on or below the equator
(defex5b-count-children-on-or-below-the-equator
(comp ex5b-select-children-on-or-below-the-equator
ex0b-map-to-value-1))
;; now count the number of children on or below the equator
(defex5b-no-children-on-or-below-the-equator
(transduce ex5b-count-children-on-or-below-the-equator +0 village))
;; create a transducer to extract the age
(defex5b-select-age (map #(:age %)))
;; create a transducer to extract the ages of all chilren;; on or below the equator
(defex5b-extract-ages-of-children-on-or-below-thew-equator
(comp
ex5b-select-children-on-or-below-the-equator
ex5b-select-age))
;; now sum the ages
(defex5b-sum-of-ages-of-children-on-or-below-the-equator
(transduce ex5b-extract-ages-of-children-on-or-below-thew-equator +0 village ))
;; calculate the average age of children on or below the equator
(defex5b-averge-age-of-children-on-or-below-the-equator
(float (/ ex5b-sum-of-ages-of-children-on-or-below-the-equator ex5b-no-children-on-or-below-the-equator)))
;; =>17.3

Example 6 - comparing the performance of reducers and transducers

Lets time the addition of the ages from Example 5 using reducersreduce and fold, and transducerstransduce.

So, in this example, its not possible to see any performance difference between reducers and transducers.

Example 7 - all the relatives visit the village!

Its that time of year again and all the relatives of the families in the village visit and
the population of the village swells enormously to 10 million people.

Example 7 - make some visitors

Lets define some functions to create an influx of visitors. Note, no attempt has been made to ensure this randomly generated data makes any sort of real world sense - it could include e.g. a child of age 100.

In the original reducers post I
found reducersfold was nearly four times faster than reducersreduce. Here fold is about three times faster. (I’m using the same
four core workstation.)

Transducers come out with around the same time as reducers reduce.

All these numbers should be taken with a large pinch of salt, they
are just a “wet finger” and this is in no way a rigorous benchmark.

Final Words

I’ve only scratched the surface of transducers in this post of
course and Rich’s post has opened only
a crack in the door to understanding the potential of transducers;
there will be more to come,
appreciate and learn I’m sure.

So far I’ve found transducers more immediately understandable than I
did reducers, maybe because I already have a reasonable grasp of the
latter and some existing mental context to understand the former. It
will be interesting to hear / learn how other people new to both
groktransducers.

I’ve been doing a project using reducersfold for the parallisation
benefits and have noticed I have needed to consciously mentally “switch” between the
core world and reducers world. In that sense I think reducers
have an “impedance mismatch” with the rest of core.

On the other
hands, writing the examples above I’ve felt transducers
are more “grounded” with core; indeed they are core (there is no transducers
namespace).

To sum up my opinion in a pithy one-liner: transducers are reducers decomplected.