Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Climb - A Generic and Dynamic Approach to Image Processing

LRDE (EPITA Research & Development Lab http://www.lrde.epita.fr/) Seminar: A Generic and Dynamic Approach to Image Processing

Abstract: "Climb is a generic image processing library. A case study of the erosion algorithm from mathematical morphology highlights the issues of a non-generic implementation. Structures such as Image, Site Set and Accumulator are deﬁned to solve them. Genericity is even increased by the concept of Morphers: a way to alter the communication between an object and the outside world. All these additions are done using the dynamic aspect of Lisp that allows for rapid prototyping."

2.
2
Copying this document
Copyright c 2010 LRDE.
Permission is granted to copy, distribute and/or modify this document under the terms of
the GNU Free Documentation License, Version 1.2 or any later version published by the Free
Software Foundation; with the Invariant Sections being just “Copying this document”, no Front-
Cover Texts, and no Back-Cover Texts.
A copy of the license is provided in the ﬁle COPYING.DOC.

4.
Chapter 1
Introduction
In software engineering, genericity is an important notion as it allows to process different types
of data while writing each algorithm only once. Image processing takes a great advantage of
genericity as it brings different points of view (and thus various types of algorithms) on diverse
types of data.
The C++ library Olena1 developed at the LRDE2 (EPITA3 Research and development labora-
tory) provides a very powerful model to implement generic image processing. It beneﬁts from
meta-programming to perform compile-time dispatches and checks. It gives good performance
with all the power of genericity. It also inherits from the major C++ drawbacks: its heavy syntax
and slow compilation to perform static operations.
The goal of Climb (Denuzière, 2009) is to provide a library with the same generic power as
Olena, however focusing on interactivity and user-friendly syntax. The Common Lisp language
provides the necessary dynamism and customizability. Its high ﬂexibility lets us consider sev-
eral implementations of the model provided by Olena.
The article is divided in two sections guided by a case study: the erosion algorithm from
mathematical morphology. The example highlights the issues of a non-generic implementation.
New structures are added to solve them and those structures are even improved by the concept
of Morphers.
1.1 Acknowledgements
Thanks to Didier Verna for being both a great teacher and project manager, Loic Denuziere for
the initial Climb project, Thierry Geraud for Olena and his presentations of Image Processing,
Lorene Poinsot and Alex Hamelin for the help during the preparation of this article.
1 http://olena.lrde.epita.fr
2 http://www.lrde.epita.fr
3 http://www.epita.fr

6.
2.2 Image Structure 6
However it is not generic enough for several reasons:
• Image structure: It only handles images that are based on a regular 2D grid that starts at
the coordinates (0, 0).
• Neighborhood: The neighborhood can be of any shape, not only a full rectangle.
• Algorithm: The code structure is the same for many algorithms, only a small part deﬁnes
the erosion algorithm.
• Image value: The erosion algorithm is not restricted to binary images, this code is.
The goal of this article is to ﬁnd a way to split this algorithm into different pieces.
2.2 Image Structure
Pixels of an image are usually represented aligned on a regular 2D grid. This is not the only
structure that exists, values can be placed on an hexagonal grid, a 3D grid, a custom grid rep-
resented by a graph or even a simplicial complex. The underneath structure of the image has
nothing to do with the erosion algorithm. Therefore an abstraction has to be established for the
image structure.
An image is traditionnaly represented by a set of pixels. In this case, a pixel holds two mean-
ings: the position and the value. Those two concepts do not ﬁt well together when working on
generic images. In this article, an image is going to be deﬁned as a function that takes a site and
return a value.
Image : Site → V alue
The underlying structure of the image is determined by the type of Site. A point is used in a
regular n-dimensional grid, a node or edge for a graph structure and so on. All the sites of the
image are grouped in a Site Set called its Domain.
Once created, the only way to maninupulate a site set is to iterate on it. The interface has been
written to be as light as possible. This will make wrapping easier later in the article.
• Site Set - reset: Restore the internal pointer at the beginning of the Site Set.
• Site Set - next: Get the next site of the set. Nil if it has reached the end of the set.
The most common use of a site set is to iterate over all the elements. A do-sites macro (Fig-
ure 2.2) is provided as a helper for this task.
( d o − s i t e s ( s i t e image )
( format t " ~A −> ~A~%" s i t e ( i r e f image s i t e ) ) )
Figure 2.2: Iterate over all points of an image

7.
7 Case Study
2.3 Neighborhood
In mathematical morphology, a structuring element is a shape describing the relation between
a pixel and its environment. The choice of the structuring element is important as it changes
the result of the algorithm. In a regular grid on the 2D space, there are two main structuring
elements representing the 4-connexity and 8-connexity (Figure 2.3). It is common to use other
structuring elements when looking for a special pattern such as an edge or a corner.
8-connexity 4-connexity Custom structuring element
Figure 2.3: Different structuring elements
The structuring element is represented by the site set concept previously deﬁned. The Box
is the only implementation of site set for points (sites in a regular grid). It’s a ﬁlled rectangle
(in 2D) between two points and perfectly ﬁts for the 8-connexity. It becomes a 4-connexity
structuring element when a binary mask is applied on top.
  
0 1 0
−1 1 
M ask  1 0 1  , Box ,
−1 1
0 1 0
All there is left to do is to center the structuring element around each site as represented in
the Figure 2.4.
Mask Center
010 101 010 (3, 3)
Figure 2.4: Structuring element implementation

8.
2.4 Algorithm 8
2.4 Algorithm
So far it is possible to iterate through all the sites of the image and ﬁnd the respective neighbors.
The accumulator concept is introduced to make the actual process generic. An accumulator
is an object that takes multiple sequential inputs and computes a result from these. Instead
of using the common reduce design in functionnal languages, accumulators have been chosen
over functions as they are real objects, thus its possible to change their implementation through
properties (Denuzière, 2010).
• Accumulator - take: Give a value to the accumulator
• Accumulator - result: Compute a value from all the given elements
• Accumulator - reset: Restore the accumulator to an empty state
Since accumulators are replacing lambda functions in the context of the reduce function, a
family of accumulator, the reduce-accus, are there to simulate it. An example of use is the Min
accumulator (Figure 2.5).
( defun min−accu ( )
( make−instance ’ reduce−accu
: f u n c t i o n ( lambda ( a b ) ( min a b ) )
: initial−value 255))
Figure 2.5: Min Accumulator
2.5 Solution
Let us deﬁne two small helper functions:
• (image-map image function): Create a new image where each value is the result of the
function applied on the associated site of the image.
• (accu-reduce site-set accu): Give all the sites of the set to the accumulator and return the
computed result.
All the pieces are now available to make a generic erosion algorithm (Figure 2.6).

10.
Chapter 3
Morphers
The previous chapter presented how to design a generic erosion algorithm, however it did not
tell how to implement the various concepts (Site set, Image and Accumulator) in a generic way.
The solution adopted in Climb is an extensive use of the Morpher concept introduced in Olena
(Ballas, 2008) with SCOOP2 (Géraud and Levillain, 2008).
A morpher is a wrapper around an object that modiﬁes how the object is seen by the outside
world. It works is by tampering communications in order to add new behaviors to the object.
It can be seen as an extension of the decorator design pattern.
In Olena, the concept of morphers is only deﬁned for images. The goal is to generalize it to
any object with a deﬁned communication protocol. In Climb, a distinction is made between
Image and Site Set, and Accumulators become good candidates for morphers.
Any message transformation can be considered as a being morpher. Two big categories of
message transformations have been identiﬁed as providing new features. Those are the Value
Morpher and Content Morpher.

11.
11 Morphers
3.1 Value Morpher
3.1.1 Deﬁnition
A value morpher is altering the values that move through the communication channel. A value
in this context is an argument of the object method. It is a simple transformation as
The main use is to have a different view of the object in a non-intrusive way. For example
view a color image as a grayscale one by altering the function that read the value of a site of an
image.
It can be implemented on the three concepts deﬁned so far: Image, Site Set and Accumulator
(Figure 3.1).
Value Morpher
Site Set
next
Update set value Image get value Query
f f
take result
Accumulator
Figure 3.1: Value Morpher
3.1.2 Image Morphers
An image is an object that associates sites with values. To use it there are two primitives: a
getter and a setter. An image value morpher is nothing more than a new getter and a new setter
(Figure 3.2). It gives the power to change the input and outputs and add new behaviors.
( make−value−morpher−image
image
( lambda ( value image s i t e ) . . . S e t t e r ...)
( lambda ( image s i t e ) . . . G e t t e r . . . ) )
Figure 3.2: Image Value Morpher Creation
It is now extremly easy to write a basic image morpher that isolates the nth component of

13.
13 Morphers
3.2 Content Morpher
3.2.1 Deﬁnition
A content morpher is altering the structure of the object. Applied on a set it changes how the
element are being sent to the outside world. Three main types of transformations are possible.
• Restriction (Figure 3.6): Some elements are removed from the original set. It is used to
work on a sub part of an image by restricting its domain, or to obtain a non-full structuring
element from a box.
• Addition: New elements are added. It can be used to simulate a mirror border on the edge
of an image.
• Order: The order of the elements may be changed, a typical use is to sort the set.
Content Morpher
Restriction
Site Set
Site
Site
f
Site
Site
Figure 3.6: Content Morpher using Restriction

14.
3.2 Content Morpher 14
3.2.2 Site Set Morphers
The 4-connexity has been previously deﬁned as being a 8-connexity structuring element masked
with a binary vector. It is implemented thanks to multiple morphers (Figure 3.8). First a content
morpher, the site-set-next operation now skips all the elements that do not match the condition.
To create such a morpher, a function that tells wether if the element is valid is required and
some work may need to be done when the site set is being resetted (Figure 3.7).
( make−restriction−content−morpher−site−set
site−set
( lambda ( s i t e ) ) ; Is valid ?
&o p t i o n a l ( lambda ( ) ) ) ) ; Reset
Figure 3.7: Prototype of a Site Set Content Restriction Morpher
Then, a value morpher is used to translate the site set around the current point.
Window (center)
Value Morpher
Shift all the points around the center
Mask (bitfield) Query
lambda alter (site)
Content Morpher
return site + center
Exclude all the points where bitfield[i] == 0
Box (topleft, botright)
lambda is_valid (site)
All sites from top left to
return bitfield[i++] = = 1
bottom right
Figure 3.8: Neighborhood

15.
15 Morphers
3.3 Combination
3.3.1 Deﬁnition
Morphers are not dependent of the object they are wrapping. Therefore once a communication
protocol has been set up, you can combine one concrete object with as many morphers as you
want implementing that protocol (Figure 3.9). Such organization allows you to write really
small morphers and combine many of them to produce complex behavior. Each morpher is easy
to unit test and the combination of morphers is less subject to bugs. The morpher design pattern
allows you to write small objects, which are easily testable, with a high degree of combination.
It follows the jQuery tagline: « Write less, Do more ».
Object 1
Object 2
Available Concrete Objects
Available Content Morphers
Available Value Morphers
Figure 3.9: Final Object = Combination of a Concrete Object, Content Morphers and Value Mor-
phers
Algorithms are written by combining pieces together.
• Concrete Objects: Speciﬁc implementation of an object.
• Algorithm: Set of rules to solve a problem that may be using other objects and algorithms.
• Morpher Objects: Transform the concrete objects to make them ﬁt the algorithm.
The genericity lies in the fact that all the pieces are independent and pluggable in many
different ways. The erosion algorithm is a good example. It makes a new image where each
pixel is the application of the "and" operation on every nearby pixel of the source image.

16.
3.3 Combination 16
3.3.2 Erosion Algorithm
To conclude this study, the full implementation of the erosion algorithm is now available repre-
sented by a schema (Figure 3.10). It includes many concrete objects and algorithms along with
morphers to make them ﬁt together.
Image Map
Apply Site Set on Accu Value Morpher
First component
Value Morpher Reduce Accu
Center Raw Image
Content Morpher and
Mask
Inputs
Box
Figure 3.10: Erosion Algorithm
• Concrete: Base objects that are not generic.
– Box: Site set returning all the sites between the start to the end point.
– Raw Image: Image stored in a vector.
– And: Function that does a logic and between the two arguments.
• Morphers: Transform the concrete objects to make them ﬁt the problem.
– Value Morpher, First component: The raw image use vectors as value, it is used to
view only the ﬁrst component of the vector.
– Content Morpher, Mask: The box gives all the sites from one point to another, a
content morpher is used as a mask to keep only the required values.
– Value Morpher, Center: A value morpher is then applied on the site set to center it
on the current pixel.
• Algorithms: Do action with the objects.
– Apply Site Set on Accu: Insert all the sites stored in the site set inside the accumulator
and returns the result of the accumulator.
– Image Map: Apply the function on all the pixels of the given image.

17.
17 Morphers
3.3.3 Top Hat Algorithm
Algorithms can be used as new bricks to build more complex algorithms. Top Hat algorithm
(Figure 3.11) uses image combine with the concrete function minus on the closing and opening
algorithms. They are both made with the erosion brick built previously.
Image Combine
minus
Closing Opening
Erosion Dilation
Dilation Erosion
Figure 3.11: Top Hat Algorithm
Many algorithms can be implemented combining together all the bricks deﬁned so far. The
following list are present in Climb.
• Erosion
• Dilation
• Hit or Miss
• Opening
• Closing
• Top Hat
• Thinning
• Gradient
• Laplacian

18.
Chapter 4
Conclusion
The erosion algorithm is easy to write in a speciﬁc version but requires the addition of many
structures such as Image, Site Set and Accumulators to become generic. The algorithm is now
written once and the addition of new image types requires localized modiﬁcations.
The structures are given genericity through the concept of Morphers. By tampering the com-
munications of an object, it is possible to extend its functionnalities in a non intrusive and trans-
parent way. It gives the ability to transform speciﬁc objects to make them ﬁt speciﬁc algorithms
without memory loss. Olena introduced the Morpher concept for the images, Climb hopes to
make it viable for everything else.
The functional aspect of Lisp helps to produce a consise code through the separation in of
small functions and built-in lambda expressions. In addition with the Lisp dynamic develop-
ment environment it makes prototyping fast and easy.
The future work consists mainly in the addition of more image types, algorithms and mor-
pher integration. The performance is a major issue in the current implementation and needs a
deep reﬂexion.