Menu 1

Tag Archives: krita

Disclaimer:This article is a duplicate of this post, originally published on the Pixls.us website, by the same authors.

The IMAGE team of the research laboratory GREYC in Caen/France is pleased to announce the release of a new major version (numbered 2.0) of its project G’MIC: a generic, extensible, and open source framework for image processing. Here, we present the main advances made in the software since our last article. The new features presented here include the work carried out over the last twelve months (versions 2.0.0 and 1.7.x, for x varying from 2 to 9).

Links:

1. G’MIC: A brief overview

G’MIC is an open-source project started in August 2008, by the IMAGE team. This French research team specializes in the fields of algorithms and mathematics for image processing. G’MIC is distributed under the CeCILL license (which is GPL compatible) and is available for multiple platforms (GNU/Linux, MacOS and Windows). It provides a variety of user interfaces for manipulating generic image data, that is to say, 2D or 3D multispectral images (or sequences) with floating-point pixel values. This includes, of course, “classic” color images.

Fig.1.1: Logo of the G’MIC project, an open-source framework for image processing, and its mascot Gmicky.

The popularity of G’MIC mostly comes from the plug-in it provides for GIMP (since 2009). To date, there are more than 480 different filters and effects to apply to your images, which considerably enlarges the list of image processing filters available by default in GIMP.

G’MIC also provides a powerful and autonomous command-line interface, which is complementary to the CLI tools you can find in the famous ImageMagick or GraphicsMagick projects. There is also a web service G’MIC Online, allowing to apply image processing effects directly from a browser. Other (but less well known) G’MIC-based interfaces exist: a webcam streaming tool ZArt, a plug-in for Krita, a subset of filters available in Photoflow, Blender or Natron… All these interfaces are based on the CImg and libgmic libraries, that are portable, thread-safe and multi-threaded, via the use of OpenMP.

G’MIC has more than 950 different and configurable processing functions, for a library of only 6.5Mio, representing a bit more than 180 kloc. The processing functions cover a wide spectrum of the image processing field, offering algorithms for geometric manipulations, colorimetric changes, image filtering (denoising and detail enhancement by spectral, variational, non-local methods, etc.), motion estimation and registration, display of primitives (2D or 3D mesh objects), edge detection, object segmentation, artistic rendering, etc. It is therefore a very generic tool for various uses, useful on the one hand for converting, visualizing and exploring image data, and on the other hand for designing complex image processing pipelines and algorithms (see these project slides for details).

2. A new versatile interface, based on Qt

One of the major new features of this version 2.0 is the re-implementation of the plug-in code, from scratch. The repository G’MIC-Qt developed by Sébastien (an experienced member of the team) is a Qt-based version of the plug-in interface, being as independent as possible of the widget API provided by GIMP.

Fig.2.1: Overview of version 2.0 of the G’MIC-Qt plug-in running for GIMP.

This has several interesting consequences:

The plug-in uses its own widgets (in Qt) which makes it possible to have a more flexible and customizable interface than with the GTK widgets used by the GIMP plug-in API: for instance, the preview window becomes resizable at will, manages zooming by mouse wheel, and can be freely moved to the left or to the right. A filter search engine by keywords has been added, as well as the possibility of choosing between a light or dark theme. The management of favorite filters has been also improved and the interface even offers a new mode for setting the visibility of the filters. Interface personalization is now a reality.

The plug-in also defines its own API, which is used to facilitate its integration in third party software (other than GIMP). In practice, a software developer has to write a single file host_software.cpp implementing the functions of the API to make the link between the plug-in and the host application. Currently, the file host_gimp.cpp does this for GIMP as a host. But there is now also a stand-alone version available (file host_none.cpp that runs this Qt interface in solo mode, from a shell (with command gmic_qt).

Boudewijn Rempt, project manager and developer of the marvelous painting software Krita, has also started writing such a file host_krita.cpp to make this “new generation” plug-in communicate with Krita. In the long term, this should replace the previous G’MIC plug-in implementation they made (currently distributed with Krita), which is aging and poses maintenance problems for developers.

Minimizing the integration effort for developers, sharing the G’MIC plug-in code between different applications, and offering a user interface that is as comfortable as possible, have been the main objectives of this complete redesign. As you can imagine, this rewriting required a long and sustained effort, and we can only hope that this will raise interest among other software developers, where having a consistent set of image processing filters could be useful (a file host_blender.cpp available soon ? We can dream!). The animation below illustrates some of the features offered by this new Qt-based interface.

Fig.2.2: The new G’MIC-Qt interface in action.

Note that the old plug-in code written in GTK was updated also to work with the new version 2.0 of G’MIC, but has fewer features and probably will not evolve anymore in the future, unlike the Qt version.

3. Easing the work of cartoonists…

One of G’MIC’s purposes is to offer more filters and functions to process images. And that is precisely something where we have not relaxed our efforts, despite the number of filters already available in the previous versions!

In particular, this version comes with new and improved filters to ease the colorization of line-art. Indeed, we had the chance to host the artist David Revoy for a few days at the lab. David is well known to lovers of art and free software by his multiple contributions in these fields (in particular, his web comic Pepper & Carrot is a must-read!). In collaboration with David, we worked on the design of an original automatic line-art coloring filter, named Smart Coloring.

Fig.3.1: Use of the “Colorize line-art [smart coloring]” filter in G’MIC.

When drawing comics, the colorization of line-art is carried out in two successive steps: The original drawing in gray levels (Fig.3.2.1) is first pre-colored with solid areas, i.e. by assigning a unique color to each region or distinct object in the drawing (Fig.3.2.3). In a second step, the colourist reworks this pre-coloring, adding shadows, lights and modifying the colorimetric ambiance, in order to obtain the final colorization result (Fig.3.2.4). Practically, flat coloring results in the creation of a new layer that contains only piecewise constant color zones, thus forming a colored partition of the plane. This layer is then merged with the original line-art to get the colored rendering (merging both in multiplication mode, typically).

Fig.3.2: The different steps of a line-art coloring process (source: David Revoy).

Artists admit it themselves: flat coloring is a long and tedious process, requiring patience and precision. Classical tools available in digital painting or image editing software do not make this task easy. For example, even most filling tools (bucket fill) do not handle discontinuities in drawn lines very well (Fig.3.3.a), and even worse when lines are anti-aliased. It is then common for the artist to perform flat coloring by painting the colors manually with a brush on a separate layer (Fig.3.3.b), with all the precision problems that this supposes (especially around the contour lines, Fig.3.3.c). See also this link for more details.

It may even happen that the artist decides to explicitly constrain his style of drawing, for instance by using aliased brushes in a higher resolution image, and/or by forcing himself to draw only connected contours, in order to ease the flat colorization work that has to be done afterwards.

The Smart Coloring filter developed in version 2.0 of G’MIC allows to automatically pre-color an input line-art without much work. First, it analyses the local geometry of the contour lines (estimating their normals and curvatures). Second, it (virtually) does contour auto-completion using spline curves. This virtual closure allows then the algorithm to fill objects with disconnected contour plots. Besides, this filter has the advantage of being quite fast to compute and gives coloring results of similar quality to more expensive optimization techniques used in some proprietary software. This algorithm smoothly manages anti-aliased contour lines, and has two modes of colorization: by random colors (Fig.3.2.2 and Fig.3.4) or guided by color markers placed beforehand by the user (Fig.3.5).

In “random” mode, the filter generates a piecewise constant layer that is very easy to recolor with correct hues afterwards. This layer indeed contains only flat color regions, and the classic bucket fill tool is effective here to quickly reassign a coherent color to each existing region synthesized by the algorithm.

In the user-guided markers mode, color spots placed by the user are extrapolated in such a way that it respects the geometry of the original drawing as much as possible, taking into account the discontinuities in the pencil lines, as this is clearly illustrated by the figure below:

This innovative, flat coloring algorithm has been pre-published on HAL (in French): A semi-guided high-performance flat coloring algorithm for line-arts. Curious people could find there all the technical details of the algorithm used. The recurring discussions we had with David Revoy on the development of this filter enabled us to improve the algorithm step by step, until it became really usable in production. This method has been used successfully (and therefore validated) for the pre-colorization of the whole episode 22 of the webcomic Pepper & Carrot.

The wisest of you know that G’MIC already had a line-art colorization filter! True, but unfortunately it did not manage disconnected contour lines so well (such as the example in Fig.3.5), and could then require the user to place a large number of color spots to guide the algorithm properly. In practice, the performance of the new flat coloring algorithm is far superior.

And since it does not see any objection to anti-aliased lines, why not create ones? That is the purpose of another new filter “Repair / Smooth [antialias]” able to add anti-aliasing to lines in cartoons that would have been originally drawn with aliased brushes.

4. …Not to forget the photographers!

“Colorizing drawings is nice, but my photos are already in color!”, kindly remarks the impatient photographer. Don’t be cruel! Many new filters related to the transformation and enhancement of photos have been also added in G’MIC2.0. Let’s take a quick look of what we have.

4.1. CLUTs and colorimetric transformations

CLUTs (Color Lookup Tables) are functions for colorimetric transformations defined in the RGB cube: for each color (Rs,Gs,Bs) of a source image Is, a CLUT assigns a new color (Rd,Gd,Bd) transferred to the destination image Id at the same position. These processing functions may be truly arbitrary, thus very different effects can be obtained according to the different CLUTs used. Photographers are therefore generally fond of them (especially since these CLUTs are also a good way to simulate the color rendering of certain old films).

In practice, a CLUT is stored as a 3D volumetric color image (possibly “unwrapped” along the z = B axis to get a 2D version). This may quickly become cumbersome when several hundreds of CLUTs have to be managed. Fortunately, G’MIC has a quite efficient CLUT compression algorithm (already mentioned in a previous article), which has been improved version after version. So it was finally in a quite relax atmosphere that we added more than 60 new CLUT-based transformations in G’MIC, for a total of 359CLUTs usable, all stored in a data file that does exceed 1.2 Mio. By the way, let us thank Pat David, Marc Roovers and Stuart Sowerby for their contributions to these color transformations.

Fig.4.1.1: Some of the new CLUT-based transformations available in G’MIC (source: Pat David).

But what if you already have your own CLUT files and want to use them in GIMP? No problem ! The new filter “Film emulation / User-defined” allows to apply such transformations from CLUT data file, with a partial support of files with extension .cube (CLUT file format proposed by Adobe, and encoded in ASCIIo_O!).

And for the most demanding, who are not satisfied with the existing pre-defined CLUTs, we have designed a very versatile filter “Colors / Customize CLUT“, that allows the user to build their own custom CLUTfrom scratch: the user places colored keypoints in the RGB color cube and these markers are interpolated in 3D (according to a Delaunay triangulation) in order to rebuild a complete CLUT, i.e. a dense function in RGB. This is extremely flexible, as in the example below, where the filter has been used to change the colorimetric ambiance of a landscape, mainly altering the color of the sky. Of course, the synthesized CLUT can be saved as a file and reused later for other photographs, or even in other software supporting this type of color transformations (for example RawTherapee or Darktable).

Fig.4.1.2: Filter “Customize CLUT” used to design a custom color transform in the RGB cube.

Fig.4.1.3: Result of the custom colorimetric transformation applied to a landscape.

To stay in the field of color manipulation, let us also mention the appearance of the filter “Colors / Retro fade” which creates a “retro” rendering of an image with grain generated by successive averages of random quantizations of an input color image.

Fig.4.1.4: Filter “Retro fade” in the G’MIC plug-in.

4.2. Making the details pop out

Many photographers are looking for ways to process their digital photographs so as to bring out the smallest details of their images, sometimes even to exaggeration, and we can find some of them in the pixls.us forum. Looking at how they perform allowed us to add several new filters for detail and contrast enhancement in G’MIC. In particular, we can mention the filters “Artistic / Illustration look” and “Artistic / Highlight bloom“, which are direct re-implementations of the tutorials and scripts written by Sébastien Guyader as well as the filter “Light & Shadows / Pop shadows” suggested by Morgan Hardwood. Being immersed in such a community of photographers and cool guys always gives opportunities to implement interesting new effects!

In the same vein, G’MIC gets its own implementation of the Multi-scale Retinex algorithm, something that was already present in GIMP, but here enriched with additional controls to improve the luminance consistency in images.

Fig.4.2.2: Filter “Retinex” for improving luminance consistency.

Our friend and great contributor to G’MIC, Jérome Boulanger, also implemented and added a dehazing filter “Details / Dcp dehaze” to attenuate the fog effect in photographs, based on the Dark Channel Prior algorithm. Setting the parameters of this filter is kinda hard, but the filter gives sometimes spectacular results.

Fig.4.2.3: Filter “DCP Dehaze” to attenuate the fog effect.

And to finish with this subsection, let us mention the implementation in G’MIC of the Rolling Guidance algorithm, a method to simplify images that has become a key step used in many newly added filters. This was especially the case in this quite cool filter for image sharpening, available in “Details / Sharpen [texture]“. This filter works in two successive steps: First, the image is separated into a texture component + a color component, then the details of the texture component only are enhanced before the image is recomposed. This approach makes it possible to highlight all the small details of an image, while minimizing the undesired halos near the contours, a recurring problem happening with more classical sharpening methods (such as the well known Unsharp Mask).

Fig.4.2.4: The “Sharpen [texture]“” filter shown for two different enhancement amplitudes.

4.3. Masking by color

As you may know, a lot of photograph retouching techniques require the creation of one or several “masks”, that is, the isolation of specific areas of an image to receive differentiated processing. For example, the very common technique of luminosity masks is a way to treat differently shadows and highlights in an image. G’MIC2.0 introduces a new interesting filter “Colors / Color mask [interactive]” that implements a relatively sophisticated algorithm (albeit computationally demanding) to help creating complex masks. This filter asks the user to hover the mouse over a few pixels that are representative of the region to keep. The algorithm learns in real time the corresponding set of colors or luminosities and deduces then the set of pixels that composes the mask for the whole image (using Principal Component Analysis on the RGB samples).

Once the mask has been generated by the filter, the user can easily modify the corresponding pixels with any type of processing. The example below illustrates the use of this filter to drastically change the color of a car

Fig.4.3.1: Changing the color of a car, using the filter “Color mask [interactive]“.

It takes no more than a minute and a half to complete, as shown in the video below:

Hence, with his cooperation, we added new commands -median_files,-median_videos, -average_files and-average_videos to compute all these image features very easily using the CLI tool gmic. The example below shows the results obtained from a sub-sequence of the « Big Buck Bunny” video. We have simply invoked the following commands from the Bash shell:

And to stay in the field of video processing, we can also mention the addition of the commands -morph_files and -morph_video that render temporal interpolations of video sequences, taking the estimated intra-frame object motion into account, thanks to a quite smart variational and multi-scale estimation algorithm.

The video below illustrates the rendering difference obtained for the retiming of a sequence using temporal interpolation, with (right) and without (left) motion estimation.

Fig.5.1.4: Video retiming using G’MIC temporal morphing technique.

5.2. Deformations and “Glitch Art”

Those who like to mistreat their images aggressively will be delighted to learn that a bunch of new image deformation and degradation effects have appeared in G’MIC.

First of all, the filter “Deformations / Conformal maps” allows one to distort an image using conformal maps. These deformations have the property of preserving the angles locally, and are most often expressed as functions of complex numbers. In addition to playing with predefined deformations, this filter allows budding mathematicians to experiment with their own complex formulas.

Fans of Glitch Art may also be concerned by several new filters whose rendering look like image encoding or compression artifacts. The effect “Degradations / Pixel sort” sorts the pixels of a picture by row or by column according to different criteria and to possibly masked regions, as initially described on this page.

Degradations / /Pixel sort also has two little brothers, filters “Degradations / Flip & rotate blocks” and “Degradations / Warp by intensity“. The first divides an image into blocks and allows to rotate or mirror them, potentially only for certain color characteristics (like hue or saturation, for instance).

It should be noted that these filters were largely inspired by the Polyglitch plug-in, available for Paint.NET, and have been implemented after a suggestion from a friendly user (yes, yes, we try to listen to our most friendly users!).

5.3. Image simplification

What else do we have in store? A new image abstraction filter, Artistic / Sharp abstract, based on the Rolling Guidance algorithm mentioned before. This filter applies contour-preserving smoothing to an image, and its main consequence is to remove the texture. The figure below illustrates its use to generate several levels of abstraction of the same input image, at different smoothing scales.

In the same vein, G’MIC also gets a filter Artistic / Posterize which degrades an image to simulate posterization. Unlike the filter with same name available by default in GIMP (which mainly tries to reduce the number of colors, i.e. do color quantization), our version adds spatial simplification and filtering to approach a little more the rendering of old posters.

Fig.5.3.2: Filter “Posterize” of G’MIC, compared to the filter with same name available by default in GIMP.

5.4. Other filters

If you still want more (and in this case one could say you are damn greedy!), we will end this section by discussing some of the new, but unclassifiable filters.

We start with the filter “Artistic / Diffusion tensors“, which displays a field of diffusion tensors, calculated from the structure tensors of an image (structure tensors are symmetric and positive definite matrices, classically used for estimating the local image geometry). To be quite honest, this feature had not been originally developed for an artistic purpose, but users of the plug-in came across it by chance and asked to make a GIMP filter from it. And yes, this is finally quite pretty, isn’t it?

From a technical point of view, this filter was actually an opportunity to introduce new drawing features into the G’MIC mathematical evaluator, and it has now become quite easy to develop G’MIC scripts for rendering custom visualizations of various image data. This is what has been done for instance, with the command -display_quiver reimplemented from scratch, and which allows to generate this type of rendering:

For lovers of textures, we can mention the apparition of two new fun effects: First, the “Patterns / Camouflage” filter. As its name suggests, this filter produces a military camouflage texture.

Fig. 5.4.3: Filter “Camouflage“, to be printed on your T-shirts to go unnoticed in parties!

Second, the filter “Patterns / Crystal background” overlays several randomly colored polygons in order to synthesize a texture that vaguely looks like a crystal seen under a microscope. Pretty useful to quickly render colored image backgrounds.

Fig.5.4.4: Filter “Crystal background” in action.

And to end this long overview of new G’MIC filters developed since last year, let us mention “Rendering / Barnsley fern“. This filter renders the well-known Barnsley fern fractal. For curious people, note that the related algorithm is available on Rosetta Code, with even a code version written in the G’MIC script language, namely:

6. Overall project improvements

All filters presented throughout this article constitute only the visible part of the G’MIC iceberg. They are in fact the result of many developments and improvements made “under the hood”, i.e., directly on the code of the G’MICscript language interpreter. This interpreter defines the basic language used to write all G’MIC filters and commands available to users. Over the past year, a lot of work has been done to improve the performances and the capabilities of this interpreter:

The mathematical expressions evaluator has been considerably enriched and optimized, with more functions available (especially for matrix calculus), the support of strings, the introduction of const variables for faster evaluation, the ability to write variadic macros, to allocate dynamic buffers, and so on.

New optimizations have been also introduced in the CImg library, including the parallelization of new functions (via the use of OpenMP). This C++ library provides the implementations of the “critical” image processing algorithms and its optimization has a direct impact on the performance of G’MIC (in this respect, note that CImg is also released with a major version 2.0).

Compiling G’MIC on Windows now uses a more recent version of g++ (6.2 rather than 4.5), with the help of Sylvie Alexandre. This has actually a huge impact on the performances of the compiled executables: some filters run up to 60 times faster than with the previous binaries (this is the case for example, with the Deformations / Conformal Maps filter, discussed in section 5.2).

The support of large .tiff images (format BigTIFF, with files that can be larger than 4Gb) is now enabled (read and write), as it is for 64-bit floating-point TIFF images

The 3D rendering engine built into G’MIC has also been slightly improved, with the support for bump mapping. No filter currently uses this feature, but we never know, and prepare ourselves for the future!

Fig.6.1: Comparison of 3D textured rendering with (right) and without “Bump mapping” (left).

And as it is always good to relax after a hard day’s work, we added the game of Connect Four to G’MIC :). It can be launched via the shell command $ gmic -x_connect4 or via the plug-in filter “Various / Games & demos / Connect-4“. Note that it is even possible to play against the computer, which has a decent but not unbeatable skill (the very simple AI uses the Minimax algorithm with a two-level decision tree).

Fig.6.2: The game of “Connect Four“, as playable in G’MIC.

Finally, let us mention the undergoing redesign work of the G’MIC Online web service, with a beta version already available for testing. This re-development of the site, done by Christophe Couronne and Véronique Robert (both members of the GREYC laboratory), has been designed to better adapt to mobile devices. The first tests are more than encouraging. Feel free to experiment and share your impressions!

7. What to remember?

First, the version 2.0 of G’MIC is clearly an important step in the project life, and the recent improvements are promising for the future developments. It seems that the number of users are increasing (and they are apparently satisfied!), and we hope that this will encourage open-source software developers to integrate our new G’MIC-Qt interface as a plug-in for their own software. In particular, we are hopeful to see the new G’MIC in action under Krita soon, this would be already a great step!

Second, G’MIC continues to be an active project, and evolve through meetings and discussions with members of artists and photographers communities (particularly those who populate the forums and IRC of pixls.us and GimpChat). You will likely able to find us there if you need more information, or just if you want to discuss things related to (open-source) image processing.

And while waiting for a future hypothetical article about a future release of G’MIC, you can always follow the day-after-day progress of the project via our Twitter feed.

Until then, long live open-source image processing!

Credit: Unless explicitly stated, the various non-synthetic images that illustrate this post come from Pixabay.

A few days ago, I released a new version of G’MIC (GREYC’s Magic for Image Computing, numbered 1.6.2.0). This is a free and generic image processing framework I’ve been developing since 2008. For this particular occasion, I thought it would be good (for me) to write a quick summary of what features I’ve been working on these last ten months (since the last release of G’MIC in the 1.5.x.x branch). Hopefully, you’ll be interested too ! This project takes a large amount of my free time (basically every week-end and evenings, except Wednesday, table tennis time 🙂 ), so it’s probably good for me to take a little break and analyze what has been done recently around the G’MIC project.

I will summarize the important features added to G’MIC recently, which are: Various color-related filters, a foreground extraction algorithm, new artistic filters, and other (more technical) cool features. As you will see this quick summary has ended up a not-so-quick summary, so you are warned. Just before I start, I’d like to thank Patrick David again a thousand times for his proof reading. Kudos Pat! And now, let’s start !

1. The G’MIC Project : Context and Presentation

The G’MIC project was born in August 2008, in the IMAGE Team of the GREYClaboratory (a public research unit, located in Caen / France, affiliated to the CNRS, the largest public research institute in France), a team to which I belong. Since then, I’ve developed it apace. G’MIC is an open-source software, distributed under the CeCILL license (a French GPL-compatible license).

Fig.1.1 Mascot and logo of the G’MIC project, an open-source framework for image processing.

What G’MIC proposes is basically several different user interfaces to manipulate generic image data (sequences or still 2D/3D images, with an arbitrary number of channels, float-valued , etc., so definitely not only 2D color images). To date, the most popular G’MIC interface is its plug-in for GIMP.

Fig.1.2. Overview of the G’MIC plug-in for GIMP.

But G’MIC also provides a command-line interface(similar to what ImageMagick has, but more flexible, perhaps?), and a web interface G’MIC Online. Other ways of using G’MIC exist, such as ZArt, a qt-based image stream processor, or a nice plug-in for Krita(developed by Lukas Tvrdy), or through some features implemented in the recent (and nice!) software Photoflow. These latter interfaces are still a bit more confidential, but not for long I guess. Actually, all of these interfaces are based on libgmic. This is the C++, portable, thread-safe and multi-threaded library (in particular through the use of OpenMP) which implements all of the G’MIC core image processing routines and structures. It also embeds its own script language interpreter, so that advanced users are allowed to write and integrate their customized image processing functions into G’MIC easily. Today, G’MIC already understands more than 900 different commands, all configurable, for a libgmic library file that takes a bit less than 5 MB. The available commands cover a wide range of the image processing field, with algorithms for the manipulation of image geometry and colors, for image filtering (denoising, sharpening with spectral, variational or non-local methods, …), for image registration and motion estimation, for the drawing of graphical primitives, for image segmentation and contour extraction, for 3D object rendering, artistic filters, and so on…

The set of tools G’MIC provides is really useful, for both converting, visualizing and exploring image data, and for creating, and applying complex image processing pipelines. In my opinion, this is one piece of free software that image processing savvy folks should have tried at least once. At least for me, there is an everyday use. 🙂

2. New G’MIC features for color processing

So, here is some of the stuff I’ve added to G’MIC recently, related to the manipulation (creation of modification) of the colors in an image.

2.1. Color curves in an arbitrary color space

The concept of color curves is well known for any artist of photograph who wish to modify the colors of their images. Basically, a color curve tool allows you to define and apply on each R,G,B component of your image, a 1d continuous function f:[0,255] -> [0,255]. Each original red, green or blue value x of a pixel (assumed to be in the range 0..255) is then mapped into a new value f(x) (in 0..255 too). To define these transfer functions (one for each of the R,G,B components), the user classically defines a set of key-points that are interpolated by spline functions.

But what happens if you want to define and apply color curves in a color space different than RGB ? Well… mostly nothing, as most of the image retouching tools (GIMP included) only propose to color curves for RGB, unfortunately.

So, one of the things I’ve done recently in G’MIC has been to implement an interactive color curves filter that works in other color spaces than RGB, that is, CMY, CMYK, HSI, HSL, HSV, Lab, Lch and YCbCr. The new command -x_color_curves is in charge of doing that (using the CLI interface). Plug-in users may use it via the filter Colors / Curves [interactive]. Here, using the plug-in is even nicer because you can easily save your favorite curves, as new Favorite filters, and apply the same color curves again and again on hundred of images afterwards. Here is how it looks:

Fig.2.1. Interactive definition of color curves in the Lab color space and application on a color image. On the left, the three user-defined curves for each of the components L (lightness), a and b (chrominances). On the right, the corresponding color transformation it creates on the color image.

I’ve also recorded a quick video to show the action of this filter live, using it from the G’MIC plug-in for GIMP:

2.2. Comics colorization

I also had the chance to talk with David Revoy, a talented French illustrator who is the man behind the nice Pepper & Carrot webcomics (among other great things). He told me that the process of comics colorization is a really boring and tedious task, even using a computer, as he explains it on his blog. He also told me that there were some helpers for colorizing comics, but all proprietary software. Nice! There was obviously something worth trying for open-source graphics here.

So I took my keyboard and started coding a color interpolation algorithm that I hoped to be smart enough for this task. The idea is very simple: Instead of forcing the artist to do all the colorization job by himself, we just ask him to put some colored key-points here and here, inside the different image regions to fill-in. Then, the algorithm tries to guess a probable colorization of the drawing, by analyzing the contours in the image and by interpolating the given colored key-points with respect to these contours. Revoy and I also discussed with Timothée Giet (on irc, channel #krita :)), another French artist, about what could be a nice “minimal” interface to make the process of putting these colored key-points comfortable enough for the artist. And finally, the result of these discussion is the new G’MIC command -x_colorize and the corresponding filter Black & White / Colorize [interactive] in the G’MIC plug-in. Apparently, this seemed to be fair enough to do the job!

From what I’ve seen, this has raised quite a bit of interest among the users of Krita (not so surprising as there are many comics creators among them!). Maybe I’m wrong, but I feel this is one of the reasons the Krita team decided to put some additional efforts on their own G’MIC plug-in (which is something to become an awesome piece of code IMHO, probably even better than the current G’MIC plug-in for GIMP I’ve done).

Here is a quick illustration on how this works for real. The images below are borrowed from the David Revoy‘s web site. He has already written very nice articles about this particular feature, so consider reading them for more details about it.

Fig.2.3. Using G’MIC for Comics colorization. Step 2 : Place some colored key-points and let the colorization algorithm interpolate the colors for you (image borrowed from the David Revoy web site)

I believe Revoy is using this colorizing algorithm in a regular manner for his creations (in particular for Pepper & Carrot), as illustrated in Fig.2.2-2.3. One can clearly see the different colored key-points put on the image and the outcome of the algorithm, all this being done under the G’MIC plug-in for Krita. After this first colorization step, the artist may add shadows and lights on the flat color regions generated by the algorithm. The nice thing is he can easily work on each color region separately because it is easy to select exactly one single flat color in the output color layer. And for those who prefer to work on more layers, the algorithm is able to output multiple layers too, one for each single reconstructed color region.

So, again, thanks a lot to David Revoy and Timothée Giet for the fruitful discussions and their feedback. We have now a very cool Comics colorization tool in a free software. This is the kind of academic / artistic collaboration I’m fond of.

Below you can see a small video I’ve made that shows how this filter runs inside the G’MIC plug-in for GIMP :

2.3. B&W picture colorization

But could this also work for old B&W photographs ? The answer is Yes! By slightly modifying the colorization algorithm, I’ve been able to propose a way to reconstruct the color chrominance of a B&W image, exactly the same way, from user-defined colored key-points. This is illustrated in the figure below, with the colorization of an old B&W portrait. The pixel-by-pixel content of a photograph being obviously more detailed than a Comics, we often need to put more key-points to get an acceptable result. So, I’m not sure this speeds up the colorization process so much in this case, but still, this is cool stuff. These two colorization techniques are available from the same filter, namely Black & white / Colorize [interactive].

Fig.2.4. Colorization of an old B&W photograh with the G’MIC colorization algorithm.(Ugh, old man, sorry for the improbable color of you hairs, which makes you look like Kim Kardashian…)

2.4. Color transfer

Here, what I call « color transfer » is the process of modifying the colors of an image A by replacing them by the colors of another image B (reference image) so that the modified image A’ gets the same “color ambiance” than the reference imageB, and this is being done in a fully automatic way. This is an ill-posed problem, quite complex to solve, and there is already a lot of scientific papers published on this topic (like this one for instance).The main issue consists in generating a new image A’ that keeps a “natural” aspect, without creating synthetic-looking flat color areas, or on the contrary high color discontinuities that are not initially present in the original image A. In short, this is far from being trivial to solve.

I’ve worked on this topic recently (with one of my colleagues Julien Rabin), and we’ve designed a pretty cool algorithm for transferring colors from an image to another. It’s not perfect of course, but this is really a good start. I’ve implemented the algorithm and put it in G’MIC, available from the plug-in filter Colors / Transfer colors [advanced]. This filter requires several input layers, one being the reference image B containing the colors to transfer to the other layers. Here is how it looks when run from the G’MIC plug-in for GIMP, with the original color image (on the left in the preview window), the reference image (bottom-left) and the filter outcome (on the right).

Fig.2.5. Overview of the new Color transfer filter in the G’MIC plug-in for GIMP.

Sometimes, this filter gives awesome results! Two color transfer examples are illustrated below. Of course, don’t expect it to work nicely with pathological cases (as transferring a colorful image to a monochrome one for instance). But it’s not lying to say it works already quite well. And the best of all, it is quite fast to render.

Fig.2.6. Two different color transfer examples from a reference picture (middle) to a color image (left). The images on the right give the outcome of the G’MIC color transfer algorithm.

I’ve also recorded a quick video tutorial that shows how this kind of results can be obtained easily from the G’MIC plug-in for GIMP. We can also think of other usages for this interesting new algorithm, like the homogenization of colors between successive frames of a video, or ensuring the color stability of a stereoscopic pair for instance. Here is the video :

2.5. Website for analog film emulation

I have the pleasure of being a friend with Pat David, an american photograph who is very active in the libre graphics community, sharing his experience and tutorials on his must-read blog. I met him at LGM’2014 in Leipzig, and honestly, many useful G’MIC filters have been suggested or improved by him. Particularly, all the filters in the Film emulation category could not have been done without his work on the design of color transfer functions (a.k.a. CLUTs). Below is a figure that shows a small subset of the 300+ color transformations concocted by Pat that has been made available in G’MIC.

Fig.2.7. Overview of some analog film emulation outcomes, available in G’MIC.(This image comes from the Pat David‘s blog)

Once added (two years ago), these filters raised much interest in the photo retouching community, and have been re-implemented in RawTherapee. To make these filters even more accessible, we have recently set up a web page dedicated to Film Emulation, where you can see/compare all the available presets and download the corresponding CLUT files. Note that you can also find the G’MIC Film Emulation filters in the G’MIC Online web site, and try them live directly in your web browser. Here again, analog film emulation was somehow a missing feature in the libre software world and the G’MIC infrastructure made it easy to fill this gap (who needs DXO FilmPack anymore now ? ;)).

3. An algorithm for foreground/background extraction.

In photo retouching, it is not uncommon to differentiate the processing of foreground and background objects. So one often needs to select the foreground object first (using the “lasso” tool for instance) before doing anything else. For objects with a complex geometry, this is a tedious task. G’MIC now integrates a user-guided segmentation algorithm to speed up this foreground extraction work. The G’MIC command -x_segment (using the CLI interface) and the plug-in filter Contours / Extract foreground [interactive] let the user play with this algorithm.

This works exactly the same way as the Comics colorization filter, except that the colored key-points are replaced by labelled key-points, with labels being either “foreground” or “background”. Then, the extraction algorithm interpolates those labels all inside the image, taking care of the contours, and deduce a foreground/background binary map. Thus, the filter is able to decompose an image into two layers, one with the foreground pixels only, and the other with the remaining pixels (background). The figure below shows a typical use of this filter : Starting from a color image of a flower (top-left), the user put few keypoints on it (top-right) and the extraction algorithm is able to generate the two background / foreground layers on the last row, from this very sparse set of points.

Doing this takes only few seconds, while the same operation made manually would have been much much longer to perform (the flower’s contour being not simple to cut).

After that, it is quite easy to process the background and foreground separately. Here for instance, I’ve just changed the hue and saturation of the foreground layer a little bit, in order to get a more purple flower without changing the colors of the background. Selecting objects in images is a common operation in photo retouching, I’ll let you imagine in how many cases this filter can be useful.

Fig.3.2. The same image, after changes on the hue and saturation applied on the extracted foreground only.

I’ve done another G’MICvideo tutorial to illustrate the whole process in real-time, using the G’MIC plug-in for GIMP:

4. Some new artistic filters.

G’MIC has always been a source of dozens of artistic filters. Below is an overview of some of the newest additions.

4.1. Engrave effect.

The filter Black & White / Engrave tries to transform an image into an etching. The high number of parameters lets you precisely tune the rendering done by the algorithm, and many different effects may be obtained with this single filter.

Fig.4.1. Overview of the « Engrave » filter, available in the G’MIC plug-in for GIMP.

What I find particularly interesting with this filter is the ability it has (with properly chosen parameters) to convert photographs into comics-like renderings. This is illustrated with the two images below.

Fig.4.2. Photo to Comics conversion, using the Engrave filter from G’MIC.

Fig.4.3. Another example of photo to Comics conversion, with the Engrave filter from G’MIC.

I’ve also recorded a video tutorial to show the different steps used to generate this kind of results. It takes only few seconds to achieve (maybe minutes for the slowest of you!).

4.2. Delaunay triangulation.

The algorithm of Delaunay triangulation has been added to G’MIC (through the -delaunay3d command). A new filter, namely Artistic / Polygonize [delaunay] inside the plug-in for GIMP usesit to transform color images into nice geometric abstraction. Each generated triangle color can be random, constant or related to the pixels under the triangle. The triangulation tries to stick to the image contours (more or less, depending on the chosen parameters).

Fig.4.4. Overview of the Polygone [delaunay] filter, and application on a color image to simulate a stained glass effect.

Applying this filter on image sequences gives also nice outcomes. I wonder if importing these triangulated images into Blender could be of any interest (even better, a full G’MIC plug-in for Blender, maybe ? :p )

Fig.4.5. Applying the Delaunay triangulation on an image sequence.

4.3. Other artistic filters.

As you can see, G’MIC is a very active project, and the number of available artistic filters increases every day. Thus, I can’t describe in details all the new filters added these ten last months, and because a picture is worth a thousand words, you will find below a quick overview of some of them.

Fig.4.10. Overview of the filter Sequences / Spatial transition, which takes several layers as input, and generates an image sequence corresponding to a custom transition between each consecutive layers.

5. A quick view of the other improvements

Of course, the few new filters I’ve presented above are only a small part of the overall work that has been done on G’MIC (the most visible part). Here is a list of other notable (mostly technical) improvements:

5.1. Global improvements and the libgmic library.

A lot of work has been done to clean and optimize the entire source code. The size of the libgmic library has been drastically reduced (currently less than 5 MB), with an improved C++ API. Using some of the C++11 features (rvalue-references) allows me to avoid temporary buffer copies which is really nice in the context of image processing as the allocated memory may become huge quickly. With the help of Lukas Tvrdy, one of the Krita developers, we have been able to improve the compilation of the project a little bit on Windows (with Visual Studio), and run various code analysis tools to detect and fix memory leaks and strange behavior. Many sessions of valgrind, gprof, g++ with the -fsanitize=address option, PVS Studio has successfully led to effective code improvements. Not very visible or exciting for me, but probably very satisfying for the final user 🙂 As a consequence, I’ve decided the code was clean enough to regularly propose G’MIC pre-releases that are considered as stable enough while getting the latest features developed. The method of releasing G’MIC is now closer to a Rolling Release scheme.

The compilation of the default G’MIC binaries on Windows has been improved. It now uses the latest g++-4.9.2 / MinGW as the default compiler, and new installers for Windows 32 bits / 64 bits are now available.

G’MIC has new commands to compress/uncompress arbitrary data on fly (through the use of the zlib). So now, the embedded G’MIC commands are stored in a compressed form and the Internet updates are now faster.

The G’MIC project got its own domain name http://gmic.eu, independent from Sourceforge. We have improved the web pages a lot, particularly thanks to the wonderful tutorials on the use of the G’MIC command-line tool, written by Garry Osgood (again, thanks Garry!). This is clearly a page to bookmark for people who slowly want to learn the basics of G’MIC.

5.2. New possibilities for image filtering.

A bunch of new commands dedicated to image filtering, and their associated filters in the G’MIC plug-in for GIMP have been added, for image sharpening (Mighty Details), deconvolution by an arbitrary kernel (Richardson-Lucyalgorithm), guided filtering, Fast Non-Local Means, Perona-Malik anisotropic diffusion, box filtering, DCT transforms, and so on. Honestly, G’MIC now implements so many different image filtering techniques that it is a must-have for any image processing person (even only for the denoising techniques it provides). For the users of the plug-in, this means potentially more diverse filters in the future. The figure below shows an application of the Mighty Details filter on a portrait, to simulate a kind of Dragan effect.

Fig.5.1 Application of the Mighty Details filter on a portrait to enhance details.

5.3. Improvements of the G’MIC plug-in for GIMP.

The plug-in for GIMP being the most used interface of the G’MIC project, it was necessary to make a lot of improvements:

The plug-in now has an automatic update system which ensures you always get the latest new filters and bug fixes. At the date of the 1.6.2.0 release, the plug-in counts 430 different filters, plus 209 filters still considered in development (in the Testing/ category). For a plug-in taking 5.5 Mb on the disk, let us say this makes an awesome ratio of number of filters / disk space. A complete list of the filters available in the plug-in for GIMP can be seen here.

During the execution of a filter that takes a bit of time, the plug-in now displays information about the elapsed time and the used memory in the progress bar.

Fig.5.2. Display of the used resources in the progress bar when running a G’MIC filter in GIMP.

The plug-in GUI has been slightly improved: the preview window is more accurate when dealing with multi-layer filters. The filters of the plug-in can be interactive now: they can open their own display window, and handle user events. A filter can also decide to modify the value of its parameters. All of this allowed me to code the new interactive filters presented below (as the one for the Comics colorization, for the color curves in an arbitrary color space, or for the interactive foreground extraction). This is something that was not possible before, except when using the command-line interface of G’MIC.

A G’MIC filter now has knowledge about all the input layer properties, such as their position, their opacities and their blending mode. A filter can modify these properties for the output layers too.

A new filter About / User satisfaction survey has been added. You can give some anonymous information about your use of G’MIC. The result of this survey is visible here (and updated in almost real-time). Of course, this result image is generated by a G’MIC command itself :).

5.4. G’MIC without any limits : Using the command-line interface.

gmic is the command-line interface of G’MIC, usable from a shell. That is undoubtly the most powerful G’MIC interface, without any limitations inherent to the input/output constraints of the other interfaces (like being limited to 2D images, 8bits / channels, 4 channels max on GIMP, or a single input image in the G’MIC Online site, etc.). With the CLI interface, you can load/save/manipulate sequences of 3D volumetric images, with an arbitrary number of channels, float-valued, without limits other than the available memory. The CLI interface of G’MIC has also gotten a lot of improvements:

It now uses by default OpenCVto load/save image sequences. We can then apply image processing algorithms frame by frame on video files (command -apply_video), on webcam streams (command -apply_camera), or generate video files from still images for instance. New commands -video2files and -files2video have been added to easily decompose/recompose video files into/from several frames. Processing video files is almost childs play with G’MIC now. The G’MIC manual (command -help) has been also improved, with a colored output, proposed corrections in case of typos, links to tutorials when available, etc..

Fig.5.3. Overview of the help command with the command line interface gmic.

The invokation of gmic on the command line without any arguments makes it enter a demo mode, where one can select between a lot of different small interactive animations from a menu, to get an idea of what G’MIC is capable of. A good occasion to play Pacman, Tetris or the minesweeper on the pretext to test a new serious image processing software!

Fig.5.4. Overview of the demo mode of G’MIC.

5.5. Other G’MIC interfaces being developed.

ZArt is another available G’MIC interface initially developed as a demonstration platform, for processing images from the webcam. We use it classically for public exhibitions like the “Science Festival” we have once a year in France. It is very convenient to illustrate what image processing are, and what they can do, for the general public. ZArt has been significantly improved. First, it is now able to import any kind of video files (instead of supporting only webcam streams). Second, most of the filters from the GIMP plug-in has been imported to ZArt. This video shows these new possibilities, for instance with real-time emulation of analog films of a video stream. This could be really interesting to get a G’MIC plug-in working for video editing software, as the brand new Natron for instance. I’m already in touch with some of the Natron developers, so maybe this is something doable in the future. Note also that the code of ZArt have been upgraded to be compatible with the newest Qt 5.

Fig.5.5. Overview of ZArt, a G’MIC-based video processing software.

It has been a long time now since Lukas Tvrdy, member of the Krita devs, has started to develop a G’MIC plug-in for Krita. This plug-in is definitely promising. It contains all of the elements of the current G’MIC plug-in for GIMP, and it already works quite well on GNU/Linux. We are discussing to try solving some problems remaining for the compilation of the plug-in on Windows. Using the Krita plug-in is really nice because it is able to generate 16bits/channel images, or even more (32bits float-valued), whereas the plug-in for GIMP is currently limited to 8bits/channel for the plug-in API. This may change in the future with GEGL, but right now, there is not enough documentation available to ease the support of GEGL for the G’MIC plug-in. So now, if you want to process 16bits/channel images, you can do this only through the command-line tool gmic, or the G’MIC plug-in for Krita.

Fig.5.6. Overview of the G’MIC plug-in running on Krita.(This image has been borrowed from the Krita website : https://krita.org/wp-content/uploads/2014/11/gmic-preview.png)

The integration of some of the G’MIC image processing algorithms has also begun in PhotoFlow. This is a quite recent and promising software which focuses on the development of RAW images into JPEG, as well as on photo retouching. This is a project done by Carmelo DrRaw, check out this project, it is really nice.

Fig.5.7. Overview of the Dream Smoothing filter from G’MIC, included in PhotoFlow.(This image has been borrowed from the PhotoFlow website : http://photoflowblog.blogspot.fr/2014/12/news-gmic-dream-smoothing-filter-added.html)

6. Perspectives and Conclusions.

Here we are! That was a quite complete summary of what happened the ten last months around the G’MIC project. All this has been made possible also thanks to many different contributors (coders, artists, and users in general), whose number is increasing everyday. Thanks again to them!

In the future, I still dream of more G’MIC integration in other open-source software. A plug-in for Natron could be really great to explore the possibilities of video processing with G’MIC. A plug-in for Blender would be really great too! Having a G’MIC-based GEGL node is again a great idea, for the upcoming GIMP 3.0, but we have probably a few decades to decide 🙂 If you feel you can help in these topics, feel free to contact me. One thing I’m sure: I won’t be bored by G’MIC in the next few years!

Having said that, I think it’s time to go back working on G’MIC again. See you next time, for other exciting news about the G’MIC project! (and if you appreciate what we are doing or just the time spent, you are free to send us a nice postcard from your place, or give a few bucks for the hot chocolates I need to keep my mind clear working on G’MIC, through the donation page). Thank you!